JP4698351B2 - Vehicle side confirmation device - Google Patents

Description

  The present invention relates to a vehicle side confirmation device that images a scene in a blind spot range of a side mirror installed on a vehicle door and displays the image on a display means inside the vehicle.

  As a conventional vehicle side confirmation device, a hologram is installed on the side glass and outside the position where the driver can see the side mirror. An apparatus that displays a field of view that can be visually recognized by a side mirror and a video field of view displayed on the hologram as a continuous field of view by projecting onto the hologram is disclosed (for example, see Patent Document 1).

  However, in this conventional vehicle side confirmation device, since the image is projected onto the hologram installed on the side glass, there is a problem that the driver cannot open the side glass. Furthermore, there is a high possibility that the scene on the other side of the hologram will appear superimposed on the image, which may lead to misrecognition by the driver, and there is also a risk that the projected part may become a new blind spot. There was a problem in terms of.

  On the other hand, as another conventional vehicle side confirmation device, a device that displays a captured image including a scene in a blind spot range on a display device in addition to a scene in a range that can be visually recognized by a side mirror is disclosed (for example, (See Patent Document 2).

  However, in this conventional vehicle side confirmation device, when the display device is installed in the meter panel or in the center part, it is difficult for the driver to intuitively understand even when looking at the vehicle side image, which gives anxiety. On the other hand, even when the display device is installed on both sides of the vehicle, the field of view reflected in the side mirror and the part of the image field of the display device overlap, and the driver can see two identical fields of view. There was a problem in terms of safety that could cause confusion.

  Therefore, it is conceivable that the viewing means is limited to the video field of view by the display device, and the side mirror is eliminated, but a psychological security that the side mirror can be used as it is can be obtained, or when the imaging device is at fault For this reason, it is desirable from the viewpoint of safety that a side mirror exists.

  Also, as another conventional vehicle side confirmation device, an image corresponding to a field of view in which the driver views the image through the side mirror by determining the position of the driver's eyes based on the position of the seat and capturing the image captured by the imaging device. An apparatus that converts the information into a display device and displays it on a display device is disclosed (for example, see Patent Document 2).

However, if the position of the driver's eyes is determined based on the position of the seat, the position of the eyes changes depending on the height of the driver, the posture at the time of sitting, etc. There has been a problem that there is a possibility that a deviation may occur between the field of view visible by the mirror and the image field of view displayed on the display device.
JP 2000-71877 A JP 2003-339044 A

  In view of the above problems, the present invention provides a side mirror that is set to an angle so that a driver can visually recognize a scene in a range that is generally a blind spot in a side mirror installed on a vehicle door, and a side mirror that is generally visually recognized. An imaging unit that captures a possible range of scenes, a display unit that is installed in the vicinity of the side mirror and inside the vehicle, and a field of view in which the video field displayed on the display unit is reflected on the side mirror with respect to the captured image on the side of the vehicle And image processing means for performing image processing so as to be continuous, it is possible for the driver to grasp the situation on the side of the vehicle without a sense of incongruity, and thus contributes to improving safety during driving An object is to provide a lateral confirmation device.

  The vehicle side confirmation device according to claim 1 of the present invention includes a side mirror that is set at an angle so that a driver can visually recognize a predetermined range of scenes outside the vehicle side, and is installed on a vehicle door, and the side mirror. Imaging means for imaging a scene in a predetermined range including a blind spot range on the vehicle side, display means installed beside the side mirror and inside the vehicle, and the display means for the captured image captured by the imaging means Image processing means for performing image processing so that the video field of view displayed on the side mirror is continuous with the field of view reflected on the side mirror, and displaying the video after the image processing on the display unit .

  A vehicle side confirmation device according to claim 2 of the present invention is the vehicle side confirmation device according to claim 1, wherein the angle detection means for detecting the angle of the side mirror and the position of the driver's eyes are provided. Position detection means for detecting the position, and the image processing means includes information on the angle of the side mirror detected by the angle detection means and information on the position of the driver's eyes detected by the position detection means. Using the information of the cutout position to calculate a cutout position for cutting out the partial video corresponding to the video view that is continuous with the range of the visual field from the captured video. Then, predetermined image processing is performed on the captured image.

  The vehicle side confirmation device according to claim 3 of the present invention is the vehicle side confirmation device according to claim 2, wherein the angle detection means is an angle of the side mirror with respect to the vertical direction and the horizontal direction of the vehicle. Is detected.

  The vehicle side confirmation device according to claim 4 of the present invention is the vehicle side confirmation device according to any one of claim 2 or 3, wherein the position detection stage detects the position of the driver's eyes. There are two cameras for detection, and using the captured images of the two cameras, the position of the driver's eyes with respect to the front-rear direction and the vertical direction of the vehicle, or the front-rear direction, the vertical direction, and the left-right direction of the vehicle The position of the driver's eyes is detected.

  A vehicle side confirmation device according to claim 5 of the present invention is the vehicle side confirmation device according to any one of claims 2 to 4, comprising recording means for recording information of a cut-out position, The image processing means calculates the cutout position at regular intervals, updates the information recorded in the recording means, and performs predetermined image processing on the captured video using the information recorded in the recording means It is characterized by that.

  A vehicle side confirmation device according to claim 6 of the present invention is the vehicle side confirmation device according to any one of claims 2 to 4, comprising recording means for recording information of a cut-out position, The image processing means calculates a cut-out position at the start of operation, records the cut-out position in the recording means, and performs predetermined image processing on the captured image using information recorded in the recording means.

  A vehicle side confirmation device according to claim 7 of the present invention is the vehicle side confirmation device according to any one of claims 2 to 6, wherein the parameter of the cut-out position calculated by the image processing means is used. The image processing unit further includes a setting unit that inputs setting information for adjustment, and the image processing unit further has a function of converting a parameter of the cutout position based on the setting information. And performing predetermined image processing on the captured video.

  Further, the vehicle side confirmation device according to claim 8 of the present invention is the vehicle side confirmation device according to claim 1, wherein the angle detection means for detecting the angle of the side mirror and the position of the driver's eyes are provided. Position detection means for detecting the image, and the position information of the cut-out position for cutting out the partial video corresponding to the video field of view that is continuous with the range of the visual field reflected on the side mirror from the captured video, the angle of the side mirror and the driving Recording means for recording in correspondence with the pattern of combinations of the positions of the eyes of the person, and the image processing means detects the angle information of the side mirror detected by the angle detection means and the position detection means. The pattern of the cutout position corresponding to the combination of the information on the driver's eye position is retrieved from the pattern information of the cutout position recorded in the recording means, and this Using the information of the cutout position search performs predetermined image processing on the captured image, characterized in that.

  The vehicle side confirmation device according to claim 9 of the present invention is the vehicle side confirmation device according to claim 8, wherein the angle detection means includes an angle of the side mirror with respect to a vertical direction and a horizontal direction of the vehicle. Is detected.

  The vehicle side confirmation device according to claim 10 of the present invention is the vehicle side confirmation device according to any one of claim 8 or 9, wherein the position detection stage detects the position of the driver's eyes. There are two cameras for detection, and using the captured images of the two cameras, the position of the driver's eyes with respect to the vehicle front-rear direction and the vertical direction, or the vehicle front-rear direction, vertical direction, and left-right direction The position of the driver's eyes is detected.

  The vehicle side confirmation device according to claim 11 of the present invention is the vehicle side confirmation device according to any one of claims 8 to 10, wherein the recording means is searched by the image processing means. The search information of the cutout position is recorded, and the image processing means searches the pattern of the cutout position at regular time intervals to update the search information recorded in the recording means, and the search recorded in the recording means Predetermined image processing is performed on the captured video using information.

  A vehicle side confirmation device according to claim 12 of the present invention is the vehicle side confirmation device according to any one of claims 8 to 10, wherein the recording means is searched by the image processing means. The search information of the cutout position is recorded, and the image processing unit searches for the pattern of the cutout position at the start of operation, records it as search information in the recording unit, and uses the search information recorded in the recording unit to A predetermined image processing is performed on the captured video.

  A vehicle side confirmation device according to claim 13 of the present invention is the vehicle side confirmation device according to any one of claims 8 to 12, wherein the parameter of the cut-out position searched by the image processing means is used. The image processing unit further includes a setting unit that inputs setting information for adjustment, and the image processing unit further has a function of converting a parameter of the cutout position based on the setting information. And performing predetermined image processing on the captured video.

  The vehicle side confirmation device according to claim 14 of the present invention is the vehicle side confirmation device according to any one of claims 2 to 13, wherein the display means is located above and below the side mirror. The width of the direction is large, and the image processing means calculates the amount of distortion caused by the angle difference between the display means and the side mirror using the information on the angle of the side mirror and the information on the position of the eyes of the driver. The method further includes a function of correcting a partial image cut out from the captured image based on the distortion amount.

  A vehicle side confirmation device according to claim 15 of the present invention is the vehicle side confirmation device according to any one of claims 2 to 13, wherein the angle of the side mirror is measured via the image processing means. The information processing device further includes a moving unit that acquires information and information on a driver's eye position and adjusts the angle and position of the display unit using the information.

  A vehicle side confirmation device according to claim 16 of the present invention is the vehicle side confirmation device according to any one of claims 1 to 15, wherein the imaging means is an infrared camera. To do.

  ADVANTAGE OF THE INVENTION According to this invention, it is possible to make a driver | operator grasp | ascertain the situation of the vehicle side without a sense of incongruity, and can contribute to the improvement of the safety | security at the time of a driving | operation by extension. In other words, the left-hand scene is displayed by the left-hand display means, and the right-hand scene is displayed by the right-hand display means. Can be suppressed. In addition, since the video field of view displayed on the display unit and the field of view reflected on the side mirror are obtained as a continuous field of view, the driver can be prevented from feeling uncomfortable. In addition, since there is no overlapping portion between the field of view reflected on the side mirror and the video field of view, anxiety can be suppressed.

  In addition, since a part of the side mirror is used for visually recognizing the scene on the side of the vehicle, the range that the imaging unit must capture can be limited, and the number of imaging units can be reduced. Further, as for the display means, only the range that cannot be visually recognized by the side mirror need be displayed, and the display means can be used effectively. Even in rainy weather, the video field of view does not become difficult to see due to raindrops on the side mirrors and side glass.

  Further, by acquiring information on the angle of the side mirror with respect to the vertical direction and the horizontal direction of the vehicle, the image processing means recognizes the range visually recognized by the driver via the side mirror according to the angle of the side mirror, and Can be displayed more accurately so as to be continuous with the field of view reflected on the side mirror, so that the driver can grasp the scene on the side of the vehicle more comfortably.

  In addition, the image processing means obtains information on the position of the driver's eyes with respect to the longitudinal direction and vertical direction of the vehicle, or with respect to the longitudinal direction, vertical direction, and lateral direction of the vehicle. The position of the eyes that change depending on the vehicle's position can be recognized, and the video field of view can be displayed more accurately so that it is continuous with the field of view reflected in the side mirror, so that the driver can grasp the scene on the side of the vehicle more comfortably. .

  In addition, the driver can keep track of the scene on the side of the vehicle without any discomfort because the video field of view can be adjusted to the field of view reflected on the side mirror in real time following the driver's change of seat position and posture. can do.

  In addition, by calculating or searching the cut-out position only at the start of operation, the angle detection means and the position detection means need only be operated at the start of operation, and it is not necessary to use these during operation, thus reducing power consumption. be able to.

  In addition, by storing the cutout position pattern corresponding to the combination of the angle of the side mirror and the driver's eye in the recording means in advance, the calculation process of the range of view and the cutout position is omitted, and the processing is performed at a higher speed. It can be performed.

  In addition, by enabling the parameter adjustment of the cutout position, the driver can adjust the range of the video field of view according to his / her preference. A plurality of presets can be registered. Further, when adjusting the parameters of the cutout position, angle detection means and position detection means may be used, and after setting a rough range using these means, it is possible to adjust according to the preference of the driver. .

  In addition, when the display means and the side mirror are different in orientation, when the driver visually recognizes both, it becomes difficult to recognize due to a difference in appearance, but distortion caused by the difference in angle between the display means and the side mirror. By calculating the amount and correcting the partial video, the video field of view displayed on the display means can be corrected according to the field of view reflected on the side mirror, and the driver can recognize both as a continuous field of view. It becomes easy to grasp the side scene. Further, the corrected image can be sufficiently displayed by increasing the display means in the vertical direction of the vehicle.

  Further, by adjusting the angle of the display means, the display means can be set to the same angle as the side mirror, and the driver can visually recognize the video field of view displayed on the display means and the field of view reflected on the side mirror at the same angle. Therefore, it becomes easy to grasp the scene on the side of the vehicle. In addition, by adjusting the position of the display means, a straight line connecting the position of the driver's eyes and the end of the side mirror on the vehicle side and a straight line connecting the position of the driver's eyes and the end of the display means on the side mirror side are obtained. Thus, the driver can grasp the video field of view by the display means and the field of view reflected on the side mirror as a continuous field of view. Further, the image processing unit does not need to correct the distortion of the field of view caused by the angle difference between the display unit and the side mirror.

  In addition, by using an infrared camera, it is possible to check objects that are difficult to discern at night, so that the safety of the driver can be further improved.

Hereinafter, a vehicle side confirmation device according to an embodiment of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a schematic configuration of the vehicle side confirmation apparatus according to the first embodiment. FIG. 2 is a diagram for explaining the imaging range of the imaging device (imaging means) in the vehicle side confirmation device according to the first embodiment and the range of the field of view visible to the driver by the side mirror.

  As shown in FIG. 2, in the first embodiment, the left and right side mirrors 201 and 202 installed on the vehicle door of the vehicle 200 are outside predetermined ranges 211 and 212 that are generally blind spots in the conventional side mirror. The angle is set so that the driver can visually recognize this scene. Conversely, the imaging ranges of the cameras (imaging devices) 101 and 102 that respectively capture the left and right side scenes of the vehicle 200 include ranges 221 and 222 that are generally visible to the driver in a conventional side mirror. That is, the cameras 101 and 102 image a scene in a predetermined range including a blind spot range on the vehicle side of the side mirrors 201 and 202. The cameras 101 and 102 are preferably installed at a position where the side mirror does not appear in the image, for example, near the bottom of the side mirror.

  In FIG. 1, cameras (imaging means) 101 and 102 respectively capture the left and right scenes of the vehicle 200. Note that the three-dimensional position information (three-dimensional coordinates) of the cameras 101 and 102 is known, and the vehicle side confirmation device is held in a predetermined storage area in advance. In addition, by using an infrared camera as the cameras 101 and 102, a good video field of view can be obtained even at night.

  The angle detection units (angle detection means) 111 and 112 are attached to a device (not shown) for adjusting the angle of the side mirrors 201 and 202, and detect the angle of the side mirrors 201 and 202 with respect to the vertical direction and the horizontal direction of the vehicle. To do.

  The position detection unit (position detection means) 120 detects the position of the driver's eyes with respect to the longitudinal direction and the vertical direction of the vehicle, or the position of the driver's eyes with respect to the longitudinal direction, the vertical direction, and the lateral direction of the vehicle. The driver's eye position may be the position of both eyes or the position of one specific eye.

  The image processing unit (image processing means) 130 uses the two-dimensional angle information from the angle detection units 111 and 112 and the two-dimensional or three-dimensional position information (two-dimensional or three-dimensional coordinate information) from the position detection unit 120, The captured images captured by the cameras 101 and 102 are subjected to image processing so that the image field of view displayed on the monitors 141 and 142 is a continuous field of view with the side mirrors 201 and 202 when viewed from the driver. The video after the image processing is displayed on the monitors 141 and 142. Monitors (display means) 141 and 142 display video after image processing by the image processing unit 130.

  FIG. 3 is a diagram illustrating an installation example of the monitor 141 and the position detection cameras 121 and 122 on the driver's seat side. The position detection unit 120 includes two position detection cameras 121 and 122 for detecting the position of the driver's eyes. These two position detection cameras 121 and 122 are arranged in the vicinity of the monitor 141 at a certain distance in the vertical direction of the vehicle. The monitor 141 is installed near the side mirror 201 and inside the vehicle. FIG. 4 is a diagram illustrating an installation example of the monitor 142 on the passenger seat side. Similarly to the monitor 141, the monitor 142 is installed beside the side mirror 202 and inside the vehicle. Note that no position detector is provided on the passenger seat side.

  An upper diagram of FIG. 5 is a schematic diagram of a captured image captured by the position detection camera (upper side) 121, and a lower diagram is a schematic diagram of a captured image captured by the position detection camera (lower side) 122. .

  The position detection unit 120 extracts the eye contour of the driver from the captured images captured by the position detection cameras 121 and 122, and determines the position of the eye contour and the size (area) of the contour in each captured image. Find the eye position.

  That is, first, using the pixel coordinates of the eye contour in each captured image, a direction connecting the position of the position detection camera and the eye position with respect to the direction indicating the center position of the imaging range of the position detection camera is Get about the angle to make.

For example, when the position detection camera captures up to 60 degrees in the vertical direction with 100 pixels vertically (the vertical center of the angle of view is 0 degree), the angle per pixel in the vertical direction is
Angle per pixel in the vertical direction = vertical angle of view / number of constituent pixels in the vertical direction = (60 − (− 60)) degrees / 100 pixels = 1.2 degrees.

  Therefore, when extracting the outline of the eye, an arbitrary point (for example, the center point) inside the outline of the eye (including the outline) is set as the eye position, and the number of pixels in the vertical direction of the captured image is the pixel representing the eye position. By obtaining the pixel, the angle α with respect to the vertical direction of the captured image can be obtained.

For example, if the eye position is the 80th pixel from the bottom,
Angle α = pixel coordinates to be calculated (pixel position corresponding to −60 degrees is 0) × vertical angle per pixel−correction angle = 80th pixel × 1.2 degrees−60 degrees = 36 degrees ( 36 degrees upward)
The eyes are present at a position of 36 degrees upward.

The calculation of the angle β with respect to the left-right direction of the captured image is similarly performed. That is, for example, when the position detection camera captures up to 70 degrees on the left and right with 200 pixels horizontally (the center of the angle of view is 0 degree), the angle per pixel in the left and right direction is
Angle per pixel in the left-right direction = angle of view in the left-right direction / number of pixels in the left-right direction = (70-(-70)) degrees / 200 pixels = 0.7 degrees, and the eye position is the 120th pixel from the left in the case of,
Angle β = pixel coordinates to be calculated (pixel position corresponding to −70 degrees is 0) × angle per pixel in the left-right direction−correction angle = 120th pixel × 0.7 degrees−70 degrees = 14 degrees ( 14 degrees to the right)
And the eyes are at a position of 14 degrees to the right.

  Subsequently, the position of the eye is detected using the angles α and β. FIG. 6 is a diagram for explaining a method of detecting the eye position with respect to the vertical direction of the vehicle. The position of the eyes in the vertical direction of the vehicle is obtained by calculating the distance x shown in FIG.

  In FIG. 6, the angle θ <b> 2 is (90−α <b> 2) using the angle α <b> 2 calculated from the captured image of the position detection camera (lower side) 122. Further, the distance A in the vertical direction of the vehicle between the position detection cameras 121 and 122 is known, and the vehicle side confirmation device is held in advance in a predetermined storage area. The angle θ1 is (90 + α1) using the angle α1 calculated from the image captured by the position detection camera (upper side) 121.

Therefore,
Distance x = Atan θ2 / {tan (180−θ1) −tan θ2}
So,
Eye position with respect to vertical direction of vehicle = position of position detection camera (upper side) 121 with respect to vertical direction of vehicle + distance x
By calculating the position of the eyes, the position of the eyes with respect to the vertical direction of the vehicle can be obtained. However, the three-dimensional positions of the position detection cameras 121 and 122 are known, and the vehicle side confirmation device is held in advance in a predetermined storage area.

  FIG. 7 is a diagram for explaining a method of detecting the position of the eyes with respect to the front-rear direction of the vehicle. The position of the eyes in the front-rear direction of the vehicle is obtained by calculating the distance y shown in FIG. In FIG. 7, the angle α1 is an angle calculated from the captured image of the position detection camera (upper side) 121, and the distance x is obtained when detecting the eye position in the vertical direction of the vehicle and is already known. is there.

Therefore,
Distance y = x x tan α1
So,
Eye position with respect to vehicle longitudinal direction = position of position detection camera (upper side) 121 with respect to vehicle longitudinal direction + distance y
By calculating, the position of the eyes with respect to the front-rear direction of the vehicle can be obtained.

  FIG. 8 is a diagram for explaining a method of detecting the eye position with respect to the left-right direction of the vehicle. The position of the eyes in the left-right direction of the vehicle is obtained by calculating the distance z shown in FIG. In FIG. 8, the angle β1 is an angle calculated from the captured image of the position detection camera (upper side) 121, and the distance y is obtained when detecting the eye position in the front-rear direction of the vehicle and is already known. is there.

Therefore,
Distance z = y / tan α1
So,
Eye position with respect to left and right direction of vehicle = position of position detection camera (upper side) 121 with respect to left and right direction of vehicle + distance z
By calculating the position of the eyes with respect to the left-right direction of the vehicle.

  FIG. 9 shows a flowchart of processing in the image processing unit 130. First, the image processing unit 130 acquires eye position information (two-dimensional or three-dimensional coordinate information) from the position detection unit 120 (step S1). The eye position information may be information on the positions of both eyes or information on the position of one specific eye.

  Next, the image processing unit 130 determines the vehicle based on the information obtained from the position detection unit 120 and the two-dimensional or three-dimensional position information (two-dimensional or three-dimensional coordinate information) of an arbitrary point on the side mirrors 201 and 202. The angle of the line of sight to the side mirrors 201 and 202 of the driver with respect to the left and right directions and the up and down directions is calculated (step S2). The three-dimensional positions of the side mirrors 201 and 202 are known, and the vehicle side confirmation device is held in advance in a predetermined storage area.

  Next, the image processing unit 130 acquires information on the angles of the side mirrors 201 and 202 with respect to the left and right directions and the vertical direction of the vehicle from the angle detection units 111 and 112 (step S3). Then, using the information on the driver's angle and the information on the angles of the side mirrors 201 and 202 obtained in steps S2 and S3, the angles of the field of view that can be visually recognized by the driver with the side mirrors 201 and 202 with respect to the vehicle lateral direction and the vertical direction. (Visibility range) is calculated (step S4).

  Next, the image processing unit 130 uses the angle of view obtained in step S4 to display only the image in the range that cannot be viewed with the side mirror on the monitors 141 and 142, and the image that is continuous with the range of the view. The coordinates of the cutout position for cutting out the partial video corresponding to the field of view from the picked-up video picked up by the cameras 101 and 102 are calculated (step S5).

  Next, the image processing unit 130 uses the coordinates of the cutout position calculated in step S5 to cut out the partial video, and align, invert, and enlarge the partial image with respect to the captured video captured by the cameras 101 and 102. Alternatively, predetermined image processing such as reduction is performed (step S6), and the image-processed video is displayed on the monitors 141 and 142 (step S7). The image processing unit 130 repeats the above processing during operation (step S8). FIG. 10 shows an image diagram of image processing on the driver's seat side.

  As shown in FIG. 10, after the partial video is cut out, the partial video is reversed left and right in order to display it in the same horizontal direction as a normal side mirror. Furthermore, the partial video is enlarged or reduced in order to display at the same enlargement ratio as the side mirror according to the number of pixels of the cameras 101 and 102, the resolution of the monitors 141 and 142, the dot pitch, and the like. In addition, alignment processing for moving the image to the side mirror side is performed so that the field of view reflected on the side mirror and the image field of view become a continuous field of view. Note that the alignment process is not limited to the horizontal direction, but also the vertical direction.

  According to the first embodiment, it is possible to cause the driver to grasp the situation on the side of the vehicle without a sense of incongruity, and thus contribute to the improvement of safety during driving. In other words, the left-hand scene is displayed on the left monitor 142 and the right-hand scene is displayed on the right monitor 141, so there is no difference between the driver and the conventional feeling of looking at the side mirror. Can be suppressed. In addition, since the video field of view displayed on the monitors 141 and 142 and the field of view displayed on the side mirrors 201 and 202 are obtained as a continuous field of view, the driver can be prevented from feeling uncomfortable. In addition, since there is no overlapping portion between the field of view reflected on the side mirrors 201 and 202 and the video field of view, anxiety can be suppressed.

  In addition, since a part of the side mirror is used to visually recognize the scene on the side of the vehicle, the range that the cameras 101 and 102 must capture can be limited, and the number of cameras can be reduced. In addition, regarding the monitors 141 and 142, only the range that cannot be visually recognized by the side mirrors 201 and 202 may be displayed, and the monitors can be used effectively. Even in the case of rain, the video field of view does not become difficult to see due to raindrops on the side mirrors 201 and 202 and the side glass.

  Further, by acquiring information on the angles of the side mirrors 201 and 202 with respect to the vertical direction and the left and right direction of the vehicle, the image processing unit 130 determines the range of the field of view that the driver visually recognizes through the side mirrors 201 and 202. , 202 can be recognized according to the angle of the display, and the video field of view can be displayed more accurately so as to be continuous with the field of view shown in the side mirrors 201 and 202, so that the driver can grasp the side view of the vehicle more comfortably. can do.

  Further, the image processing unit 130 acquires information on the position of the driver's eyes with respect to the vehicle front-rear direction and the vertical direction, or the vehicle front-rear direction, vertical direction, and left-right direction. The position of the eyes that change depending on the position and posture can be recognized, and the video field of view can be displayed more accurately so that it is continuous with the field of view reflected on the side mirrors 201 and 202, so that the driver can see the scene on the side of the vehicle more comfortably I can grasp it.

(Embodiment 2)
FIG. 11 is a block diagram illustrating a schematic configuration of the vehicle side confirmation apparatus according to the second embodiment. However, members corresponding to those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 11, a recording unit (recording unit) 150 records cutout position information (cutout position coordinates) for cutting out a partial video from the picked-up images picked up by the cameras 101 and 102.

  Further, the cut-out position coordinate calculation processing by the image processing unit 131 is the same processing as steps S1 to S5 described in FIG. 9 described above, and the image processing unit 131 executes this calculation processing at regular intervals. The cut-out position coordinates are calculated, and the information recorded in the recording unit 150 is updated. In addition, the image processing and the video display processing by the image processing unit 131 are the same as those in steps S6 and S7 described with reference to FIG. 9 described above, and the first embodiment uses the information recorded in the recording unit 150. Done in the same way.

  According to the second embodiment, the coordinates of the cutout position are calculated at regular intervals, the information in the recording unit 150 is updated, the information on the cutout position is read from the recording unit 150, and image processing is executed. Following the change of the seat position and posture, the video field of view can be adjusted to the field of view reflected on the side mirror in real time, and the driver can always grasp the scene on the side of the vehicle without a sense of incongruity.

  In addition, although the case where the cutout position is calculated every fixed time has been described here, the cutout position is calculated and recorded in the recording unit 150 only at the start of operation, and the cutout position calculated at the start of operation is always during operation. The information may be used.

  By calculating the cut-out position only at the start of operation, the angle detection units 111 and 112 and the position detection unit 120 need only be operated at the start of operation, and it is not necessary to use these during operation, thereby reducing power consumption. be able to.

(Embodiment 3)
FIG. 12 is a block diagram showing a schematic configuration of the vehicle side confirmation apparatus according to the third embodiment. However, members corresponding to those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 12, a recording unit (recording unit) 151 is used to cut out a partial video corresponding to the video field of view that is continuous with the range of the field of view shown on the side mirrors 201 and 202 from the captured video captured by the cameras 101 and 102. Cutout position pattern information (cutout position coordinate pattern information) is recorded in advance in association with the combination pattern of the angle information of the side mirrors 201 and 202 and the driver's eye position information.

  The image processing unit 132 also cuts out information corresponding to the combination of the angle information of the side mirrors 201 and 202 detected by the angle detection units 111 and 112 and the information of the driver's eye position detected by the position detection unit 120. A pattern of position coordinates is searched from the recording unit 151, and image processing and video display processing similar to those of the first embodiment are performed using the information of the clipped position coordinates thus searched.

  FIG. 13 shows a flowchart of processing in the image processing unit 132. However, the processing corresponding to the processing described in FIG. 9 is denoted by the same reference numeral, and description thereof is omitted. The image processing unit 132 acquires eye position information from the position detection unit 120 (step S1), acquires the angles of the side mirrors 201 and 202 from the angle detection units 111 and 112 (step S3), and the eye position. The pattern of the cut-out position coordinates corresponding to the combination of the information of and the angle information of the side mirrors 201 and 202 is searched from the recording unit 151 (step S11).

  Next, the image processing unit 132 records / updates the retrieved cut-out position coordinate information in a recording unit such as the recording unit 151 (step S12), and uses the cut-out position coordinates recorded in step S12 to perform the implementation. As in the first mode, predetermined image processing and video display processing are performed (steps S6 and S7).

  In this way, by recording the pattern information of the cut-out position coordinates in advance corresponding to the combination pattern of the angle information of the side mirrors 201 and 202 and the information of the driver's eye position, the range of view and the cut-out are recorded. The position calculation process can be omitted, and the process can be performed at a higher speed.

  In the vehicle side confirmation device according to the third embodiment, as in the second embodiment, the cut-out position coordinates are searched at regular intervals, and recorded / updated in the recording means such as the recording unit 151. For a certain period of time, predetermined image processing may be performed using the recorded search information.

  Of course, the cut-out position coordinates are searched only at the start of operation and recorded in the recording means such as the recording unit 151. During the operation, the predetermined image processing is always performed using the search information searched at the start of the operation. It may be. This not only makes it possible to eliminate the cut-out position, but also eliminates the need to operate the angle detection units 111 and 112 and the position detection unit 120 after the start of operation, thereby reducing power consumption.

(Embodiment 4)
FIG. 14 is a block diagram showing a schematic configuration of the vehicle side confirmation apparatus according to the fourth embodiment. However, members corresponding to those described in the first to third embodiments are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 14, a setting unit (setting unit) 160 is a unit through which the driver inputs setting information for adjusting the parameters of the cut-out position coordinates searched by the image processing unit 133. As an example, the driver sets, with respect to the video field of view displayed on the monitors 141 and 142, whether it is outside or inside of the current state, above or below, or to be enlarged or reduced. This setting information can be preset registered in recording means such as the recording unit 151. Multiple preset registrations can be made, and preference information of multiple drivers can be registered.

  FIG. 15 shows a flowchart of processing in the image processing unit 133. However, the processes corresponding to the processes described in FIGS. 9 and 13 are denoted by the same reference numerals, and the description thereof is omitted. The image processing unit 133 first acquires information on the position of the driver's eyes (step S1), acquires information on the angles of the side mirrors 201 and 202 (step S3), and cuts out positions, as in the third embodiment. The coordinates are retrieved (step S11), and the retrieved cut-out position coordinates are recorded / updated in the recording means such as the recording unit 151 (step S12).

  Next, the image processing unit 133 determines whether or not the setting information has been updated by the setting unit 160 (step S21). If the setting information has been updated, the setting information registered in the recording unit such as the recording unit 151 is updated. (Step S22).

  Next, the image processing unit 133 reads out the information of the cutout position coordinates recorded in step S12 and the registered setting information, performs the parameter conversion of the cutout position coordinates based on the setting information, and the recording unit 151 and the like. Recording is performed on the recording means (step S23).

  Thereafter, the image processing unit 133 performs predetermined image processing and video display processing in the same manner as in the first embodiment using the coordinates of the cutout position recorded in step S23 (steps S6 and S7).

  According to the fourth embodiment, the parameter of the cutout position can be adjusted, so that the driver can adjust the range of the video field of view according to his / her preference. A plurality of presets can be registered. In addition, the angle detection units 111 and 112 and the position detection unit 120 may be used when adjusting the parameters of the cut-out position, and after adjusting a rough range using these means, the adjustment is performed according to the driver's preference. You can also

  In the fourth embodiment, the setting unit 160 and the parameter conversion function of the cut-out position coordinates by the image processing unit are added to the configuration of the vehicle side confirmation device in the third embodiment described above. The present invention can also be applied to the vehicle side confirmation device according to the first embodiment. Of course, as described in the second and third embodiments, the cut-out position coordinates are set at regular intervals or only at the start of driving. It can also be configured to calculate or search.

(Embodiment 5)
FIG. 16 shows a display example of the monitor on the passenger seat side in the fifth embodiment. For example, when the angle of the side mirror 202 and the monitor 142 is different, and there is a difference in the appearance of the field of view and the image field of view displayed on the side mirror 202, the image processing unit of the vehicle side confirmation apparatus according to the fifth embodiment Information on the angle of the side mirror 202 detected by the angle detection unit 112, information on the position of the eyes of the driver detected by the position detection unit 120, information on the position of the side mirror 202, and the position and angle of the monitor 142 Of the monitor 142 and the side mirror 202 based on the above information (the three-dimensional position information and two-dimensional angle information of the monitor are already known, and the vehicle side confirmation device is held in a predetermined storage area in advance). It has a function of calculating a distortion amount generated by the difference and correcting a partial image cut out from the captured image based on the distortion amount.

  Here, the amount of distortion refers to the amount of difference generated between the length and angle of the object reflected on the side mirror and the length and angle of the object reflected on the monitor due to the difference between the angle of the side mirror and the monitor. In addition, for example, the angle information of the camera, side mirror, and monitor with respect to the vehicle left-right direction and the up-down direction, and information on the angle between the side mirror and the eyes of the driver and the vehicle left-right direction and the up-down direction are parameters. The distortion amount may be obtained by previously recording the image conversion table in the recording unit.

  As a result, the video field of view displayed on the monitors 141 and 142 can be corrected in accordance with the field of view displayed on the side mirrors 201 and 202, and it is possible to display an image that is more comfortable when viewed from the driver. In addition, a monitor having a larger width in the vertical direction of the vehicle than the side mirrors 201 and 202 is used for displaying the corrected image.

(Embodiment 6)
FIG. 17 is a block diagram showing a schematic configuration of the vehicle side confirmation apparatus according to the sixth embodiment. However, members corresponding to those described in the first to fourth embodiments are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 17, the control unit 170 performs drive control of the movable units (not shown) of the monitors 141 and 142. In the sixth embodiment, the moving means for adjusting the angle and position of the monitor is configured by the control unit 170 and the movable unit.

  The control unit 170 acquires the information on the angles of the side mirrors 201 and 202, the information on the position of the eyes of the driver, and the information on the position of the side mirror 201 via the image processing unit 133. Are different from each other, the angle displacement amount for adjusting the angle of the monitor 142 to the angle of the side mirror 202 is calculated based on the angle information of the side mirror 202 and the current angle information of the monitor 142, and the movable part of the monitor 142 is calculated. Is controlled so that the angles of the monitor 142 and the side mirror 202 coincide. Then, based on the information on the position of the eyes of the driver, the information on the position of the side mirror 202, and the current position information on the monitor 142, the moving amount of the monitor 142 is calculated and the movable part is driven and controlled. Adjust the position. The current angle and position of the monitor can be obtained from the state of the movable part.

  FIG. 18 shows an installation example and a display example of the passenger side monitor in the sixth embodiment. As shown in FIG. 18, the angle and position of the monitor 142 are adjusted so that the driver can grasp the video field of view by the monitor 142 and the field of view reflected on the side mirror 202 as a continuous field of view.

  FIG. 19 shows the positional relationship between the monitor and the side mirror in the sixth embodiment. The angles of the monitors 141 and 142 can be changed within the range of the monitor movable range 181 and 182. Hereinafter, adjustment of the position of the monitor will be described. As illustrated in FIG. 19, the control unit 170 uses the information on the position of the driver's eyes, the information on the positions of the side mirrors 201 and 202, and the current position information on the monitor 142 to determine the position of the driver's eyes. And the amount of movement of the monitors 141 and 142 for substantially matching the straight line connecting the vehicle side ends of the side mirrors 201 and 202 and the straight line connecting the driver's eye position and the side mirror side ends of the monitors 141 and 142. After the calculation, the monitors 141 and 142 are moved. FIG. 20 is a diagram showing an example of the movable part of the monitor. The monitor is attached to the arm, and each operating shaft has a structure that can be finely controlled by a stepping motor.

  As described above, the control unit 170 changes the angles of the monitors 141 and 142 according to the angles of the side mirrors 201 and 202 so that the monitors 141 and 142 and the side mirrors 201 and 202 have the same angle. This makes it easier to recognize the field of view.

  Further, by adjusting the positions of the monitors 141 and 142, the driver can grasp the video field of view displayed on the monitors 141 and 142 and the field of view displayed on the side mirrors 201 and 202 as a continuous view.

  As described above, according to the first to sixth embodiments, since the video field of view displayed on the monitor and the field of view displayed on the side mirror are continuous, the driver can grasp the scene on the side of the vehicle without a sense of incongruity. be able to. Further, since the scene on the side of the vehicle can be visually recognized by looking at the same direction as before, a sense of security can be obtained. Moreover, since the field of view by the side mirror is also efficiently used, a field of view that is easy to grasp can be obtained without seeing two buildings. In addition, the monitor can be used effectively.

  The vehicle side confirmation device according to the present invention makes the field of view shown on the side mirror and the image field of view displayed on the display means a continuous field of view, and allows the driver to intuitively grasp the side of the vehicle without a sense of incongruity. It can also be applied to applications that widen the field of view on the side of the vehicle.

The block diagram which shows schematic structure of the vehicle side confirmation apparatus in Embodiment 1 of this invention. The figure for demonstrating the range of the visual field which a driver | operator can visually recognize with the imaging range and side mirror of the imaging device in the embodiment The figure which shows the example of installation of the monitor and the camera for position detection in the driver's seat side of the embodiment The figure which shows the example of installation of the monitor in the passenger seat side of the embodiment Schematic diagram of a captured image captured by the position detection camera in the same embodiment The figure for demonstrating the method to detect the position of the eye with respect to the up-down direction of the vehicle in the embodiment The figure for demonstrating the method to detect the position of the eyes with respect to the front-back direction of the vehicle in the embodiment The figure for demonstrating the method to detect the position of the eye with respect to the left-right direction of the vehicle in the embodiment The figure which shows the flowchart of the process in the image process part of the embodiment Image of image processing on the driver's seat side in the same embodiment The block diagram which shows schematic structure of the vehicle side confirmation apparatus in Embodiment 2 of this invention. The block diagram which shows schematic structure of the vehicle side confirmation apparatus in Embodiment 3 of this invention. The figure which shows the flowchart of the process in the image process part of the embodiment The block diagram which shows schematic structure of the vehicle side confirmation apparatus in Embodiment 4 of this invention. The figure which shows the flowchart of the process in the image process part of the embodiment The figure which shows the example of a display of the monitor by the side of a passenger seat in Embodiment 5 of this invention FIG. 9 is a block diagram showing a schematic configuration of a vehicle side confirmation device according to Embodiment 6 of the present invention. The figure which shows the installation example and display example of the monitor by the side of the passenger seat in the embodiment The figure which shows the positional relationship of the monitor and side mirror in the embodiment The figure which shows the example of the movable part of the monitor in the embodiment

Explanation of symbols

101, 102 Camera 111, 112 Angle detection unit 120 Position detection unit 121, 122 Position detection camera 130, 131, 132, 133 Image processing unit 141, 142 Monitor 150, 151 Recording unit 160 Setting unit 170 Control unit 181, 182 Monitor Range of motion 200 Vehicle 201, 202 Side mirror 211, 212 Range of view 221, 222 Range of video view

Claims (16)

A side mirror that is set at an angle so that the driver can visually recognize a predetermined range of scenes outside the side of the vehicle,
Imaging means for imaging a predetermined range of scenes including a blind spot range on the vehicle side of the side mirror;
Display means installed beside the side mirror and inside the vehicle;
The captured image captured by the imaging unit is subjected to image processing so that the video field of view displayed on the display unit is continuous with the field of view displayed on the side mirror, and the image after the image processing is displayed on the display unit Image processing means to be displayed on,
A vehicle lateral confirmation device comprising: The vehicle side confirmation device according to claim 1,
Angle detection means for detecting the angle of the side mirror;
Position detecting means for detecting the position of the driver's eyes;
The image processing means includes a field of view reflected on the side mirror using information on the angle of the side mirror detected by the angle detection means and information on the position of the eyes of the driver detected by the position detection means. A cut-out position for cutting out the partial video corresponding to the video field of view that is continuous with the range of the visual field from the picked-up video, and using the information on the cut-out position, a predetermined image is added to the picked-up image. Process,
The vehicle side confirmation apparatus characterized by the above-mentioned.   The vehicle side confirmation device according to claim 2, wherein the angle detection means detects an angle of the side mirror with respect to a vertical direction and a horizontal direction of the vehicle.   The position detection stage has two cameras for detecting the position of the driver's eyes, and using the captured images of the two cameras, the position of the driver's eyes with respect to the front-rear direction and the vertical direction of the vehicle 4. The vehicle side confirmation device according to claim 2, wherein the position of the driver's eyes with respect to the front-rear direction, the vertical direction, and the left-right direction of the vehicle is detected. The vehicle side confirmation device according to any one of claims 2 to 4,
A recording means for recording the information of the cut-out position;
The image processing unit calculates a cutout position at regular intervals, updates information recorded in the recording unit, and performs predetermined image processing on the captured video using the information recorded in the recording unit. Apply,
The vehicle side confirmation apparatus characterized by the above-mentioned. The vehicle side confirmation device according to any one of claims 2 to 4,
A recording means for recording the information of the cut-out position;
The image processing means calculates a cut-out position at the start of operation, records it in the recording means, and performs predetermined image processing on the captured video using information recorded in the recording means.
The vehicle side confirmation apparatus characterized by the above-mentioned. The vehicle side confirmation device according to any one of claims 2 to 6,
A setting unit for inputting setting information for adjusting a parameter of the cutout position calculated by the image processing unit;
The image processing means further has a function of converting the parameter of the cutout position based on the setting information, and performs predetermined image processing on the captured video using the information of the cutout position after the conversion.
The vehicle side confirmation apparatus characterized by the above-mentioned. The vehicle side confirmation device according to claim 1,
Angle detection means for detecting the angle of the side mirror;
Position detecting means for detecting the position of the driver's eyes;
The pattern information of the cut-out position for cutting out the partial video corresponding to the video field of view that is continuous with the range of the visual field reflected on the side mirror, from the combination of the angle of the side mirror and the position of the eyes of the driver Recording means for recording corresponding to the pattern;
And the image processing means includes a pattern of cutout positions corresponding to a combination of information on the angle of the side mirror detected by the angle detection means and information on the position of the eyes of the driver detected by the position detection means. Is retrieved from the pattern information of the cutout position recorded in the recording means, and predetermined image processing is performed on the captured video using the information of the cutout position thus searched.
The vehicle side confirmation apparatus characterized by the above-mentioned.   The vehicle side confirmation device according to claim 8, wherein the angle detection means detects an angle of the side mirror with respect to a vertical direction and a horizontal direction of the vehicle.   The position detection stage has two cameras for detecting the position of the driver's eyes, and using the captured images of the two cameras, the position of the driver's eyes with respect to the front-rear direction and the vertical direction of the vehicle The vehicle side confirmation device according to claim 8 or 9, wherein the position of the driver's eyes with respect to the front-rear direction, the vertical direction, and the left-right direction of the vehicle is detected. The vehicle side confirmation device according to any one of claims 8 to 10,
The recording unit records search information of a cutout position searched by the image processing unit,
The image processing means searches for the pattern of the cutout position at regular intervals, updates the search information recorded in the recording means, and uses the search information recorded in the recording means to predetermine the captured image. Image processing,
The vehicle side confirmation apparatus characterized by the above-mentioned. The vehicle side confirmation device according to any one of claims 8 to 10,
The recording unit records search information of a cutout position searched by the image processing unit,
The image processing means searches for the pattern of the cutout position at the start of operation, records it as search information in the recording means, and performs predetermined image processing on the captured video using the search information recorded in the recording means ,
The vehicle side confirmation apparatus characterized by the above-mentioned. The vehicle side confirmation device according to any one of claims 8 to 12,
Further comprising setting means for inputting setting information for adjusting the parameter of the clipping position searched by the image processing means;
The image processing means further has a function of converting the parameter of the cutout position based on the setting information, and performs predetermined image processing on the captured video using the information of the cutout position after the conversion.
The vehicle side confirmation apparatus characterized by the above-mentioned. The vehicle side confirmation device according to any one of claims 2 to 13,
The display means is wider in the vertical direction of the vehicle than the side mirror,
The image processing means calculates the amount of distortion caused by the angle difference between the display means and the side mirror using the side mirror angle information and the driver's eye position information. Further having a function of correcting a partial image cut out from the captured image,
The vehicle side confirmation apparatus characterized by the above-mentioned.   The vehicle side confirmation device according to any one of claims 2 to 13, wherein information on an angle of the side mirror and information on a driver's eye position are acquired via the image processing means, and the information is obtained. The vehicle side confirmation apparatus further comprising a moving means for adjusting an angle and a position of the display means by using a display. The vehicle side confirmation device according to any one of claims 1 to 15, wherein the imaging means is an infrared camera.

Images