JP2005079966A - Circumference monitor for vehicle and circumference display method for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make possible for a driver to easily grasp parallelism between a driver's vehicle and an object becoming a target when the vehicle is stopped. <P>SOLUTION: A target object, e.g. a white line, is detected based on a video image photographed by means of a CCD camera 1. A marker line parallel with the driver's vehicle is displayed while being superposed on a display 3 with reference to the position of the detected target together with the video image photographed by means of the CCD camera 1. The white line and a marker line are displayed while being superposed if the driver's vehicle is parallel with the white line, otherwise they are not displayed while being superposed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an apparatus for monitoring the surroundings of a host vehicle using an in-vehicle camera and a method for displaying an image of the surroundings of the host vehicle. The present invention relates to a vehicle surroundings monitoring device that assists driving operations and a vehicle surroundings display method.

  There is known an apparatus that captures an image of a vehicle periphery that becomes a blind spot of a driver with an in-vehicle camera and displays information that assists driving operation on the captured image (see Patent Document 1). In this device, the vehicle's outermost line is lowered perpendicularly to the ground, and the line extending the vehicle in the longitudinal direction of the vehicle is overlaid on the photographed image to display the subject vehicle closer to the target. Making it easy.

JP 2000-172996 A

  However, in the conventional apparatus, since the position that becomes the blind spot of the driver is photographed, even if the optical axis of the camera does not coincide with the straight traveling direction of the vehicle and the own vehicle and the target such as the white line are actually parallel to each other, However, it is not displayed in parallel on the screen, and there is a problem that it is difficult to understand the parallel state with the own vehicle.

  The vehicle surroundings monitoring device and the vehicle surroundings display method according to the present invention detect a line-shaped target that is a target when stopping the vehicle based on the video imaged by the imaging unit, and detect the detected target object. A marker line parallel to the host vehicle is superimposed on a captured image and displayed on the basis of the position.

  According to the vehicle surrounding monitoring device and the vehicle surrounding display method according to the present invention, the marker line parallel to the host vehicle is superimposed and displayed on the basis of the detected position of the line-shaped target. Regardless of the positional relationship with the target such as, the parallel state between the target and the host vehicle can be easily confirmed.

  FIG. 1 is a diagram showing a configuration of an embodiment in which a vehicle surrounding monitoring apparatus according to the present invention is applied to a vehicle 10. The vehicle surrounding monitoring apparatus in one embodiment includes a CCD camera 1, a control unit 2, a display 3, and a vehicle speed sensor 4.

  The CCD camera 1 is built in the left door mirror 11 of the vehicle (see FIG. 2), and photographs the side of the vehicle including a part of the side surface of the vehicle 10, particularly the vicinity of the left front wheel 6. The control unit 2 includes a CPU, a ROM, and a RAM (not shown), detects a curb or white line on the road side based on an image captured by the CCD camera 1, and superimposes a marker line parallel to the host vehicle on the captured image. It is displayed on the display 3. A method for detecting a roadside curb or white line and a method for superimposing a marker line on a captured image will be described later.

The display 3 is provided at a position that is easy for the driver to see, and can display at least an image captured by the CCD camera 1. When the vehicle 10 includes a car navigation device, map information including route information to the destination can be displayed on the display 3. The vehicle speed sensor 4 detects the speed of the host vehicle and outputs the detected vehicle speed signal to the control unit 2.

  FIG. 2 is a diagram illustrating a situation in which the vehicle 10 is brought closer to the curb 13 existing on the road side as viewed from the rear of the vehicle 10. As shown in FIG. 2, the CCD camera 1 captures a position that is a blind spot for a driver of a right-hand drive vehicle.

FIG. 3 is a diagram in which an image captured by the CCD camera 1 built in the left door mirror 11 is displayed on the display 3 under the situation shown in FIG. As shown in FIG. 3, a part of the body of the left front wheel 6 and the vehicle 10 is shown on the right side of the screen displayed on the display 3, and the asphalt surface on which the white line 14 is drawn is on the left side thereof. 12 and curb 13 are shown.

  When the vehicle 10 is brought closer to the road side, the driver looks at the video imaged by the CCD camera 1 and displayed on the display 3, and performs the width adjustment based on the curb 13 or the white line 14. At this time, as shown in FIG. 3, since the optical axis of the CCD camera 1 is adjusted so that the vicinity of the left front wheel 6 is imaged, the parallel state between the host vehicle 10 and the curb 13 or the white line 14 is difficult to understand. This point will be described in detail with reference to FIGS.

  FIG. 4 is a view of the state in which the vehicle 10 is stopped in parallel with the curb 13 and the white line 14 as viewed from above the vehicle 10. FIG. 5 is a view showing a screen on which video images taken by the CCD camera 1 are displayed on the display 3 under the situation shown in FIG. As shown in FIG. 5, an angle θ1 exists between a straight line L1 extending along the curb 13 and a straight line L2 parallel to the host vehicle. That is, even if the curb 13 and the host vehicle 10 are parallel, they are not reflected in parallel on the image taken by the CCD camera 1.

  FIG. 6 is a diagram showing a state in which the marker line 15 parallel to the host vehicle is superimposed on the image captured by the CCD camera 1 and displayed on the display 3 as described in Patent Document 1 described above. is there. Since the marker line 15 is a line that is displayed at a position fixed with respect to the host vehicle 10, that is, a predetermined position on the screen, between the straight line L1 extending along the curb 13 There is an angle θ2. That is, also in this case, it is not easy to check the parallel state between the host vehicle 10 and the curb 13 or the white line 14.

  FIG. 7 is a diagram illustrating a state in which the vehicle 10 is parked while retreating in a parking frame delimited by a white line drawn in the parking lot. The driver starts an operation for parking from the position A shown in FIG. 7, and stops the vehicle 10 at the position B while steering a steering wheel (not shown). At this time, the driver looks at the image captured by the CCD camera 1 and displayed on the display 3 as shown in FIG. 8 and stops the vehicle 10 in parallel with the parking frame line 20. However, as described above, it is difficult to grasp the parallel state between the host vehicle 10 and the parking frame line 20.

  In the vehicle surrounding monitoring apparatus according to the embodiment, a target (parking frame line 20 in the example shown in FIG. 7) is detected based on an image captured by the CCD camera 1, and feature points of the detected target are detected. Based on the above, a straight line (marker line) parallel to the host vehicle 10 is superimposed and displayed on the captured image.

  FIG. 9 is a diagram illustrating straight lines (marker lines) 30a and 30b parallel to the host vehicle 10 drawn based on the feature points of the target detected by the vehicle surrounding monitoring apparatus according to the embodiment. A method of drawing the straight lines 30a and 30b will be briefly described. First, a line E1-E2 passing through the contact C between the left front wheel 6 of the host vehicle 10 and the road surface 12 and parallel to the horizontal axis of the screen is drawn, and intersections D1, D2 between the line E1-E2 and the parking frame line 20 are drawn. To detect. Lines passing through the intersections D1 and D2 and parallel to the host vehicle 10 become straight lines 30a and 30b. In this case, as shown in FIG. 10, if the host vehicle 10 is parallel to the parking frame line 20, the straight lines 30 a and 30 b overlap the white line 20.

  On the other hand, as shown in FIG. 11, when the own vehicle 10 and the parking frame line 20 are not parallel, straight lines 30a and 30b drawn on the screen by the same method are shown in FIG. In this case, the straight lines 30 a and 30 b do not overlap with the white line 20. Therefore, the driver can grasp at a glance whether or not the host vehicle 10 is parallel to the parking frame line 20 by looking at the positional relationship between the straight lines 30 a and 30 b and the white line 20.

  Below, based on the image | video image | photographed with the CCD camera 1, the method which detects the parking frame line 20 and draws a marker line parallel to the own vehicle is shown in the flowchart shown in FIG. 13, FIG. 14, FIG. It explains using. The process starting from step S10 starts, for example, when the vehicle is started and is performed by the control unit 2.

  In step S10, based on the vehicle speed signal input from the vehicle speed sensor 4, it is determined whether or not the speed V of the host vehicle 10 is equal to or less than a predetermined threshold value Vh. The threshold value Vh is, for example, 15 km / h. If it is determined that the vehicle speed V is higher than the predetermined threshold value Vh, the process waits in step S10. If it is determined that the vehicle speed V is equal to or lower than the predetermined threshold value Vh, the process proceeds to step S20.

  In step S20, an image captured by the CCD camera 1 is captured, and the process proceeds to step S30. In step S30, an area for detecting a target such as a white line or a curb is set based on the video captured in step S20. FIG. 14A is a diagram illustrating an image captured by the CCD camera 1 and an area 40 for detecting a target. Here, the horizontal axis (horizontal axis) of the screen displayed on the display 3 is defined as the h axis, and the vertical axis (vertical axis) is defined as the v axis.

The region 40 is a rectangular region having a predetermined size centered on a line E1-E2 parallel to the h axis, passing through the point C where the left front wheel 6 of the host vehicle 10 and the ground 12 are in contact. A point C where the left front wheel 6 of the host vehicle 10 and the ground 12 are in contact with each other on the photographed image is a point determined by the installation position and the photographing direction of the CCD camera 1. When the area 40 for detecting the target is set, the process proceeds to step S40.

  White lines and curbs that serve as a guide when parking a vehicle have a vertical edge (hereinafter referred to as a vertical edge) on an image captured by the CCD camera 1. Here, the edge extending in the vertical direction in the region 40 for detecting the target is referred to as a vertical edge. Accordingly, in step S40, a vertical edge existing in the detection area 40 set in step S30 is detected. Specifically, the vertical edge is detected by performing a differential operation in the h-axis direction on the luminance value of the image in the detection region 40. FIG. 14B shows the vertical edges detected in the detection area 40. If a vertical edge is detected, the process proceeds to step S50.

  In step S50, an edge that becomes noise is removed from the vertical edges detected in step S40. Specifically, among the detected vertical edges, edges continuously extending from the upper end to the lower end of the detection area 40 are detected. FIG. 14C is a diagram showing the detection region 40 after removing an edge that becomes noise.

  In step S60, intersections D1 and D2 between the vertical edge detected in step S50 and the horizontal line E1-E2 are detected (see FIG. 14C). When the intersection points D1, D2 are detected, the process proceeds to step S70. In step S70, a marker line closest to the target object is selected from among a plurality of marker lines stored in ROM (not shown) of the control unit 2 and parallel to the host vehicle.

A method for selecting the marker line closest to the target will be described with reference to FIG. FIG. 15 is a diagram in which a plurality of marker lines stored in the ROM of the control unit 2 are displayed together with an image taken by the CCD camera 1. However, not all marker lines are displayed on the display 3 as shown in FIG. Here, it is assumed that n (n is a natural number) marker lines parallel to the host vehicle 10 are stored in the ROM of the control unit 2. The n marker lines are represented by the following formula (1).
v = fn (h) (n = 1, 2, 3,...) (1)

Further, on the photographed video, the coordinates of the point C where the left front wheel 6 of the host vehicle 10 and the ground 12 are in contact are obtained in advance and stored in the ROM. Assuming that the value of the v coordinate of this point C is a, the equation representing the line E1-E2 that passes through the point C and is parallel to the h axis is the following equation (2).
v = a (2)

Therefore, the intersection of the n marker lines and the horizontal line E1-E2 can be obtained by solving the following equation (3) from equations (1) and (2).
a = fn (h) (n = 1, 2, 3,...) (3)
The solution h of equation (3) is defined as the following equation (4).
h = hn (n = 1, 2, 3,...) (4)

Here, assuming that the coordinates of the intersection point D1 detected in step S60 are (h _D , v _D ), the distance dn in the h-axis direction between the intersection point D1 and the n intersection points represented by the equation (4) is It is represented by Formula (5).
dn = | h _D −h n | (n = 1, 2, 3,...) (5)

  Therefore, the marker line closest to the target can be selected by obtaining n that minimizes the distance dn represented by Expression (5). In the example shown in FIG. 15, the marker line of n = 2 is the marker line closest to the target. When the marker line closest to the target is selected, the process proceeds to step S80. In step S80, the marker line selected in step S70 is drawn on the image captured by the CCD camera 1. Thereafter, the processing from step S10 to step S80 is repeatedly performed.

  In the above description, the method for selecting and drawing the marker line closest to the intersection D1 has been described. However, the marker line closest to the intersection D2 can also be selected and drawn by the same method.

  According to the vehicle periphery monitoring device in one embodiment, a target (white line or curb) that is a target when the vehicle is stopped is detected based on an image captured by the CCD camera 1, and the detected target is detected. A marker line parallel to the host vehicle is superimposed on the photographed image and displayed on the display 3 with the position of. Thereby, irrespective of the positional relationship between the host vehicle and a target such as a white line, it is possible to easily grasp the parallel state of both.

  In addition, the driver looks at a video taken by a door mirror or a camera that monitors the rear of the host vehicle, puts the vehicle in a parking frame, stops the vehicle, and then displays the display by the vehicle surrounding monitoring device in one embodiment. By looking at the marker line displayed at 3, it is possible to confirm the parallel state of the parking frame and the host vehicle.

  Further, according to the vehicle surrounding monitoring apparatus in the embodiment, the marker line is displayed on the display 3 when the speed V of the host vehicle is equal to or lower than the predetermined speed Vh. Thus, it is possible to prevent the marker line from being displayed when the marker line display is unnecessary.

  The present invention is not limited to the embodiment described above. For example, only one marker line can be drawn to be superimposed on the display 3. That is, in the flowchart shown in FIG. 13, a vertical edge is detected based on the captured video, and a marker line parallel to the host vehicle is drawn based on the detected vertical edge. When a vertical edge is detected, a plurality of marker lines are drawn.

  FIG. 16A is a diagram showing a state in which all vertical edges on the video are detected in a place where the curb 13 exists in addition to the white line 14 and marker lines are drawn on all the detected vertical edges. is there. In this case, since a plurality of marker lines are drawn, it is difficult to see the video. FIG. 16B is a diagram illustrating a state in which a marker line is drawn on the basis of the vertical edge closest to the host vehicle among the detected vertical edges. In this case, the troublesomeness of drawing a plurality of marker lines can be prevented.

The vehicle surrounding monitoring apparatus according to the present invention can be applied to a system having a function of drawing a fixed marker line prepared in advance on the display 3. This fixed marker line is a line displayed at a predetermined position on the screen, like the marker line 15 shown in FIG. FIG. 17A is a diagram in which a fixed marker line and a marker line drawn by the vehicle surrounding monitoring apparatus according to the embodiment are both displayed on the display 3. In this case, it becomes impossible to distinguish which is the marker line drawn by the vehicle surrounding monitoring apparatus in the embodiment. Therefore, in this case, as shown in FIG. 17 (b), the marker line drawn by the vehicle periphery monitoring device in one embodiment is drawn with a dotted line to distinguish it from the fixed marker line drawn with a solid line. Can do.

  When the marker line is used as a guideline for knowing the distance between the host vehicle and the target, it is easier to understand that only the fixed marker line is displayed on the display 3. Therefore, a switch is provided to switch between a mode for displaying a marker line for confirming the degree of parallelism between the host vehicle and the target and a mode for displaying a fixed marker line for grasping the interval. By doing so, only one of the marker lines may be displayed. In this case, a mode in which no marker line is displayed may be provided.

  The marker line is not limited to the solid line shown in FIG. 16B or the dotted line shown in FIG. 17B, and may be a broken line, a chain line, a wavy line, a thick line, or the like. Further, if the marker line is displayed so as to be emphasized in the image displayed on the display 3 more than the image other than the marker line (for example, red), the visibility of the occupant can be improved.

  The camera that captures the vicinity of the front wheel of the vehicle is not limited to a CCD camera, and may be a camera having a CMOS image sensor or another type of camera. When the right side of the vehicle is a blind spot for a driver, such as a left-hand drive vehicle, the CCD camera 1 may be attached to the right door mirror to photograph the vicinity of the right front wheel. That is, the present invention is not limited by the mounting position of the camera and the shooting range.

  The target to be used when the driver stops the vehicle is not limited to the white line or curb as described above, and may be a side wall of a parking lot. That is, what is necessary is just to be able to grasp the degree of parallelism with the host vehicle by drawing a marker line on the display 3.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the CCD camera 1 constitutes a photographing means, the control unit 2 constitutes a target detection means and a marker line drawing means, the display 3 constitutes a display means, and the vehicle speed sensor 4 constitutes a vehicle speed detection means. In addition, unless the characteristic function of this invention is impaired, each component is not limited to the said structure.

The figure which shows the structure of one Embodiment which applied the surroundings monitoring apparatus for vehicles by this invention to the vehicle. The figure which shows a mode that the situation where a vehicle approaches a curb from the back of a vehicle The figure which shows the image | video which is imaged with a CCD camera and displayed on a display in the condition shown in FIG. A view of the vehicle stopping from parallel to the curb and the white line, as seen from above the vehicle FIG. 5 is a diagram showing a screen in which an image taken by a CCD camera is displayed on the display under the situation shown in FIG. 4, and a straight line L1 extending along the curb and a straight line L parallel to the host vehicle are parallel on the image. Illustration to explain what does not become The figure for demonstrating that the marker line drawn by the method of the prior art and the straight line L1 extended along a curb are not parallel on an image | video. The figure which shows a mode that a vehicle is parked while retreating in the parking frame divided by the line drawn in the parking lot The figure which shows the image | video image | photographed with the CCD camera and displayed on a display in the condition shown in FIG. The figure which shows the straight lines (marker lines) 30a and 30b parallel to the own vehicle drawn based on the feature point of the detected target object when the vehicle is stopped parallel to the parking frame line The figure which shows the vehicle stopped parallel to the parking frame line A figure showing a vehicle that does not stop parallel to the parking frame 11 is a diagram showing straight lines (marker lines) 30a and 30b parallel to the host vehicle drawn based on the detected feature points of the target object in the situation shown in FIG. The flowchart which shows the processing content performed by the vehicle periphery monitoring apparatus in one embodiment 14A shows a detection area 40 set on the captured video, FIG. 14B shows a vertical edge detected in the detection area, and FIG. 14C shows detection. Figure with the vertical edge that causes noise removed from the vertical edge Figure depicting n marker lines parallel to the vehicle FIG. 16A shows a plurality of marker lines, and FIG. 16B shows only one marker line. FIG. 17A is a diagram showing a state in which a marker line parallel to the detected target object is drawn together with the fixed marker line, and FIG. 17B is a diagram in which the fixed marker line is drawn with a solid line, The figure which shows a mode that the marker line parallel to the detected target object was drawn with the dotted line

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... CCD camera 2 ... Control unit 3 ... Display 4 ... Vehicle speed sensor 6 ... Left front wheel 10 ... Vehicle 11 ... Left side mirror 12 ... Asphalt surface of roadway 13 ... Curb 14 ... White line 15 ... Marker line 20 ... Parking frame line 30a, 30b ... Marker line 40 ... Detection area

Claims (8)

Photographing means for photographing the side of the vehicle;
Display means for displaying at least the video imaged by the imaging means;
A target detection means for detecting a line-like target as a target when stopping the vehicle based on the video imaged by the imaging means;
Marker line drawing means for drawing a marker line parallel to the host vehicle on the image displayed by the display means on the basis of the position of the target detected by the target detection means. A vehicle surroundings monitoring device. The vehicle surrounding monitoring device according to claim 1,
The photographing means is provided on a door mirror to photograph the vicinity of the front wheel of the vehicle,
The vehicle surrounding monitoring apparatus, wherein the target object detecting unit detects the line-shaped target object by detecting a vertical edge based on an image photographed by the photographing unit. The vehicle surrounding monitoring device according to claim 1 or 2,
Storage means for storing a plurality of marker lines parallel to the host vehicle, which are respectively displayed at different positions of the display means;
Selecting means for selecting a marker line closest to the position of the target detected by the target detection means from a plurality of marker lines stored in the storage means;
The vehicle surrounding monitoring apparatus, wherein the marker line drawing means draws a marker line selected by the selection means. In the vehicle surrounding monitoring device according to any one of claims 1 to 3,
The target detection means detects a target closest to the host vehicle,
The vehicle periphery monitoring device, wherein the marker line drawing unit draws the marker line with reference to the position of the target closest to the host vehicle detected by the target detection unit. In the vehicle periphery monitoring device according to any one of claims 1 to 4,
Vehicle speed detecting means for detecting the speed of the vehicle,
The vehicle surrounding monitoring device, wherein the marker line drawing means draws the marker line when the vehicle speed detected by the vehicle speed detecting means is equal to or lower than a predetermined speed. In the vehicle surrounding monitoring device according to any one of claims 1 to 5,
The vehicle surrounding monitoring device, wherein the display means highlights the marker line. In the vehicle surrounding monitoring device according to any one of claims 1 to 6,
The vehicle surrounding monitoring device, wherein the marker line is at least one of a dotted line, a broken line, a chain line, a wavy line, and a thick line. Take a picture of the side of the vehicle,
Based on the photographed image, a line-shaped target that is a target when stopping the vehicle is detected,
A vehicle surrounding display method, wherein a marker line parallel to the host vehicle is superimposed on the captured image and drawn on a display means with the position of the detected target as a reference.

Images