JP6048384B2 - Image determination device - Google Patents

Description

  The present disclosure relates to an image determination apparatus that determines the presence or absence of deposits or reflections in a captured image of a camera.

  A technique related to a method for detecting an adhering matter attached to a camera that photographs the front of the host vehicle is known (see, for example, Patent Document 1). The apparatus disclosed in Patent Document 1 compares the pixel points of the feature amount of the latest image data and the past image data, and detects pixel points that coincide with each other for a predetermined time, thereby detecting the presence or absence of an attachment on the camera. Is detected.

JP 2007-172504 A

  However, for example, when the host vehicle is traveling straight ahead, a lane such as a white line of a road and a roadside band are detected as the same pixel point. For this reason, in the apparatus disclosed in Patent Document 1, there is a possibility that it is erroneously determined that there is an adhering matter or a reflection even though there is no adhering matter to the lens of the camera or no reflection to the image.

  Therefore, in view of the above points, it is desirable to provide an image determination device that can accurately determine the presence or absence of deposits or reflections in a captured image of a camera.

  According to one aspect of the present disclosure, there is provided an image determination device that determines the presence or absence of deposits or reflections in a captured image of a camera, and a feature that detects a pixel point of an image feature amount from image data in the captured image Amount detection means and turning state detection means for detecting the turning state of the host vehicle, and the position of the pixel point of the image feature amount when the host vehicle is running straight and the host vehicle is turning. When the difference between the image feature amount and the position of the pixel point at that time is equal to or less than a predetermined value, it is determined that there is an adhering substance or a reflection in the photographed image.

  According to the present disclosure, it is possible to obtain an image determination device that can accurately determine the presence or absence of deposits or reflections in a captured image of a camera.

The figure explaining the detection method of the edge point of a lane when the own vehicle is going straight ahead (FIG. 1 (a)), The figure explaining the detection method of the edge point of a lane when the own vehicle is running in a curve (FIG. 1B). FIG. 2A illustrates a method for detecting an edge point of an object when the host vehicle is traveling straight (FIG. 2A), and a diagram for describing a method for detecting an edge point of the object when the host vehicle is traveling in a curve. (FIG. 2B). It is a flowchart explaining the processing operation of the control apparatus of the image determination apparatus which concerns on one Example.

  Hereinafter, an image determination apparatus according to an embodiment will be described.

(Configuration of image determination device)
The image determination apparatus according to the present embodiment is an apparatus that determines the presence or absence of reflection of an attached object or an object 15 in a captured image of a camera in a vehicle environment recognition system that recognizes a vehicle environment captured by a camera. Specifically, the image determination device 1 compares the pixel points of the image feature amount detected from two types of image data with different steering angles of the vehicle's steering wheel and the like, and the image of the attached matter or the object 15 or the like. Judgment of presence or absence of inclusion is performed. More specifically, when it is recognized that the object exists, it is determined that there is a reflection object.

  The image feature amount refers to a pixel value change amount at a position (pixel point) where a change in pixel value occurs on the captured image frame 10 taken by the camera. In the following description, the pixel point where the pixel value changes is called an edge point, and the coordinates of the edge point are called edge coordinates. The edge coordinates are expressed as coordinates in the x-axis direction (x coordinate) and coordinates in the y-axis direction (y coordinate) (see FIGS. 1 and 2). Details will be described below.

  The image determination device includes a control device that controls the device centrally.

  The camera is an imaging unit that captures an image of the periphery of the host vehicle (for example, the traveling direction of the host vehicle) at a predetermined angle of view and outputs the captured image. The traveling direction of the host vehicle may be the forward direction or the reverse direction.

  The control device includes a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and the like.

  A vehicle speed sensor that detects the speed of the vehicle and a steering sensor are connected to the control device. A yaw rate sensor may be connected instead of the steering sensor.

  The vehicle speed sensor is attached to, for example, the wheel of the host vehicle, and acquires wheel speed information related to the speed of the wheel. A vehicle speed sensor transmits the wheel speed information regarding the acquired wheel speed to a control apparatus.

  The steering sensor is attached to a steering rod of the host vehicle, for example, and detects the steering angle of the steering operated by the driver. The steering sensor transmits steering angle information related to the detected steering angle to the control device.

  The yaw rate sensor is attached to the body of the host vehicle, for example, and detects the yaw rate (change speed of the rotation angle in the turning direction) when the host vehicle rotates horizontally with respect to the road surface. Acceleration is detected. The yaw rate sensor transmits yaw rate information related to the detected yaw rate to the control device.

(Configuration of control device)
Next, the configuration of the control device will be described.

  The control device detects a captured image captured by the camera as image data and detects a pixel point of the image feature amount from the image data in the captured image, and a change in the steering angle of the host vehicle, or The turning state detecting means for detecting the presence / absence of the yaw rate (turning state) and the notification means are included.

(Control mechanism of control device)
Next, the control mechanism of the control device will be described.

  When wheel speed information of the host vehicle is detected by the vehicle speed sensor, the wheel speed information is transmitted to the control device. When the transmitted wheel speed information is greater than or equal to a predetermined value, the steering angle information or yaw rate information detected by the steering sensor or yaw rate sensor is transmitted to the control device. The control device detects the turning state of the host vehicle based on the steering angle information or the yaw rate information (turning state information). This detection is performed by the turning state detecting means based on the steering angle information or the yaw rate information. Next, when a detection result indicating that the steering angle of the host vehicle is changed is obtained by the turning state detection means, the position of the edge point of the image feature value captured by the camera is detected. This detection is performed by the feature amount detection means provided in the control device. Then, the positions of the edge points of the image feature amount before and after the change of the steering angle of the steering wheel of the host vehicle, that is, the edge coordinates are compared. When no difference is recognized in the edge coordinates, the control device determines that “there is a reflection object” and notifies the driver of the fact by the notification means.

(Edge point detection method)
Next, a method for detecting edge points of image feature amounts before and after a change in steering angle will be described in detail with reference to the drawings. FIG. 1 is a diagram for explaining a lane edge point detection method when the host vehicle is traveling straight (FIG. 1A), and a lane edge point detection method when the host vehicle is traveling in a curve. It is a figure (FIG.1 (b)) explaining these. FIG. 2 is a diagram for explaining a method for detecting an edge point of the object 15 when the host vehicle is traveling straight (FIG. 2A), and an edge of the object 15 when the host vehicle is traveling in a curve. It is a figure (FIG.2 (b)) explaining the detection method of a point. The upper diagram shows an example of the captured image frame 10 by the camera, and the lower graph shows the distribution of pixel values of the captured image frame 10. FIG. 1B shows a state where the host vehicle is traveling on a left curve.

  First, the captured image frame 10 by the camera shown in the upper part of FIG. 1 will be described. When the host vehicle is traveling straight, the left and right lane positions appear as edge points (see points A1 to A4), as shown in FIG.

  The detection of the edge point is performed by sequentially calculating the difference value of the left pixel value with respect to the target pixel value. This calculation is performed by the CPU of the control device. Here, the target pixel value is a pixel value of a target pixel to be detected on a line segment LL (a broken line portion in FIG. 1A) extending in the horizontal direction in the captured image frame 10. Say. Further, the left pixel value means a pixel value of a left pixel adjacent to the target pixel. In the present embodiment, edge points are detected by comparing adjacent pixel values.

  As shown in FIG. 1A, when the target pixel value and the left pixel value are different, the amount of change is illustrated as an edge point on the lower graph (see points A1 to A4). Edge points are detected based on the pixel value change amount (image feature amount), and the edge coordinates are detected accordingly. As the pixel value changes abruptly, the degree of change in the edge point appears more prominently.

  Note that the case where the host vehicle is traveling in a curve (see FIG. 1B) is the same as the case where the host vehicle is traveling straight (see FIG. 1A), and thus detailed description thereof is omitted. .

  Here, consider a case where the driver changes the steering angle by switching from straight traveling to curve traveling (turning traveling). As shown in FIGS. 1A and 1B, the corresponding lane widths (m1 and m3, m2 and m4) in the image frame 10 taken by the camera change. That is, the position (edge coordinate) of the edge point before and after the steering angle change also changes. Specifically, the x and y coordinates of the corresponding edge points (A1 and A5, A2 and A6, A3 and A7, A4 and A8) are different. Therefore, when a lane such as a white line is projected by the camera, the outside environment recognition system recognizes this as a lane.

  By the way, the case where the thing 15 (object) is put on the dashboard etc. in the own vehicle is also assumed. In such a case, there is a possibility that the object 15 is projected on the windshield, and an extra contrast is generated within the shooting range of the camera, and the outside environment recognition system misrecognizes this contrast as a lane. There is a risk of it.

  However, the relative positional relationship of the objects 15 placed on the dashboard hardly changes depending on the turning state (straight traveling or curve traveling) of the host vehicle. For this reason, the position of the edge point (edge coordinates) in the contrast portion (image feature amount) also substantially matches before and after the change of the steering angle. When the edge coordinates are compared between the straight traveling and the curve traveling, as shown in FIGS. 2A and 2B, the edge points are illustrated at the same position in the lower graph (points A9 to A12). reference). Specifically, the x and y coordinates of the corresponding edge points A9 and A11 have the same value, and the x and y coordinates of the A10 and A12 have the same value. In this case, the image determination apparatus determines that “there is a reflected object”.

  Therefore, according to the image determination apparatus according to the present embodiment, the vehicle environment recognition system does not erroneously recognize the contrast due to the object 15 being projected as a lane.

  Note that the method for detecting the edge point of the object 15 is basically the same as the method for detecting the edge point of the lane, and a detailed description thereof will be omitted.

(Control processing by the control device)
Next, the flow of control processing of the control device will be described with reference to the flowchart of FIG. FIG. 3 is a flowchart illustrating the processing operation of the control device of the image determination device according to the embodiment.

  First, the current traveling speed of the host vehicle is read. This reading is performed by transmitting wheel speed information from the vehicle speed sensor to the control device. Then, the control device determines whether or not the traveling speed of the host vehicle is equal to or higher than a predetermined speed (step S1). In the present embodiment, the predetermined speed is set to 50 km / h.

  When the traveling speed of the host vehicle is 50 km / h or more (Yes in Step S1), it is determined whether or not a change is recognized in the steering angle or yaw rate of the steering wheel of the host vehicle by the control device provided in the image determination device ( Step S2). The steering angle or yaw rate of the steering wheel of the host vehicle is detected by a steering sensor or a yaw rate sensor connected to the control device. By transmitting these pieces of detection information to the control device, it is determined whether the steering angle of the steering wheel of the host vehicle is changing every moment. This determination is performed by the turning state detection means provided in the control device.

  For example, if it is determined by the turning state detection means that the steering angle of the steering wheel of the host vehicle has changed (Yes in step S2), the edge point of the captured image frame before and after the steering angle of the steering wheel of the host vehicle changes is detected. The position (edge coordinates) is detected. Specifically, edge coordinate detection (step S3) of a past captured image frame (hereinafter referred to as “past frame”) by the camera before the steering angle change, and a current captured image frame (by the camera after the steering angle change) The edge coordinates (hereinafter “current frame”) are detected (step S4). The edge coordinate detection method is as described above. Detection of these edge coordinates is performed by a feature amount detection means provided in the control device.

  Subsequently, the edge coordinates of the current and past frames are compared (step S5). When it is determined that there is no difference in the edge coordinates or the difference is very small (Yes in step S5), the object 15 is placed on the dashboard or the like in the own vehicle. It is determined as “Yes”. This is because the position of the object 15 often does not change depending on the turning state. The determination of “there is a reflection object” is notified to the driver by the notification means provided in the control device (step S6).

  The control device is further provided with a feature amount difference calculation means, and the comparison of the edge coordinates of the current and past frames is performed by the feature amount difference calculation means. Specifically, it is performed by calculating a difference value between the edge coordinates of the past frame with respect to the edge coordinates of the current frame. The edge coordinates of the current and past frames are compared depending on the presence or absence of the difference value. This calculation is performed by calculating a difference value between the x and y coordinates of the corresponding edge point before and after the change of the steering angle at each edge point (see points A1 to A12 in FIGS. 1 and 2).

  On the other hand, as a result of comparing the edge coordinates of the current and past frames, if it is determined that there is a change in the edge coordinates (No in step S5), the image determination apparatus determines that there is a reflection object. Instead, the process ends.

If it is determined that there is no change in the steering angle of the steering wheel of the host vehicle (No in step S2), it is considered that the change in the scenery outside the vehicle is small, and therefore the edge coordinates of the current and past frames are detected. If the traveling speed of the host vehicle is less than 50 km / h (No in step S1), the vehicle exterior environment recognition system is in a stopped state. The process is terminated without determining whether there is an object. This is because there is little change in the environment outside the vehicle. For example, when the host vehicle is stopped and the vehicle speed information from the vehicle speed sensor indicates 0 km / h, the environment outside the vehicle does not change, so the turning state is determined by the image determination device, the determination is “There is a reflection object”, etc. The process is terminated without performing the process.

  Since it is configured as described above, according to the image determination device according to the present embodiment, the reflection of the object 15 caused by the reflection of the front window or the adhering matter attached to the lens of the camera can be changed to the lane or The recognition accuracy of the environment recognition system outside the vehicle is not deteriorated because it is mistakenly recognized as a roadside belt or the like. For this reason, it can prevent effectively that a vehicle exterior environment recognition system recognizes incorrectly.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications can be made to the above-described embodiments without departing from the scope of the present invention. And substitutions can be added.

10 shot image frame 15 object

Claims (1)

An image determination device for determining the presence or absence of deposits or reflections in a captured image of a camera,
Feature amount detecting means for detecting pixel points of image feature amounts from image data in the captured image, and turning state detecting means for detecting the turning state of the host vehicle,
When the difference between the position of the pixel point of the image feature amount when the host vehicle is traveling straight and the position of the pixel point of the image feature amount when the host vehicle is turning is less than or equal to a predetermined value An image determination apparatus that determines that there is an attached object or a reflection in the photographed image.

Images