JP2009237781A - Image processor for vehicle, vehicle, and image processing program for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor for a vehicle, the vehicle and an image processing program for a vehicle, which precisely detect a lane mark on a road where the vehicle travels based on an image obtained by processing a surrounding image of the vehicle to reduce an influence of fluctuation in brightness and color on a road face. <P>SOLUTION: This image processor 10 for the vehicle is provided with: a kernel setting means 12 for setting a plurality of kernels for smoothing having a width supposed to be between a width of the lane mark and a width of the road, in the image acquired through an imaging means 2; a smoothing means 13 for smoothing the acquired image by filtering processing using the set kernels for smoothing; a change degree calculation means 14 for calculating a change in a pixel value of each pixel, between the acquired image and a smoothed image; and a pixel value replacement means 15 for converting the pixel value of the pixel having a predetermined value or less of change, in the acquired image, into a predetermined fixed value. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an apparatus for processing an image around a vehicle acquired via an imaging unit, a vehicle, and a program for causing a computer to execute processing of the apparatus.

  Conventionally, a technique for processing an image obtained by imaging the periphery of a vehicle by an imaging unit and recognizing a lane mark such as a white line on a road on which the vehicle travels and an object such as a pedestrian or another vehicle existing around the vehicle Is known (see, for example, Patent Document 1). This technology makes it possible to present information to the driver and to control the driving of the vehicle according to the recognition result of the lane mark and the object.

In the vehicle environment recognition device of Patent Document 1, an image in front of the vehicle captured by a camera mounted on the vehicle is input for each processing cycle, a window to be processed is set in the image, and the lower end portion of the window Is defined as a reference color corresponding to the road surface area (an area other than the white line in the road), and the reference color is similar to the reference color (the color similarity is greater than a predetermined value). Separate road surface areas (white lines and areas outside the road). The apparatus detects a white line using a straight line obtained from the left and right boundaries of the separated road surface area as an edge portion of the white line. At this time, one window is set in the image using a fixed value based on the position of the two white lines detected in the previous processing cycle or predetermined road information such as the road width and the driving lane width. The
Japanese Patent Laid-Open No. 7-200997

  However, on the road surface, for example, there may be a road surface repair mark or a local shadow. In many cases, the repaired trace is locally different in color from the surrounding road surface. In addition, a local shadow portion is also irradiated only by scattered light, and the color is added, so that the color is separated from the surrounding road surface. For this reason, in the apparatus of Patent Document 1, a portion corresponding to a local repair trace or shadow in the window is separated from the reference color and separated as a non-road surface area, and a white line is detected according to the boundary. Therefore, there is a possibility of false detection. This also applies to the luminance, not the color.

  Further, the color of the road surface may be distributed in the road surface region due to, for example, a change over time, or may be distributed in the road surface region, which is not originally made constant depending on the country. For this reason, it is assumed that there is a wide range of partial areas having different colors from the surroundings in the road surface area. When there is such a partial region, it is difficult to separate the road surface region and the non-road surface region using the color of the lower end portion of the window as the reference color of the road surface as in the device of Patent Document 1. In other words, if the color is not constant within the road surface area, it is difficult to properly set the reference color and the conditions that approximate the reference color to the color, and if the setting is loose, the lane mark is included in the road surface area. If severe, this partial area is separated as a non-road surface area. If the boundary between the road surface area and the non-road surface area is erroneously obtained as described above, a white line or a preceding vehicle is detected according to the erroneously obtained boundary, and there is a possibility of erroneous detection. This also applies to the luminance, not the color.

  The present invention has been made in view of such a background, and an image obtained by imaging the periphery of a vehicle is processed so as to reduce the influence of variations in luminance and color on the road surface, and the vehicle travels from the image. An object of the present invention is to provide a vehicle image processing apparatus, a vehicle, and a vehicle image processing program capable of accurately detecting a lane mark on a road.

  In order to achieve such an object, the vehicle image processing apparatus of the present invention processes an image acquired via an imaging means mounted on a vehicle, and displays a lane mark on a road on which the vehicle is traveling. In the vehicular image processing device to be detected, a kernel whose width in a predetermined direction is within a predetermined range larger than the width in the predetermined direction of the image portion of the lane mark with respect to the image acquired through the imaging unit. Kernel setting means for setting a size, smoothing means for smoothing the acquired image by filter processing using a smoothing kernel of the size set by the kernel setting means, and the smoothing means A degree-of-change calculating means for calculating a degree of change in the pixel value of each pixel of the acquired image with respect to the image after smoothing; and the degree of change of the acquired image is a predetermined value Pixel value replacement means for replacing the pixel value of the lower pixel with a predetermined constant value, and lane mark detection means for detecting the lane mark from the image after the pixel value is replaced by the pixel value replacement means (First invention).

  According to the vehicle image processing apparatus of the first aspect of the invention, the kernel size is set by the kernel setting means, and the image obtained by the smoothing means is filtered using the smoothing kernel of the set size. Smooth by processing. In the smoothing by the smoothing means, for each pixel in the image, a smoothing kernel is arranged with the pixel as the central pixel, and the average of the pixel values of all the pixels included in the range of the smoothing kernel is determined as the center. Processing to obtain the pixel value of the pixel is performed. As a result, the range of change in pixel value in the image is narrowed, and a low-contrast image is obtained.

  At this time, the road image is smoothed by using a kernel having a size set so that the width in the predetermined direction is within a predetermined range larger than the width in the predetermined direction of the image portion of the lane mark by the kernel setting means. Thus, when the kernel is arranged with the pixel of the image portion such as the lane mark, the local repair trace, and the shadow as the central pixel, the image portion of the surrounding road surface is included in the range of the kernel. Therefore, in the image after smoothing, the pixel values of the pixels constituting the image portion of the surrounding road surface are the local repair traces on the road surface, the shadow image portion, and the pixel value constituting the lane mark image portion. The pixel value is close to the pixel value. In addition, for the road surface image portion, the road surface image portion and the size of the partial region are larger than the image portion of the lane mark, including the case where there is a wide range of partial regions that have different pixel values from the surroundings. Since it is assumed to be large, when the kernel is arranged with the pixel of the road surface image portion and the partial region as the central pixel, the image area and partial region of the road surface are included in the kernel range. Therefore, the image portion and the partial area on the road surface are less affected by smoothing, and the pixel value in the image before smoothing is retained even in the smoothed image.

  For this reason, when the change degree calculation means calculates the change degree of the pixel value of each pixel of the acquired image with respect to the image smoothed by the smoothing means, the change degree is calculated from the image portion of the road surface. The image portion of the lane mark having a large value becomes a positive large value, and the value is small (including a negative value) in the local repair trace or the shadow image portion having a pixel value smaller than that of the road surface image portion. In addition, the image portion of the road surface has a small change in pixel value before and after smoothing, including a wide range of partial areas where the pixel values are different from the surroundings, and the change degree is a small value (including a negative value). Therefore, the pixel value replacement means replaces the pixel value of a pixel whose degree of change is equal to or less than a predetermined value in the acquired image with a predetermined constant value. The constant value is, for example, a value (for example, 0) that is sufficiently smaller than a value that the pixel value of the image portion of the lane mark can take. As a result, the pixel value of the image of the road surface including the partial region where the pixel value is different from the surroundings, or the pixel value of the local repair trace or shadow image portion is replaced with a constant value, and the image portion of the lane mark image portion is replaced. Only the pixel value of the pixel is held as it is. Therefore, in the image after the pixel value is replaced by the pixel value replacement means, only the image portion of the lane mark remains. Therefore, the lane mark detection means detects the lane mark using the replaced image. Thus, it becomes possible to detect the lane mark with high accuracy.

  In the vehicular image processing apparatus according to the first aspect of the present invention, the kernel setting means sets the image portion of the predetermined area on the image as the distance of the predetermined area of the road from the vehicle increases. It is preferable to set the size of the kernel so that the size of the kernel is reduced (second invention).

  According to the vehicle image processing apparatus of the second aspect of the invention, the larger the distance from the vehicle in the predetermined area of the road in the real space, the smaller the size of the image portion of the predetermined area on the image. Therefore, by setting the kernel size so that the larger the distance from the vehicle in the predetermined area of the road, the smaller the kernel size set for the image portion of the predetermined area on the image, In each of the above positions, if the kernel is placed with the pixel of the image part such as the lane mark, local repair trace, shadow, etc. as the central pixel, make sure that the kernel area contains many image parts of the surrounding road surface be able to.

  In the vehicular image processing apparatus according to the first or second invention, the kernel setting means preferably sets the shape of the kernel to a rectangle or a trapezoid (third invention).

  According to the image processing apparatus for a vehicle of the third invention, the shape of the lane mark such as a white line or a yellow line on the road on which the vehicle is traveling is linear. Therefore, by setting the smoothing kernel to a rectangle or trapezoid, the smoothing kernel can be arranged so as to match the shape of the image portion of the lane mark.

  In the image processing apparatus for a vehicle according to the third aspect of the present invention, the kernel setting means is arranged so that the inclination of the center line of the long side of the kernel with respect to the direction on the image corresponding to the traveling direction of the vehicle becomes a predetermined angle. It is preferable to set the orientation of the kernel (fourth invention).

  According to the vehicle image processing device of the fourth aspect of the invention, the lane mark on the road on which the vehicle is traveling is imaged so as to go to the vanishing point in the direction on the image corresponding to the traveling direction of the vehicle on the image. It is assumed. Therefore, by setting the kernel direction so that the inclination of the center line of the long side of the kernel with respect to the direction on the image corresponding to the traveling direction of the vehicle becomes a predetermined angle, the direction of the image portion of the lane mark is set. A smoothing kernel can be set to match.

  In the vehicular image processing device according to any one of the first to fourth inventions, the change degree calculating means subtracts the smoothed image from the acquired image as the change degree. It is preferable to use a pixel value difference of each pixel obtained or a ratio of pixel values of each pixel obtained by dividing the acquired image by the smoothed image (fifth invention).

  According to the vehicle image processing device of the fifth aspect of the present invention, the difference between the pixel values of each pixel obtained by subtracting the smoothed image from the acquired image, or after the acquired image is smoothed The pixel value ratio of each pixel obtained by dividing by the image is a value indicating the degree of change of the pixel value. Therefore, the pixel value replacement means replaces the pixel value of the image using these differences or ratios. Can be done appropriately.

  In the vehicular image processing device according to any one of the first to fifth inventions, the image smoothed by the smoothing means is preferably an image having luminance as a pixel value (sixth invention). .

  According to the vehicle image processing apparatus of the sixth aspect of the present invention, there are a bright white or yellow lane mark image portion, a dark gray or brown road surface image portion, and a darker local repair trace or shadow image portion. Since the brightness is assumed to be different, the pixel value is replaced so that the local repair trace and the shadow image part are removed and the lane mark image part remains as it is for the image having the brightness as the pixel value. Thus, the image portion of the lane mark can be detected with high accuracy from the image after the replacement.

  Alternatively, in the vehicular image processing device according to any one of the first to fifth inventions, the image smoothed by the smoothing unit has a luminance or saturation calculated from a color component of a color image as a pixel value. It is preferable that the image is (Seventh invention).

  According to the vehicle image processing apparatus of the seventh aspect of the present invention, there are a bright white or yellow lane mark image portion, a dark gray or brown road image portion, and a darker local repair trace or shadow image portion. Since the brightness and saturation will be different, for the image with luminance or saturation as the pixel value, the local repair trace and the shadow image part are removed, and the image part of the lane mark remains as it is. By replacing the pixel value, the image portion of the lane mark can be accurately detected from the replaced image.

  In the vehicular image processing device according to any one of the first to seventh inventions, the vehicle has an area formed by pixels whose pixel values are not the constant value in the image after being replaced by the pixel value replacing means. It is preferable to include lane mark detection means for detecting the image portion of the lane mark on the road that is running (eighth invention).

  According to the vehicle image processing apparatus of the eighth aspect of the invention, since the pixel value of the image portion other than the lane mark is replaced with a constant value in the replaced image, the replacement is performed by the lane mark detection means. A region composed of pixels whose pixel values are not constant in the subsequent image can be detected as the image portion of the lane mark. Therefore, the lane mark detection unit can easily detect the lane mark on the road on which the vehicle is traveling without further performing edge extraction processing or the like.

  Next, a vehicle according to the present invention includes an imaging unit, and has a function of detecting a lane mark on a running road by processing an image acquired via the imaging unit. Kernel setting means for setting a size of a kernel in which a width in a predetermined direction is within a predetermined range larger than a width in the predetermined direction of an image portion of a lane mark with respect to an image acquired via the imaging means; A smoothing unit that smoothes the acquired image by a filtering process using a smoothing kernel of a size set by the kernel setting unit, and an image that has been smoothed by the smoothing unit, A degree-of-change calculating means for calculating a degree of change in the pixel value of each pixel of the acquired image, and a predetermined pixel value of the pixel in which the degree of change of the acquired image is equal to or less than a predetermined value. A pixel value replacing means for replacing the pixel value by the pixel value replacing means is characterized by comprising a lane mark detection means for detecting the lane mark from the image after being replaced (9th invention).

  According to the vehicle of the ninth aspect, as described with respect to the image processing apparatus for a vehicle of the first aspect, the predetermined range in which the width in the predetermined direction is larger than the width in the predetermined direction of the image portion of the lane mark by the kernel setting means. The size of the kernel is set so as to be within, and the obtained image is smoothed by filtering using the smoothing kernel of the set size, so that the image before smoothing is In the smoothed image, the pixel values of the local repair traces and shadows on the road surface, the pixel values of the lane mark image part, and the pixel values of the image part of the surrounding road surface are Become. In addition, with respect to the image portion of the road surface, the pixel value in the image before smoothing is also included in the image after smoothing, including a wide range of partial areas whose pixel values are different from the surroundings in the image portion of the road surface. Retained. Therefore, in the image after the pixel value has been replaced by the pixel value replacement means, the pixel portion of the pixel of the road surface image portion including the partial region where the pixel value is different from the surroundings, or the local repair trace or shadow image portion pixel Since the value is replaced with a constant value and only the image portion of the lane mark remains, the lane mark can be detected with high accuracy by detecting the lane mark using the image after the replacement.

  Next, the image processing program for a vehicle according to the present invention processes the image acquired through the imaging means mounted on the vehicle and detects the lane mark on the road on which the vehicle is traveling. An image processing program for a vehicle to be executed in a predetermined range in which a width in a predetermined direction is larger than a width in the predetermined direction of an image portion of a lane mark with respect to an image acquired via the imaging unit A kernel setting process for setting a size of the kernel, a smoothing process for smoothing the acquired image by a filter process using a smoothing kernel of a size set by the kernel setting process, and the smoothing A change degree calculation process for calculating a change degree of a pixel value of each pixel of the acquired image with respect to the image after being smoothed by the conversion process; and A pixel value replacement process for replacing a pixel value of a pixel whose change degree is a predetermined value or less with a predetermined constant value, and the lane mark from the image after the pixel value is replaced by the pixel value replacement process. It has a function of causing a computer to execute lane mark detection processing to detect (tenth invention).

  According to the vehicle image processing program of the tenth aspect of the invention, it is possible to cause a computer to execute a process that can achieve the effects described in regard to the first aspect of the invention.

  An embodiment of the present invention will be described with reference to FIGS. Referring to FIG. 1, a vehicle image processing apparatus 10 according to an embodiment of the present invention is used by being mounted on a host vehicle 1 (a vehicle of the present invention), and is captured by a camera 2 that captures the front of the host vehicle 1. The lane mark on the road on which the host vehicle 1 is traveling is detected from the image. The camera 2 (imaging means of the present invention) is a CCD camera or the like, and is attached to the front portion of the host vehicle 1.

  The position data of the lane mark detected by the vehicle image processing apparatus 10 is output to an ECU (Electronic Control Unit) 20 of the vehicle 1. The ECU 20 recognizes the lane of the road on which the host vehicle 1 is traveling based on the position data of the lane mark, determines the possibility that the host vehicle 1 will depart from the lane, and the host vehicle 1 A warning output process for performing a warning output (by voice output from a speaker (not shown), etc.) and a deviation from the lane of the host vehicle 1 are avoided when there is a possibility of departure from the lane. In addition, a lane departure avoidance control process for assisting driving of the brake and steering of the host vehicle 1 is executed.

  Next, referring to FIG. 2, the vehicle image processing apparatus 10 functions as an image acquisition unit 11, a kernel setting unit 12, a smoothing unit 13, a change degree calculation unit 14, and a pixel value replacement. Means 15 and lane mark detection means 16 are provided.

  The image acquisition unit 11 receives an image signal output from the camera 2 and acquires a color image composed of pixel data from the input image signal. At this time, the pixel data is composed of color components composed of an R value, a G value, and a B value. Then, the image acquisition unit 11 calculates the luminance Y of the pixel from the color components (R, G, B) of the acquired color image, and acquires a luminance image having the luminance Y as a pixel value. At the same time, the image acquisition unit 11 calculates the saturation S of the pixel from the color components (R, G, B) of the pixel of the acquired color image, and acquires a saturation image using the saturation S as a pixel value. To do.

  The kernel setting unit 12 sets the size of the kernel in which the width in the predetermined direction is within a predetermined range larger than the width in the predetermined direction of the image portion of the lane mark for the image acquired via the camera 2. . Details of the setting of the smoothing kernel will be described later.

  The smoothing unit 13 smoothes the luminance image and the saturation image acquired via the camera 2 by filter processing using a smoothing kernel having a size set by the kernel setting unit 12. In the smoothing by the smoothing means 13, for each pixel in the image, a smoothing kernel is arranged with the pixel as the central pixel, and the average of the pixel values of all the pixels included in the range of the kernel is determined as the pixel of the central pixel. Processing to make a value is performed.

  The degree-of-change calculating means 14 uses the luminance image to calculate the degree of change in the pixel value of each pixel of the luminance image acquired via the camera 2 with respect to the luminance image smoothed by the smoothing means 13. To do. At the same time, the degree-of-change calculating means 14 uses the saturation image, and the pixel of each pixel of the saturation image acquired via the camera 2 with respect to the saturation image smoothed by the smoothing means 13. The degree of change in value is calculated. Specifically, the change degree calculation unit 14 subtracts the smoothed luminance image (saturation image) from the acquired luminance image (saturation image) as the change degree. Use the difference in pixel values.

  The pixel value replacement unit 15 determines in advance a pixel value of a pixel whose change degree calculated using the brightness image by the change degree calculation unit 14 of the brightness image acquired via the camera 2 is equal to or less than a predetermined value. Replace with a constant value. At the same time, the pixel value replacement means 15 is a pixel value of a pixel in which the degree of change calculated using the saturation image by the degree-of-change calculation means 14 of the saturation image acquired via the camera 2 is equal to or less than a predetermined value. Is replaced with a predetermined constant value (for example, 0).

  The lane mark detection unit 16 detects a lane mark on the road on which the host vehicle 1 is traveling from the luminance image and the saturation image after being replaced by the pixel value replacement unit 15. Specifically, the lane mark detection means 16 is a lane mark on the road on which the vehicle 1 is traveling in an area composed of pixels with non-constant pixel values in the replaced luminance image and saturation image. Is detected as an image portion of. At this time, the lane mark detection means 16 extracts a linear component as a lane mark candidate by Hough transform or the like from the point sequence data composed of pixels whose pixel values are not constant in the luminance image after replacement. At the same time, the lane mark detection unit 16 extracts a linear component as a lane mark candidate by Hough transform or the like from the point sequence data composed of pixels whose pixel values are not constant in the replaced saturation image. The lane mark detection means 16 detects a lane mark that defines the lane of the road on which the host vehicle 1 is traveling based on the position on the image and the like from among the extracted lane mark candidates.

  The vehicle image processing apparatus 10 has an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, and accesses (reads and writes) data stored in the image memory. A computer having an interface circuit for performing various kinds of arithmetic processing on an image stored in the image memory (an arithmetic processing circuit comprising a CPU, a memory, an input / output circuit, etc., or a combination of these functions) An electronic unit composed of a microcomputer. The vehicle image processing apparatus 10 causes the computer to execute the vehicle recognition program of the present invention so that the computer acquires the image acquisition means 11, the kernel setting means 12, the smoothing means 13, and the change degree calculation means. 14, pixel value replacement means 15, and lane mark detection means 16.

  Next, the operation (vehicle image processing) of the vehicle image processing apparatus 10 will be described with reference to the flowchart shown in FIG. The process according to the flowchart shown in FIG. 3 is repeatedly executed every control cycle of the vehicle image processing apparatus 10.

In STEP 1 of FIG. 3, the image acquisition unit 11 acquires a color image via the camera 2. The acquired color image is A / D converted and stored in the image memory. FIG. 4 illustrates a color image I ₁ acquired via the camera 2 at a certain control cycle time. The color image I ₁ is an image composed of m × n pixels as illustrated in FIG. Each pixel P ₁ of the color image I ₁ has an R value, a G value, and a B value as color components, and is represented by P ₁ (i, j) = (R _ij , G _ij , B _ij ). . Here, i and j are the coordinates of each pixel, and are integers satisfying 0 ≦ i <m and 0 ≦ j <n.

  In the example of FIG. 4, the host vehicle 1 is traveling in the direction of the arrow, the lane mark that defines the left side of the lane of the road on which the host vehicle 1 is traveling is the solid yellow line A1, and the lane that defines the right side. A case where the mark is a broken white line A2 is shown. At this time, on the road surface of the lane of the road on which the host vehicle 1 is traveling, there is a linear repair mark (black line) B1 darker than the surroundings and a black spot group B2 due to local unevenness and dirt. In the road surface area, there is a wide partial area C1 whose pixel values are different from those of the surrounding area.

Next, in STEP 2, the image acquisition unit 11 calculates the luminance from the color components (R, G, B) of the pixels of the color image I ₁ acquired in STEP 1 to acquire the luminance image I ₂ , and the saturation. To obtain a saturation image I ₅ .

Specifically, the image acquisition unit 11 calculates Y _ij = α × R _ij + β × G _ij from the color components (R _ij , G _ij , B _ij ) of each pixel P ₁ (i, j) of the color image I _1. The luminance Y _ij is calculated from + γ × B _ij . However, α, β, and γ are predetermined coefficients such that α + β + γ = 1. Thereby, as illustrated in FIG. 5A, a luminance image I ₂ composed of m × n pixels is obtained with the luminance Y _ij as data of each pixel P ₂ (i, j). In the luminance image I _2, the image portion of the white line A2 has a large brightness, image portion of the yellow line A1 is the luminance is smaller than the white line A2. In addition, the image portion of the road surface has a smaller luminance and an average road surface luminance, and the partial area C1 has a slightly higher luminance than the average road surface luminance. Further, the repair mark B1 and the black spot group B2 have lower luminance than the average road surface.

The image acquisition means 11 uses the maximum value MAX _ij and the minimum value MIN _ij of (R _ij , G _ij , B _ij ) as the luminance Y _ij , and Y _ij = (MAX _ij + MIN _ij ) / 2. You may calculate by. Alternatively, the image acquisition unit 11 may use a G _ij value as the luminance Y _ij .

At the same time, the image acquisition means 11 calculates S _ij = (MAX _ij ) from the R value, G value, and B value (R _ij , G _ij , B _ij ) of each pixel P ₁ (i, j) of the color image I _1. to calculate the saturation S _ij by -MIN _ij) / MAX _ij. As a result, as shown in FIG. 5D, a saturation image I ₅ composed of m × n pixels is obtained with the saturation S _ij as the data of each pixel P ₅ (i, j). . In the saturation image I ₅ , the image portion of the yellow line A1 has high saturation, and the image portion of the white line A2 has lower saturation than the yellow line A1. Further, the image portion of the road surface is further reduced in saturation and has an average road surface saturation, and the partial region C1 has a slightly higher saturation than the average road surface. Further, the repair mark B1 and the sunspot group B2 are less saturated than the average road surface.

Next, in STEP 3, the kernel setting unit 12 sets a plurality of sizes of smoothing kernels for the luminance image I ₂ and the saturation image I ₅ acquired in STEP 2.

Here, referring to FIG. 6, the setting of the smoothing kernel will be described. FIG. 6 illustrates five smoothing kernels K _{1 to} K ₅ set for the luminance image I ₂ . For the saturation image I ₅ , smoothing kernels K _{1 to} K ₅ are set similarly to the luminance image I ₂ . The smoothing kernels K _{1 to} K ₅ are used when smoothing the pixels included in the regions R _{1 to} R ₅ of the luminance image I ₂ as the center pixels, respectively. The vertical size Y _f [Pixel] of the smoothing kernels K _{1 to} K ₅ is set to several pixels (for example, 1 to 3 [Pixel]). Regions R _{1 to} R ₅ are gradually narrowed in the vertical direction as they move away from the host vehicle 1. The size Y _f may be a value that decreases as the distance from the host vehicle 1 increases. The horizontal size X _f [Pixel] of the smoothing kernels K _{1 to} K ₅ is a value corresponding to the predetermined width ΔX in the real space. The predetermined width ΔX is larger than a value assumed to be the width of the lane mark (for example, 0.1 to 0.75 [m]) (for example, several times the width of the lane mark, specifically 0.5 to 1 [m]). Is used. The predetermined width ΔX is a value smaller than a value assumed to be the road width. The horizontal sizes _Xf of the smoothing kernels K _{1 to} K ₅ are, for example, 10, 20, 50, 100, and 150 [Pixel], respectively. The size _Xf is set from the relationship between the real space coordinate system and the image coordinate system of the host vehicle 1 as shown in FIG. As shown in FIG. 7C, the image coordinate system has coordinates (x, y) with the upper left as the origin. In addition, the physical interval of one pixel is (δx, δy).

At this time, as shown in FIG. 7A, the distance Z of the object on the road surface to the host vehicle 1, the focal length f of the camera 2, the height h from the road surface of the camera 2 in real space, The physical height Y _c of the object imaged in is expressed by the following formula (1)
Y _c = f × h / Z (1)
There is a relationship. Further, the physical height Y _c , the image coordinate y of the object imaged on the image, and the physical interval δy per pixel are expressed by the following formula (2).
Y _c = y × δy (2)
There is a relationship.

Further, as shown in FIG. 7B, the distance Z of the object on the road surface to the own vehicle 1, the focal length f of the camera 2, the width ΔX of the object in real space, and the object imaged on the image The physical width X _c of the object is expressed by the following formula (3)
X _c = f × ΔX / Z (3)
There is a relationship. Further, the physical width X _c of the object imaged on the image, the image coordinate x, and the physical interval δx per pixel are expressed by the following equation (4).
X _c = x × δx (4)
There is a relationship.

From the formulas (1) to (4), the size _Xf is
X _f = (ΔX / h) × y (5)
It becomes. Since the width ΔX of the object in the real space and the height h from the road surface of the camera 2 in the real space are fixed values, the size _Xf becomes smaller as the image coordinate y becomes smaller. Furthermore, from the formulas (1) and (2), as the distance Z increases, the image coordinate y decreases, so the size _Xf also decreases. That is, the size _Xf is set to a value that decreases as the distance from the host vehicle 1 increases.

Returning to FIG. 3, next, in STEP 4, the smoothing means 13 smoothes the luminance image I ₂ using the smoothing kernels K _{1 to} K ₅ set in STEP 3. FIG. 5B shows the luminance image I ₃ after smoothing. Thereby, the width of the change in the pixel value in the luminance image I ₂ is narrowed, and a low-contrast image is obtained. In the smoothed luminance image I ₃ , the pixel values of the image portions of the repair trace B1 and the black spot group B2 become pixel values close to the average road surface luminance, and cannot be discriminated from the image portions of the surrounding road surfaces. In the smoothed image I ₃ , the pixel values of the image portions of the yellow line A1 and the white line A2 are also close to the average road surface luminance. Also, the image portion of the road surface and the partial area C1, small influence of larger size for smoothing the image portion of the road surface and the partial area C1 than the size of the kernel, the luminance image I ₃ after smoothed luminance image I The pixel values of the road surface ₂ and the partial area C1 are retained.

Next, the processing of STEP 5 to 7 is executed for all the pixels (i = 1 to m, j = 1 to n) of the luminance image ₂ .

First, in STEP5, the change degree calculation means 14, from the pixel values of the luminance image I ₂ acquired in _{STEP2 P 2 (i, j)} , the pixel value P ₃ of the image I ₃ after smoothed STEP4 (i, j ) Is subtracted to calculate the degree of change ΔI _Y (i, j) = P ₂ (i, j) −P ₃ (i, j) of the pixel value of the pixel (i, j).

Next, in STEP 6, the pixel value replacement unit 15 determines whether or not the change degree ΔI _Y (i, j) of the pixel value is equal to or less than a predetermined value I _Yth . The predetermined value I _Yth is a value (for example, a value equal to or less than 0) that is determined according to a range in which the luminance change degree of the image portion of the lane mark (white line) can take. When the determination result in STEP 6 is YES, the pixel (i, j) has an image portion having a small change degree ΔI _th (i, j) of the pixel value, that is, the luminance of the road surface is smaller than the white line A2 and average road surface luminance. It is assumed that it is an image part of a near yellow line A1, an image part of a road surface (including the partial area C1) whose luminance changes little before and after smoothing, or an image part of the repair mark B1 and the black spot group B2 whose luminance is smaller than the road surface. Is done. Therefore, proceeding to STEP 7, the pixel value replacing unit 15 replaces the pixel value P ₂ (i, j) of the luminance image I ₂ with a constant value, and returns to STEP 5. When the determination result in STEP 6 is NO, it is assumed that the pixel (i, j) is an image portion of the white line A2 having high luminance. Therefore, the pixel value replacing unit 15 returns to STEP 5 as it is without replacing the pixel value of the pixel (i, j).

FIG. 5C shows the replaced luminance image I ₄ obtained as a result of the processing of STEPs 5 to 7 being executed for all the pixels of the luminance image I ₂ . As shown in FIG. 5C, in the luminance image I ₄ after replacement, the image portion of the yellow line A1, the repair mark B1, the sunspot group B2, and the road surface (including the partial area C1) is replaced with a constant value. only the image portion of the white line A2 is kept at the pixel values of the luminance image I _2. Therefore, only the image portion of the white line A2 remains in the replaced image I ₄ .

Next, in STEP 8, the smoothing means 13 smoothes the saturation image I ₅ using the smoothing kernels K _{1 to} K ₅ set in STEP 3. FIG. 5E shows the saturation image I ₆ after smoothing. Thereby, the width of the change in the pixel value in the saturation image I ₅ is narrowed, and a low contrast image is obtained. In the saturation image I _{6 after} smoothing, the pixel values of the image portions of the repair trace B1 and the black spot group B2 become pixel values close to the average saturation of the road surface, and cannot be discriminated from the image portions of the surrounding road surfaces. Yes. In the smoothed image I ₆ , the pixel values of the image portions of the yellow line A1 and the white line A2 are also close to the average road surface saturation. Also, the image portion of the road surface and the partial area C1, small influence of larger size for smoothing the image portion of the road surface and the partial area C1 than the size of the kernel, in the saturation image I ₆ smooth basket, saturation The pixel values of the road surface and the partial area C1 of the image I ₅ are held.

Next, the processing of STEPs 9 to 11 is executed for all pixels (i = 1 to m, j = 1 to n) of the saturation image I ₅ .

First, in STEP 9, the change degree calculating means 14 uses the pixel value P ₆ (smoothed saturation image I _{6 in} STEP 8 from the pixel value P ₅ (i, j) of the saturation image I ₅ acquired in STEP 2. i, j) is subtracted to calculate the degree of change ΔI _S (i, j) = P ₅ (i, j) −P ₆ (i, j) of the pixel value of pixel (i, j).

Next, in STEP 10, the pixel value replacement unit 15 determines whether or not the change degree ΔI _S (i, j) of the pixel value is equal to or less than a predetermined value I _Sth . The predetermined value I _Sth is a value (for example, a value equal to or less than 0) determined according to a range in which the degree of change in saturation of the image portion of the lane mark (yellow line) can be taken. When the determination result in STEP 10 is YES, the pixel (i, j) is an image portion where the degree of change ΔI _S (i, j) of the pixel value is small, that is, the average road surface is smaller in saturation than the yellow line A1. The image portion of the white line A2 close to saturation, the image portion of the road surface (including the partial region C1) in which the change in saturation is small before and after smoothing, or the image portion of the repair mark B1 and the sunspot group B2 whose saturation is smaller than the road surface It is assumed that Therefore, proceeding to STEP 11, the pixel value replacing unit 15 replaces the pixel value P ₅ (i, j) of the saturation image I ₅ with a constant value, and returns to STEP 9. When the determination result in STEP 10 is NO, it is assumed that the pixel (i, j) is an image portion of the yellow line A1 with high saturation. Therefore, the pixel value replacing unit 15 returns to STEP 9 without replacing the pixel value of the pixel (i, j).

FIG. 5F shows a saturation image I ₇ after replacement, which is obtained as a result of executing the processing of STEPs _{9 to 11} for all the pixels of the saturation image I 5. As shown in FIG. 5 (f), the saturation image I ₇ after the replacement, the white line A2, repaired B1, black dots group B2, the image portion of the road surface (partial region C1 included) is replaced by a constant value, only the image portion of the yellow line A1 is kept at the pixel value of the saturation image I _5. Therefore, in the saturation image I ₇ after replacement, only the image portion of the yellow line A1 remains.

Next, proceeding to STEP 12, the lane mark detection means 16 detects a lane mark from the replaced luminance image I ₄ obtained at STEP 7 and the replaced saturation image I ₇ obtained at STEP 11. In the luminance image I ₄ after replacement and the saturation image I ₇ after replacement, only the image portions of the yellow line A1 and white line A2 remain, so the lane mark detection means 16 does not further perform edge extraction processing or the like. The lane mark on the road on which the host vehicle 1 is traveling can be easily detected. At this time, since only the image portion of the white line A2 remains in the replaced luminance image I ₄ , the lane mark detection unit 16 detects the lane mark using the replaced luminance image I ₄ . The white line A2 is detected with high accuracy. Further, the saturation image I ₇ after the replacement, so there remains only the image portion of the yellow line A1, the lane mark detecting unit 16, detecting the lane mark by using the saturation image I ₇ after the replacement Thus, the yellow line A1 is detected with high accuracy.

  The above is the vehicle image processing in the vehicle image processing apparatus 10 of the present embodiment. According to the present embodiment, an image obtained by capturing the periphery of the host vehicle 1 is processed so as to reduce the influence of luminance and color variations in the image, and the host vehicle 1 is traveling from the image. Lane marks A1 and A2 on the road can be detected with high accuracy.

In the present embodiment, the degree-of-change calculating unit 14 uses the difference in pixel value as the degree of change. However, in other embodiments, the degree of change is obtained by dividing the acquired image by the smoothed image. The ratio of the pixel values of the obtained pixels may be used (ΔI _Y (i, j) = P ₂ (i, j) / P ₃ (i, j), ΔI _S (i, j) = P ₅ (I, j) / P ₆ (i, j)). In this case, the predetermined values I _Yth and I _Sth are values (for example, values of 1 or less) that are determined according to a range in which the degree of change in the pixel value of the image portion of the lane mark can be taken.

In the present embodiment, as shown in FIG. 8, the smoothing kernel has a predetermined inclination of the center line L of the long side of the kernel with respect to the direction L _c on the image corresponding to the traveling direction of the host vehicle 1. The orientation of the kernel may be set so as to be an angle (for example, 45 °).

  In this embodiment, the shape of the smoothing kernel is rectangular. However, as another embodiment, the shape of the smoothing kernel may be trapezoidal.

  In the present embodiment, the image acquisition unit 11 acquires a color image composed of color components whose pixel data includes R value, G value, and B value. However, as another embodiment, pixel data May acquire a color image composed of color components including CMY outputs.

  In this embodiment, smoothing processing and pixel value replacement processing are performed for each of the luminance image and the saturation image, and the lane mark is detected from the luminance image and the saturation image after the replacement. One image may be subjected to smoothing processing and pixel value replacement processing, and lane marks may be detected from the image after the replacement.

  In this embodiment, a color image is acquired via the camera 2. However, as another embodiment, a grayscale image (luminance image) in which pixel data is composed of luminance may be acquired. . In this case, smoothing processing and pixel value replacement processing are performed on the acquired luminance image, and a lane mark is detected from the image after the replacement.

  Moreover, in this embodiment, the front of the own vehicle 1 is imaged with the camera 2, and the vehicle image processing apparatus 10 detects the lane mark from the image of the road ahead of the own vehicle 1, but other embodiments. As an example, the rear of the host vehicle 1 may be captured by a camera, and the vehicle image processing apparatus 10 may detect a lane mark from an image of a road behind the host vehicle 1.

Explanatory drawing which showed the mounting aspect to the own vehicle of the image processing apparatus for vehicles of this invention. The block diagram of the image processing apparatus for vehicles of this invention. The flowchart which shows the image process by the image processing apparatus for vehicles shown in FIG. 4 is a processing image example of the image processing in FIG. 3. 4 is a processing image example of the image processing in FIG. 3. FIG. 4 is an explanatory diagram regarding setting of a smoothing kernel in the image processing of FIG. 3. Explanatory drawing regarding the size of the kernel for smoothing of FIG. Explanatory drawing regarding the setting of the kernel for smoothing in the image processing apparatus for vehicles of other embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Vehicle (own vehicle), 2 ... Camera (imaging means), 10 ... Image processing apparatus for vehicles, 11 ... Image acquisition means, 12 ... Kernel setting means, 13 ... Smoothing means, 14 ... Change degree calculation means, 15 ... pixel value replacement means, 16 ... lane mark detection means.

Claims (10)

In an image processing apparatus for a vehicle that processes an image acquired via an imaging unit mounted on a vehicle and detects a lane mark on a road on which the vehicle is traveling,
Kernel setting means for setting a size of a kernel in which a width in a predetermined direction is within a predetermined range larger than a width in the predetermined direction of an image portion of a lane mark with respect to an image acquired via the imaging means;
Smoothing means for smoothing the acquired image by filter processing using a smoothing kernel of a size set by the kernel setting means;
A degree-of-change calculating means for calculating the degree of change of the pixel value of each pixel of the acquired image with respect to the image smoothed by the smoothing means;
Pixel value replacement means for replacing a pixel value of a pixel in which the degree of change of the acquired image is a predetermined value or less with a predetermined constant value;
A vehicle image processing apparatus comprising: a lane mark detection unit that detects the lane mark from an image after the pixel value is replaced by the pixel value replacement unit. The vehicle image processing device according to claim 1,
The kernel setting means is configured to reduce the size of the kernel set for the image portion of the predetermined area on the image as the distance of the predetermined area of the road from the vehicle increases. An image processing apparatus for a vehicle characterized by setting a size. The image processing apparatus for a vehicle according to claim 1 or 2,
The vehicle image processing apparatus, wherein the kernel setting means sets the shape of the kernel to a rectangle or a trapezoid. The vehicle image processing device according to claim 3,
The kernel setting means sets the orientation of the kernel such that the inclination of the center line of the long side of the kernel with respect to the direction on the image corresponding to the traveling direction of the vehicle is a predetermined angle. An image processing apparatus for a vehicle. In the vehicle image processing device according to any one of claims 1 to 4,
The degree-of-change calculating means, as the degree of change, calculates a difference between pixel values of pixels obtained by subtracting the smoothed image from the acquired image, or smoothes the acquired image. A vehicular image processing apparatus using a ratio of pixel values of pixels obtained by division by an image after being processed. In the vehicular image processing device according to any one of claims 1 to 5,
The vehicle image processing apparatus, wherein the image smoothed by the smoothing means is an image having luminance as a pixel value. In the vehicular image processing device according to any one of claims 1 to 5,
The vehicle image processing apparatus, wherein the image smoothed by the smoothing means is an image having luminance or saturation calculated from a color component of a color image as a pixel value. In the vehicular image processing device according to any one of claims 1 to 7,
Lane mark detection means for detecting, as an image portion of a lane mark on a road on which the vehicle is running, an area composed of pixels whose pixel values are not the constant value of the image after being replaced by the pixel value replacement means. An image processing apparatus for a vehicle, comprising: A vehicle having an imaging means, having a function of detecting a lane mark on a running road by processing an image acquired through the imaging means;
Kernel setting means for setting a size of a kernel in which a width in a predetermined direction is within a predetermined range larger than a width in the predetermined direction of an image portion of a lane mark with respect to an image acquired via the imaging means;
Smoothing means for smoothing the acquired image by filter processing using a smoothing kernel of a size set by the kernel setting means;
A degree-of-change calculating means for calculating the degree of change of the pixel value of each pixel of the acquired image with respect to the image smoothed by the smoothing means;
Pixel value replacement means for replacing a pixel value of a pixel in which the degree of change of the acquired image is a predetermined value or less with a predetermined constant value;
A vehicle comprising: lane mark detection means for detecting the lane mark from an image after the pixel value is replaced by the pixel value replacement means. A vehicle image processing program for processing an image acquired via an imaging means mounted on a vehicle and causing a computer to execute a process of detecting a lane mark on a road on which the vehicle is running.
A kernel setting process for setting a size of a kernel in which a width in a predetermined direction is within a predetermined range larger than a width in the predetermined direction of an image portion of a lane mark for an image acquired via the imaging unit;
A smoothing process for smoothing the acquired image by a filter process using a smoothing kernel of a size set by the kernel setting process;
A change degree calculation process for calculating a change degree of a pixel value of each pixel of the acquired image with respect to the image smoothed by the smoothing process;
A pixel value replacement process for replacing a pixel value of a pixel in which the degree of change of the acquired image is a predetermined value or less, with a predetermined constant value;
An image processing program for a vehicle having a function of causing a computer to execute a lane mark detection process for detecting the lane mark from an image after a pixel value is replaced by the pixel value replacement process.