JP2015001812A - Image processing device, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device with a simple configuration capable of precisely determining whether a vehicle is travelling along a lane marker.SOLUTION: An image processing device 100 includes: an image acquisition section 101 that acquires a road surface image which is an image of a road including a lane marker by taking an image from a running vehicle; a motion blur calculation section 102 that calculates the width of a motion blur which is generated in a boundary area between the lane marker and an area other than the lane marker in a partial area of the road surface image; and a travelling state determination section 103 that determines, when the width of the motion blur is smaller than a threshold value, the vehicle is travelling along the lane marker.

Description

  The present invention relates to calibration of an in-vehicle camera, position / posture display of the own vehicle in a car navigation system, lane departure warning in a safety image processing system, lane keeping assistance, and an automatic driving system. The present invention relates to an image processing apparatus, an image processing method, and an image processing program that accurately determine that a vehicle is traveling along a lane marker such as a white line in steering control, formation of formation in an automated driving / convoy running system, and steering control.

  As a conventional image processing device, the vehicle-mounted camera images are integrated while the vehicle goes straight, the intersection point of the straight line group determined based on the luminance edge extracted from the integrated image is specified as the vanishing point, and the vanishing point coordinates and the reference coordinates are used. In order to determine whether the camera parameter is an appropriate value, there is known one that determines that the vehicle is traveling straight using a steering angle sensor (see, for example, Patent Document 1).

International Publication No. 2010/146695

  However, the technique of Patent Document 1 has a problem that the configuration is complicated because a steering angle sensor is required to determine that the vehicle is traveling straight.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of accurately determining that a vehicle is traveling along a lane marker with a simpler configuration.

  An image processing apparatus according to an aspect of the present invention includes an image acquisition unit that acquires a first road surface image that is an image of a road including a lane marker by photographing from a traveling vehicle, and the first road surface image. A motion blur calculation unit that calculates the width of a motion blur that is generated based on the lane marker, and a traveling state that determines that the vehicle is traveling along the lane marker when the width of the motion blur is smaller than a threshold value And a determination unit.

  An image processing method according to an aspect of the present invention includes a step of acquiring a road surface image that is an image of a road including a lane marker by photographing from a traveling vehicle, and a motion generated based on the lane marker from the road surface image. Calculating a blur width, and determining that the vehicle is traveling along the lane marker when the motion blur width is smaller than a threshold value.

  An image processing program according to an aspect of the present invention includes a process of acquiring a road surface image that is an image of a road including a lane marker by shooting from a traveling vehicle, and a motion that is generated based on the lane marker from the road surface image. Processing for calculating the width of the blur and processing for determining that the vehicle is traveling along the lane marker when the width of the motion blur is smaller than a threshold value are executed by the computer.

  The present invention can accurately determine that the vehicle is traveling along the lane marker with a simpler configuration.

1 is a block diagram illustrating a configuration example of an image processing apparatus according to Embodiment 1 of the present invention. The top view which shows the example of the driving | running | working state which the vehicle which concerns on Embodiment 1 of this invention drive | works along a lane marker The top view which shows the example of the driving | running | working state which the vehicle which concerns on Embodiment 1 of this invention is drive | working along a lane marker 7 is a flowchart illustrating an operation example of the image processing apparatus according to the first embodiment of the present invention. The figure which each shows the example and the enlarged image which were image | photographed when the vehicle which concerns on Embodiment 1 of this invention is driving | running along a lane marker The figure which each shows the example and the enlarged image which were image | photographed when the vehicle which concerns on Embodiment 1 of this invention is drive | working along a lane marker The graph which shows the luminance value for every pixel which the image processing apparatus which concerns on Embodiment 1 of this invention can calculate FIG. 3 is a block diagram showing a configuration example of an image processing apparatus according to Embodiment 2 of the present invention. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 2 of this invention. The graph which shows the luminance value for every pixel which the image processing apparatus which concerns on Embodiment 2 of this invention can calculate FIG. 3 is a block diagram showing an example of the configuration of an image processing apparatus according to Embodiment 3 of the present invention. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 3 of this invention. The graph which shows the luminance value for every pixel which the image processing apparatus which concerns on Embodiment 3 of this invention can calculate FIG. 6 is a block diagram showing a configuration example of an image processing apparatus according to Embodiment 4 of the present invention. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 4 of this invention. FIG. 9 is a block diagram showing a configuration example of an image processing apparatus according to Embodiment 5 of the present invention. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 5 of this invention. The side view which shows the example of the driving | running | working state which the vehicle which concerns on Embodiment 5 of this invention drive | works without a pitch angle fluctuation The side view which shows the example of the driving | running | working state which the vehicle which concerns on Embodiment 5 of this invention is driving | running, changing a pitch angle -1 degree. The side view which shows the example of the driving | running | working state which the vehicle which concerns on Embodiment 5 of this invention is driving | running, changing a pitch angle + 1 degree | times. The figure which each shows the example and the enlarged image which were image | photographed when the vehicle which concerns on Embodiment 5 of this invention is drive | working without a pitch angle fluctuation | variation The figure which each shows the example and the enlarged image which were image | photographed when the vehicle which concerns on Embodiment 5 of this invention is drive | working changing a pitch angle -1 degree | times. The figure which each shows the example and the enlarged image which were image | photographed when the vehicle which concerns on Embodiment 5 of this invention is drive | working changing a pitch angle + 1 degree | times. Graph showing luminance values for each pixel that can be calculated by the image processing apparatus according to the fifth embodiment of the present invention. Graph showing luminance values for each pixel that can be calculated by the image processing apparatus according to the fifth embodiment of the present invention. FIG. 9 is a block diagram illustrating a configuration example of an image processing apparatus according to a sixth embodiment of the present invention. The flowchart which shows the operation example of the image processing apparatus which concerns on Embodiment 6 of this invention. The figure which shows the example of the image used as the object imaged by the image processing apparatus which concerns on Embodiment 6 of this invention, the imaged image, and the image-processed image

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
An image processing apparatus according to Embodiment 1 of the present invention will be described.

  First, the configuration of the image processing apparatus 100 according to the present embodiment will be described. FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus 100 according to the present embodiment.

  In FIG. 1, the image processing apparatus 100 includes an image acquisition unit 101, a motion blur calculation unit 102, and a running state determination unit 103.

  The image acquisition unit 101 acquires a road surface image captured by an in-vehicle camera (hereinafter also referred to as “camera” as appropriate). A road surface image is an image of a road including a lane marker. The lane marker referred to in this specification means a display or an object that can specify a lane (lane), and examples thereof include a white line, a botsdot, a pole, a guardrail, and the like. Hereinafter, a case where the lane marker is a white line will be described as an example.

  The motion blur calculation unit 102 calculates the width of the motion blur that occurs in the boundary area other than the white line and the white line in the partial area of the road surface image. The original motion blur is subject blur that occurs when a moving subject is shot with a camera that does not move. In this embodiment, when the camera moves along a white line that is a non-moving subject, the motion blur is regarded as the white line moving without moving the camera in the captured road surface image. It is defined as subject blurring that occurs with respect to the subject.

  When a vehicle traveling along a lane marker captures a white line as a subject by an in-vehicle camera, the motion blur of the white line is generated in the extension direction of the white line, which is the traveling direction of the vehicle. For this reason, the luminance component of the white line is superimposed in the white line region and does not spread outside the extending direction of the white line. That is, the luminance change in the horizontal direction of the road surface image in the boundary area between the white line area and the asphalt area where the white line is not applied becomes steep, and motion blur does not occur at the boundary between the white line area and the asphalt area.

  On the other hand, when a vehicle that is traveling along a lane marker captures a white line as a subject by an in-vehicle camera, the motion blur of the white line is generated in the traveling direction of the vehicle, so that the white line is shifted from the extended line direction. Therefore, the luminance component of the white line is superimposed outside the white line region and spreads in directions other than the extending direction of the white line. That is, the horizontal luminance change of the road surface image at the boundary between the white line area and the asphalt area becomes gentle, and motion blur occurs at the boundary between the white line area and the asphalt area.

  The traveling state determination unit 103 determines that the vehicle is traveling along the lane marker when the width of the motion blur calculated by the motion blur calculation unit 102 is smaller than a predetermined threshold. To do.

  Next, an example of a traveling state of a vehicle equipped with the image processing apparatus 100 of the present embodiment will be described. FIG. 2 is a top view showing a state where the vehicle is traveling on a road.

  2A and 2B, the vehicle 202 has an in-vehicle camera (not shown) mounted on the rear part thereof, and travels straight between the white line 203 and the white line 204 on the road 201. White lines 203 and white lines 204 are examples of lane markers. The white line 203 is located on the left side in the traveling direction of the vehicle 202, the white line 204 is located on the right side in the traveling direction of the vehicle 202, and the white line 203 and the white line 20 are parallel. The reference straight line 205 is the center line of the white line 203 and the white line 204. A vector 206 indicates the traveling direction of the vehicle 202. A shooting range 207 is a shooting range of the vehicle-mounted camera of the vehicle 202. A line segment 208 is a line segment crossing the white line 204 and orthogonal to the vector 206.

  In the present embodiment, the in-vehicle camera is described as an example in which the vehicle-mounted camera is mounted on the rear portion of the vehicle 202 and images the rear of the vehicle 202. However, in the example in which the vehicle-mounted camera is mounted in front of the vehicle 202 and images the front of the vehicle 202. There may be. However, the road surface image taken by the in-vehicle camera can be determined with higher accuracy when the road image is closer to the vehicle 202. Therefore, as a more preferred embodiment, an example will be described in which an in-vehicle camera is installed at the rear part of the vehicle 202 so that a road surface image closer to the vehicle 202 can be taken, and a road surface image behind the vehicle direction is taken. .

  In FIG. 2A, the vector 206 coincides with the reference line 205. At this time, the vehicle 202 is traveling along the white lines 203 and 204. On the other hand, in FIG. 2B, the vector 206 is shifted from the reference straight line 205 in the direction of the white line 204, for example, once. At this time, the vehicle 202 is not traveling along the white lines 203 and 204.

  Next, the operation of the image processing apparatus 100 according to the present embodiment will be described. FIG. 3 is a flowchart illustrating an operation example of the image processing apparatus 100 according to the present embodiment.

  In step S301, the image acquisition unit 101 sets the exposure time of the camera (hereinafter also referred to as “exposure time” as appropriate) to 300 milliseconds, which is longer than the exposure time of 16 milliseconds of the NTSC camera, for example. Here, the exposure time is about ½ of the frame frequency, and the determination and setting of the exposure time and the acquisition of the image at the set exposure time are the determination and setting of the frame frequency and the acquisition of the image at the set frame frequency, respectively. As good as Specifically, in order to capture an image with an exposure time of 16 milliseconds, the frame frequency is set to 30 frames per second (the duration of one frame is 33 milliseconds).

  In step S302, the image acquisition unit 101 acquires a road surface image from the camera.

  Here, an example of a road surface image (hereinafter also referred to as “image” as appropriate) acquired by the image acquisition unit 101 will be described. Note that the image described below is described as an image captured by the camera as it is, but is an image obtained by performing a predetermined process on the image captured by the camera. May be.

  FIG. 4A is a diagram illustrating an example of a road surface image captured by the in-vehicle camera in the traveling state of FIG. 2A and an image obtained by enlarging a part of the road surface image. FIG. 4B is a diagram illustrating an example of a road surface image captured by the in-vehicle camera in the traveling state of FIG. 2B and an image obtained by enlarging a part of the road surface image.

  4A and 4B, road surface images 407a and 407b are images obtained by photographing an imaging range 207 with an exposure time of 300 milliseconds (hereinafter also referred to as “ms”) by an in-vehicle camera of a vehicle 202 traveling at a speed of 60 km / h. . The road surface images 407a and 407b include an asphalt area 401 that is a photographed image of the road 201, a white line area 403 that is a photographed image of the white line 203, and a white line area 404 that is a photographed image of the white line 204. The asphalt area 401 is a photographed image of an area where no white line is applied on the road (for example, an asphalt part having no white line).

  4A and 4B, rectangular areas 405a and 405b are rectangular areas that are part of the road surface images 407a and 407b, and include a part of the white line area 404 and a part of the asphalt area 401. The line segment areas 408 a and 408 b are part of the rectangular areas 405 a and 405 b, and are line segment areas including a part of the white line area 404 and a part of the asphalt area 401. The line segment areas 408 a and 408 b are areas of the line segment 208 that crosses the white line 204 and is orthogonal to the vector 206.

  In the road surface images 407a and 407b and the rectangular areas 405a and 405b, the x axis and the y axis are defined. The numerical value shown on each axis indicates the coordinates of the pixel. The line segment regions 408a and 408b are regions composed of the 90th to 190th pixels on the x axis in the 350th pixel line on the y axis, and are parallel to the x axis direction of the road surface image 407a.

  In the present embodiment, the line segment region extracted from the road surface images 407a and 407b is the 350th pixel line on the y axis, but is not limited thereto. As described above, since the road surface image is preferably taken at a position closer to the vehicle, the line segment region may be an image extracted by a line of pixels of the 350th pixel or less on the y axis, for example.

  In step S302, the image acquisition unit 101 further extracts a line segment area 408a (or 408b) including a part of the white line area 404 from the acquired road surface image 407a (or 407b).

  In step S303, the motion blur calculation unit 102 calculates the width of the motion blur from the line segment region 408a (or 408b) acquired by the image acquisition unit 101. Here, first, the motion blur calculation unit 102 calculates luminance values in order from the 90th pixel to the 190th pixel on the x-axis in the line segment region 408a or 408b. An example of the calculation result of the luminance value is shown in FIG.

  FIG. 5 is a graph showing a change in luminance value for each pixel calculated by the motion blur calculation unit 102. In FIG. 5, the horizontal axis indicates the pixel number of the x-axis of the line segment region 408a or 408b in FIG. 4, and the vertical axis indicates the luminance value of the pixel of each pixel number. A graph 501 is a graph in which the luminance values of the pixels included in the line segment region 408a photographed with an exposure time of 300 ms when the vehicle speed is 60 km / h are plotted with ◯. A graph 502 is a graph in which the luminance values of the pixels included in the line segment region 408b photographed with an exposure time of 300 ms when the vehicle speed is 60 km / h are plotted with Δ.

  In the case of the graph 501, the motion blur calculation unit 102 calculates the number of pixels between the maximum luminance value 250 (the luminance value of the white line region 404) and the minimum luminance value 150 (the luminance value of the asphalt region 401), and the motion blur is calculated. Width. Specifically, three pixels (number of circles) near the 130th or 180th on the horizontal axis are set as the width of the motion blur.

The same applies to the graph 502. The motion blur calculation unit 102 calculates the number of pixels between the maximum luminance value 250 (luminance value of the white line region 04) and the minimum luminance value 150 (luminance value of the asphalt region 401), and sets it as the width of the motion blur. Specifically, 27 pixels (number of Δ) from the 110th to 130th or 150th pixel to the 180th pixel on the horizontal axis are set as the width of the motion blur.

  As described above, the motion blur calculation unit 102 calculates the luminance value for each pixel included in the line segment region 408a or 408b, and the number of pixels on the horizontal axis while the luminance value changes from the maximum value to the minimum value. Is the width of the motion blur.

  In step S304, the traveling state determination unit 103 determines the traveling state of the vehicle based on the motion blur width calculated by the motion blur calculation unit 102 and a predetermined threshold value. An example of this running state determination process will be described below.

  In step S304, the traveling state determination unit 103 determines whether or not the motion blur width is less than a threshold value. When the width of the motion blur is less than the threshold value (step S304: YES), the flow proceeds to step S305. On the other hand, when the width of the motion blur is equal to or greater than the threshold (step S304: NO), the flow proceeds to step S306.

  In step S305, the traveling state determination unit 103 determines that the vehicle 202 is traveling along the reference line 205 (a state where the vector 206 matches the reference line 205 as shown in FIG. 2A). To do. Therefore, in this case, it is determined that the vehicle 202 is traveling along the white lines 203 and 204.

  In step S306, the traveling state determination unit 103 determines that the vehicle 202 is traveling away from the reference straight line 205 (the state where the vector 206 does not match the reference straight line 205 as shown in FIG. 2B). To do. Therefore, in this case, it is determined that the vehicle 202 is not traveling along the white lines 203 and 204.

  As described above, the image processing apparatus 100 according to the present embodiment uses a motion blur that occurs at the boundary between the white line area and the asphalt area described above, and thus has a simpler configuration (only a road surface image acquired from an in-vehicle camera). Thus, it can be accurately determined that the vehicle is traveling along the white line.

  Specifically, the image processing apparatus 100 can determine whether the vehicle is traveling along the white line as illustrated in FIG. 2A or whether the vehicle is traveling along the white line as illustrated in FIG. 2B. .

(Embodiment 2)
An image processing apparatus according to Embodiment 2 of the present invention will be described.

  A configuration of the image processing apparatus 600 according to the present embodiment will be described. FIG. 6 is a block diagram illustrating a configuration example of the image processing apparatus 600 according to the present embodiment. In FIG. 6, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The image acquisition unit 101 notifies the traveling state determination unit 603 of the set exposure time of the camera.

  If the exposure time of the camera is different, the width of the motion blur is also different. Specifically, when the exposure time is doubled, the amount of blurring of the white line that is the subject during exposure is doubled at the same vehicle speed, and the width of the motion blur integrated in the photographed image is also doubled. become.

  The traveling state determination unit 603 has data in which an exposure time and a threshold value are associated with each other. This data is set so that, for example, the threshold value increases as the exposure time increases. When the traveling state determination unit 603 receives the exposure time notification from the image acquisition unit 101, the traveling state determination unit 603 selects a threshold value associated with the exposure time. The traveling state determination unit 603 determines the traveling state of the vehicle based on the selected threshold value and the motion blur width calculated by the motion blur calculation unit 102.

  Next, the operation of the image processing apparatus 600 of this embodiment will be described. FIG. 7 is a flowchart illustrating an operation example of the image processing apparatus 600 according to the present embodiment. In FIG. 7, the same operations as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In step S701, the image acquisition unit 101 sets the exposure time of the camera. Examples of the value set here include 100 ms, 300 ms, and 500 ms. Then, the image acquisition unit 101 sends the acquired road surface image to the motion blur calculation unit 102 and notifies the traveling state determination unit 603 of the set exposure time of the camera.

  Description of steps S302 to S303 is omitted.

  In step S702, the traveling state determination unit 603 selects a threshold value used for determination. That is, the traveling state determination unit 603 selects a threshold value corresponding to the exposure time notified from the image acquisition unit 101 from data in which a predetermined exposure time and a threshold value are associated with each other. For example, the threshold value is N / 20 when the exposure time is Nms. Specifically, it is 5 when the exposure time is 100 ms, 15 when the exposure time is 300 ms, and 25 when the exposure time is 500 ms.

  In step S304, the traveling state determination unit 603 determines the traveling state of the vehicle based on the width of the motion blur calculated by the motion blur calculation unit 102 and the threshold value. An example of this running state determination process will be described below.

  An example of the calculation result of the luminance value by the motion blur calculation unit 102 in step S304 is shown in FIG.

  FIG. 8 is a graph showing a change in luminance value for each pixel calculated by the motion blur calculation unit 102. The horizontal and vertical axes in FIG. 8 are the same as those in FIG. In FIG. 8, the graph 501 and the graph 502 are the same as those in FIG. A graph 803 is a graph in which the luminance values of pixels included in the line segment region 408b photographed with an exposure time of 100 ms when the vehicle speed is 60 km / h is plotted as +. A graph 804 is a graph in which the luminance values of pixels included in the line segment region 408b photographed with an exposure time of 500 ms at a vehicle speed of 60 km / h are plotted with x.

  In the case of the graph 501, the running state determination unit 603 sets the threshold 15 because the exposure time is 300 ms for the motion blur width 3 of the graph 501, and the motion blur width is less than the threshold. Therefore, it is determined that the vehicle is traveling along the white line. In the case of the graph 502, the running state determination unit 603 sets a threshold value 15 because the exposure time is 300 ms with respect to the motion blur width 27 of the graph 502, and the motion blur width is equal to or greater than the threshold value. Therefore, it is determined that the vehicle is not traveling along the white line. Similarly, in the case of the graph 803, the running state determination unit 603 sets a threshold value 5 because the exposure time is 100 ms with respect to the motion blur width 10 of the graph 803, and the motion blur width is equal to or greater than the threshold value. Therefore, it is determined that the vehicle is not traveling along the white line. Similarly, in the case of the graph 804, the traveling state determination unit 603 sets a threshold value 25 because the exposure time is 500 ms with respect to the motion blur width 41 of the graph 804, and the motion blur width is equal to or greater than the threshold value. Therefore, it is determined that the vehicle is not traveling along the white line.

  Description of steps S305 to S306 is omitted.

  As described above, according to the image processing apparatus 600 of the present embodiment, in addition to the characteristics of the first embodiment, the threshold value is selected according to the exposure time of the camera. Therefore, the image processing apparatus 600 can accurately determine whether or not the vehicle is traveling along the white line according to an arbitrary exposure time with a simpler configuration.

(Embodiment 3)
An image processing apparatus according to Embodiment 3 of the present invention will be described.

  First, the configuration of the image processing apparatus 900 of this embodiment will be described. FIG. 9 is a block diagram illustrating a configuration example of the image processing apparatus 900 according to the present embodiment. In FIG. 9, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The vehicle speed acquisition unit 905 acquires the vehicle speed of the running vehicle. Then, the vehicle speed acquisition unit 905 notifies the acquired vehicle speed to the imaging control unit 904 and the traveling state determination unit 903.

  The width of the motion blur varies with the vehicle speed. Specifically, when the vehicle speed is doubled, the amount of blurring of the white line that is the subject during exposure is doubled for the same exposure time, and the width of the motion blur added to the captured image is doubled. become. When the vehicle speed is doubled and the exposure time is halved, the motion blur width is the same.

  The imaging control unit 904 determines the exposure time of the camera according to the vehicle speed acquired by the vehicle speed acquisition unit 905. For example, the imaging control unit 904 determines that the exposure time of the camera is shortened as the vehicle speed increases. Then, the imaging control unit 904 notifies the image acquisition unit 901 and the traveling state determination unit 903 of the determined exposure time.

  The image acquisition unit 901 is an in-vehicle camera, and acquires a road surface image that has been shot for the exposure time notified from the imaging control unit 904.

  The traveling state determination unit 903 has data in which a vehicle speed, an exposure time, and a threshold value are associated in advance. This data is set such that, for example, the threshold value increases as the vehicle speed increases and the exposure time increases. When the driving state determination unit 903 receives the vehicle speed notification from the vehicle speed acquisition unit 905 and receives the exposure time notification from the imaging control unit 904, the driving state determination unit 903 selects the threshold associated with the vehicle speed and the exposure time. To do. Then, the traveling state determination unit 903 determines the traveling state of the vehicle based on the selected threshold value and the motion blur width calculated by the motion blur calculation unit 102.

  Next, the operation of the image processing apparatus 900 according to this embodiment will be described. FIG. 10 is a flowchart illustrating an operation example of the image processing apparatus 900 according to the present embodiment. In FIG. 10, the same operations as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In step S1000, the vehicle speed acquisition unit 905 acquires the vehicle speed of the running vehicle and notifies the imaging control unit 904 and the traveling state determination unit 903.

  In step S <b> 1001, the imaging control unit 904 determines the exposure time of the camera based on the vehicle speed notified from the vehicle speed acquisition unit 905, and notifies the image acquisition unit 901 and the traveling state determination unit 903. The image acquisition unit 901 sets the exposure time of the camera notified from the imaging control unit 904.

  Description of steps S302 to S303 is omitted.

  In step S1002, the traveling state determination unit 903 selects a threshold value used for determination. In other words, the traveling state determination unit 903 determines a threshold value corresponding to the vehicle speed notified from the vehicle speed acquisition unit 905 and the exposure time notified from the imaging control unit 904 as a predetermined vehicle speed, exposure time, and threshold value. Select from the linked data. For example, the threshold value is N / 20 · M / 60 when the exposure time is Nms and the vehicle speed is Mkm / h. Specifically, it is 10 when the exposure time is 300 ms and the vehicle speed is 40 km / h, 15 when the exposure time is 300 ms and the vehicle speed is 60 km / h, and 25 when the exposure time is 300 ms and the vehicle speed is 100 km / h.

  In step S304, the traveling state determination unit 903 determines the traveling state of the vehicle based on the motion blur width calculated by the motion blur calculation unit 102 and a predetermined threshold value. An example of this running state determination process will be described below.

  An example of the calculation result of the luminance value by the motion blur calculation unit 102 in step S304 is shown in FIG.

  FIG. 11 is a graph showing a change in luminance value for each pixel calculated by the motion blur calculation unit 102. The horizontal and vertical axes in FIG. 11 are the same as those in FIG. Further, in FIG. 11, a graph 501 and a graph 502 are the same as those in FIG. A graph 1103 is a graph in which the luminance values of the pixels included in the line segment region 408b photographed with an exposure time of 300 ms when the vehicle speed is 40 km / h are plotted with +. A graph 1104 is a graph in which the luminance values of pixels included in the line segment region 408b photographed with an exposure time of 300 ms when the vehicle speed is 100 km / h are plotted with x.

  In the case of the graph 501, the running state determination unit 903 sets the threshold value 15 for the motion blur width 3 of the graph 501, because the exposure time is 300 ms and the speed is 60 km / h, and the motion blur width is less than the threshold value. Therefore, it is determined that the vehicle is traveling along the white line. Further, in the case of the graph 502, the running state determination unit 903 sets a threshold value 15 for the motion blur width 27 of the graph 502 because the exposure time is 300 ms and the speed is 60 km / h, and the motion blur width is the threshold. Since it is more than a value, it determines with not drive | working along a white line. Similarly, in the case of the graph 1103, the running state determination unit 903 sets the threshold 10 because the exposure time is 300 ms and the speed is 40 km with respect to the motion blur width 20 of the graph 1103, thereby reducing the motion blur width. Since it is above the threshold, it is determined that the vehicle is not traveling along the white line. Similarly, in the case of the graph 1104, the running state determination unit 903 sets the threshold value 25 for the motion blur width 36 of the graph 1104 because the exposure time is 300 ms and the speed is 100 km / h, thereby reducing the motion blur width. Since it is above the threshold, it is determined that the vehicle is not traveling along the white line.

  Description of steps S305 to S306 is omitted.

  As described above, according to the image processing apparatus 900 of the present embodiment, in addition to the features of the first embodiment, the threshold value is selected according to the vehicle speed and the exposure time of the camera. Therefore, the image processing apparatus 900 can accurately determine that the vehicle is traveling along the white line according to an arbitrary vehicle speed and exposure time with a simpler configuration.

(Embodiment 4)
An image processing apparatus according to Embodiment 4 of the present invention will be described.

  First, the configuration of the image processing apparatus 1200 according to the present embodiment will be described. FIG. 12 is a block diagram illustrating a configuration example of the image processing apparatus 1200 according to the present embodiment. In FIG. 12, the same components as those in the third embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The motion blur calculation unit 102 notifies the calculated motion blur not only to the traveling state determination unit 903 but also to the imaging control unit 1204.

  The imaging control unit 1204 estimates the vehicle speed based on the motion blur calculated by the motion blur calculation unit 102. Then, the imaging control unit 1204 determines the exposure time of the camera according to the estimated vehicle speed. Then, the imaging control unit 1204 notifies the image acquisition unit 901 of the exposure time, and notifies the traveling state determination unit 903 of the vehicle speed and the exposure time.

  Next, the operation of the image processing apparatus 1200 according to the present embodiment will be described. FIG. 13 is a flowchart illustrating an operation example of the image processing apparatus 1200 according to the present embodiment. In FIG. 13, the same operations as those in the third embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In step S1001, the imaging control unit 1204 sets the first exposure time as a default value, and notifies the image acquisition unit 901 of the first exposure time.

  In step S302, the image acquisition unit 901 sets the first exposure time notified from the imaging control unit 1204, acquires the first road surface image photographed at the first exposure time, and the first road surface image. Is sent to the motion blur calculation unit 102.

  In step S303, the motion blur calculation unit 102 notifies the imaging control unit 1204 of the width of the first motion blur calculated from the first road surface image captured at the first exposure time.

  In step S1300, the imaging control unit 1204 estimates the vehicle speed of the running vehicle based on the first exposure time and the width of the first motion blur notified from the motion blur calculation unit 102.

In step S1001, the imaging control unit 1204 sets a second exposure time based on the estimated vehicle speed. Specifically, an approximate polynomial is obtained from a set of a plurality of measured speeds per hour and a motion blur width. For example, when the vehicle is deviated from the reference straight line by 1 degree, the width of the motion blur when the speed is 40 km is 20, the width of the motion blur when the speed is 60 km is 27, the width of the motion blur when the speed is 100 km is 36, and The approximate polynomial in which the width of the motion blur at 0 km / h is 0 is y = 0.0019x ³ −0.056x ² + 2.377x, where y is the estimated hourly speed and x is the width of the motion blur to be calculated. Become. Next, the vehicle speed is obtained by substituting the calculated width of the first motion blur into the obtained approximate polynomial. For example, the motion blur width 20 is substituted to obtain the vehicle speed of 40.2 km / h. Subsequently, the degree of deviation of the traveling vehicle from the reference straight line, that is, the angle formed by the vehicle traveling direction and the reference straight line (the direction of the white line) is calculated. For example, when the imaging blur control unit 1204 sets the width of the motion blur calculated by the motion blur calculation unit 102 to d and the width of the motion blur when the vehicle travels once shifted by Δd, in step S1300, the vehicle Degree d / Δd deviating from the reference straight line is calculated. Specifically, when Δd is 25 pixels, in FIG. 11, the value on the horizontal axis until the graph 502 starts to decrease from the luminance value 250 and finishes decreasing to 150 is 25 from 150 to 175, and the vehicle is the reference. The degree d / Δd deviating from the straight line is calculated as 1 degree. If the degree of deviation of the vehicle when obtaining the approximate polynomial is substantially the same as the degree of deviation of the calculated vehicle from the reference line, the estimated vehicle speed is set as an effective estimated value.

  In step S1001, the imaging control unit 1204 notifies the image acquisition unit 901 of the second exposure time, and notifies the traveling state determination unit 903 of the vehicle speed and the second exposure time.

  In step S <b> 302, the image acquisition unit 901 sets the second exposure time notified from the imaging control unit 1204, acquires the second road surface image, and sends the second road surface image to the motion blur calculation unit 102.

  In step S303, the motion blur calculation unit 102 calculates the width of the second motion blur from the second road surface image, and notifies the imaging control unit 1204 and the traveling state determination unit 903 of the width of the second motion blur. The width of the second motion blur notified to the imaging control unit 1204 is used for estimating the vehicle speed in the next processing cycle.

  Description of steps S1002 and S304 to S306 is omitted.

  As described above, according to the image processing apparatus 1200 of the present embodiment, in addition to the features of the third embodiment, the vehicle speed is estimated based on the exposure time and the width of motion blur. Therefore, since image processing apparatus 1200 does not need to include vehicle speed acquisition unit 905, the vehicle travels along a white line according to an arbitrary vehicle speed and exposure time with a simpler configuration than that of the third embodiment. Can be determined accurately.

(Embodiment 5)
An image processing apparatus according to Embodiment 5 of the present invention will be described.

  First, the configuration of the image processing apparatus 1400 of this embodiment will be described. FIG. 14 is a block diagram illustrating a configuration example of the image processing apparatus 1400 according to the present embodiment. In FIG. 14, the same components as those in the third embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The motion blur calculation unit 1402 calculates the width of the motion blur for each of the two line segment regions from the road surface image acquired by the image acquisition unit 901. Then, the motion blur calculation unit 1402 notifies the travel state determination unit 903 and the vehicle posture determination unit 1406 of the calculated two motion blur widths.

  The vehicle posture determination unit 1406 determines whether or not the vehicle posture has a posture variation in the pitch angle direction (hereinafter referred to as “pitch angle variation”), and notifies the traveling state determination unit 903.

  The driving state determination unit 903 is based on at least one of the two motion blur widths notified from the motion blur calculation unit 1402 only when there is no change in the pitch angle of the vehicle notified from the vehicle posture determination unit 1406. Then, the traveling state of the vehicle is determined.

  Here, an example of the posture of the vehicle according to the present embodiment will be described. FIG. 16 is a side view showing a state where the vehicle is traveling on a road. In FIG. 16, the same reference numerals are given to the portions common to FIG. 2.

  FIG. 16A shows a state where there is no change in the posture of the vehicle. In FIG. 16A, a vector 206 indicating the traveling direction of the vehicle 202 is parallel to the plane of the road 201. FIG. 16A shows a state in which the vehicle 202 is changing in time series from the state of FIG. 16A to the state of FIG. 16A, and a vector 206 indicating the traveling direction of the vehicle 202 is Similarly, the state changes from the state to the state parallel to the surface of the road 201, and the pitch angle variation, which is the variation of the angle formed by the vector 206 and the surface of the road 201, is 0 degree.

  FIG. 16B shows a first state where there is a change in the attitude of the vehicle. In FIG. 16B, the vector 206 is shifted once downward with respect to the plane of the road 201 in the traveling direction. FIG. 16B shows a state in which the vehicle 202 is changing from the state of FIG. 16A to the state of FIG. 16B in time series. The pitch angle variation, which is a variation in the angle formed by the vector 206 and the surface of the road 201, is -1 degree.

  FIG. 16C shows a second state where there is a change in the attitude of the vehicle. In FIG. 16C, the vector 206 is offset once in the traveling direction with respect to the plane of the road 201. FIG. 16C shows a state in which the vehicle 202 is changing in time series from the state of FIG. 16A to the state of FIG. 16C, and a vector 206 indicating the traveling direction of the vehicle 202 is parallel to the plane of the road 201. The pitch angle variation, which is a variation in the angle formed by the vector 206 and the surface of the road 201, is +1 degree.

  Next, examples of images taken in each of FIGS. FIG. 17A is a diagram illustrating an example of a road surface image captured in a state where there is no posture change of FIG. 16A and an image in which a part thereof is enlarged. FIG. 17B is a diagram illustrating an example of a road surface image photographed in the first state with the posture variation of FIG. 16B and an enlarged image of a part thereof. FIG. 17C is a diagram illustrating an example of a road surface image captured in the second state with the posture variation of FIG. 16C and an enlarged image of a part thereof. Each of the images shown in FIGS. 17A to 17C is described as an image captured by the camera as it is, but is an image obtained by performing a predetermined process on the image captured by the camera. Also good.

  The road surface images shown in FIGS. 17A to 17C are, for example, images captured at an exposure time of 300 ms when the vehicle 202 is traveling along the white lines 203 and 204 at 60 km / h.

  In FIG. 17A, the road surface image 1707a includes an asphalt area 401, a white line area 403, and a white line area 404. The rectangular area 1705a is a rectangular area that is a part of the road surface image 1707a, and includes a part of the white line area 404 and a part of the asphalt area 1701a. On the other hand, the rectangular area 1706a is a rectangular area that is a part of the road surface image 1707a, and includes a part of the white line area 303 and a part of the asphalt area 1701a. The line segment area 1708a is a line segment area that is a part of the rectangular area 1705a, and includes a part of the white line area 404 and a part of the asphalt area 1701a. On the other hand, the line segment area 1709a is a line segment area that is a part of the rectangular area 1706a, and includes a part of the white line area 403 and a part of the asphalt area 1701a.

  Further, the x-axis and the y-axis are defined in the road surface image 1707a and the rectangular areas 1705a and 1706a. The numerical value shown on each axis indicates the coordinates of the pixel. The line segment area 1708a is an area composed of pixels from the 40th pixel to the 130th pixel on the x axis in the line of the 270th pixel on the y axis. On the other hand, the line segment area 1709a is an area composed of the 590th to 680th pixels on the x axis in the line of the 270th pixel on the y axis.

  Further, the rectangular area 1705a and the line segment area 1708a, and the rectangular area 1706a and the line segment area 1709a are in a symmetric positional relationship (an example of a specified positional relationship) with respect to the pixel line X1 at the center of the x axis. As described above, in this embodiment, the motion blur calculation unit 1402 extracts the line segment region 1708a and the line segment region 1709a, and calculates the width of each motion blur.

  In the present embodiment, the line segment region extracted from the road surface image 1707a is the line of the 270th pixel on the y axis, but is not limited to this. As described above, since the road surface image is preferably taken at a position closer to the vehicle, the line segment area may be an image extracted by a line of the 270th pixel or less of the y axis, for example.

  The configurations in FIGS. 17B and 17C are the same as those in FIG. 17A described above, and thus description thereof is omitted here. In FIG. 17B and FIG. 17C, the pixel line X1 shown in FIG. 17A is omitted, but the line segment region 1708b and the line segment region 1709b, and the line segment region 1708c and the line segment region 1709c are respectively The pixel line x1 has a symmetrical positional relationship with respect to the pixel line x1.

  From FIG. 17B and FIG. 17C, it can be seen that even if the vehicle 202 is traveling along the white lines 203 and 204, if the pitch angle of the vehicle 202 varies, motion blur will occur in the road surface image. This is because, during the exposure time, the shot image is shifted in the vertical direction due to fluctuations in the pitch angle, so the white line area that is the subject is not the extension line direction of the white line (diagonal direction on the imaging screen), but the top and bottom of the shot image. This is because the exposure is blurred in the direction (because the camera shakes, not the subject blur). Therefore, when there is a variation in the pitch angle of the vehicle, it may be erroneously determined that the vehicle 202 is not traveling along the white lines 203 and 204 even though the vehicle 202 is traveling along the white lines 203 and 204. In addition, when the position of the white line is shifted, a problem occurs when performing image processing that uses a vanishing point or the like.

  Therefore, the image processing apparatus 1400 according to the present embodiment operates so as to perform the traveling state determination only when there is no change in the pitch angle of the vehicle in order to prevent the erroneous determination and the malfunction. Hereinafter, the operation of the image processing apparatus 1400 of the present embodiment will be described. FIG. 15 is a flowchart illustrating an operation example of the image processing apparatus 1400 according to the present embodiment. In FIG. 15, the same operations as those in the third embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  Description of steps S1000 to S1002, S302, and S305 to S306 is omitted.

  In step S1503, the motion blur calculation unit 1402 extracts a region including a white line region from the road surface image acquired by the image acquisition unit 901. For example, in the case of a road surface image 1707a, the motion blur calculation unit 1402 extracts line segment areas 1708a and 1709a. Similarly, in the case of the road surface image 1707b or the road surface image 1707c, two line segment regions are extracted.

  The motion blur calculation unit 1402 calculates the width of the motion blur from each of the extracted regions. Here, first, the motion blur calculation unit 1402 calculates luminance values in order from the 130th pixel to the 40th pixel in the line segment region 1708a. In addition, the motion blur calculation unit 1402 calculates the luminance value in order from the 680th pixel to the 590th pixel in the line segment region 1709a. An example of the calculation result of the luminance value is shown in FIG.

  FIG. 18 is a graph showing a change in luminance value for each pixel calculated by the motion blur calculation unit 1402. In FIG. 18, the horizontal axis indicates the pixel numbers on the x-axis of the line segment regions 1708a and 1709a in FIG. 17, and the vertical axis indicates the luminance value of the pixel with each pixel number. A graph 1801R is a graph in which the luminance values of the pixels included in the line segment region 1708a are plotted, and a graph 1801L is a graph in which the luminance values of the pixels included in the line segment region 1709a are plotted with circles. A graph 1802R is a graph in which luminance values of pixels included in the line segment region 1708b are plotted, and a graph 1802L is a graph in which the luminance values of pixels included in the line segment region 1709b are plotted with Δ. A graph 1803R is a graph in which the luminance values of the pixels included in the line segment region 1708c are plotted, and a graph 1803L is a graph in which the luminance values of the pixels included in the line segment region 1709c are plotted with +. Since the method of calculating the motion blur width based on each graph has been described in the first embodiment, description thereof is omitted here.

  Then, the motion blur calculation unit 1402 notifies the vehicle posture determination unit 1406 of the change in the luminance value for each pixel calculated for each line segment area, and the motion blur width (first motion) calculated for each line segment area. At least one of the width of the blur and the width of the second motion blur is notified to the traveling state determination unit 903.

  Although the present embodiment has been described with reference to FIG. 18 as an example when the vehicle is traveling along the lane marker, an example when the vehicle is not traveling along the lane marker is shown in FIG. . FIG. 19 is a graph showing the result of the motion blur calculation unit 1402 calculating the luminance values of the two line segment regions when the vehicle is not traveling along the lane marker.

  In FIG. 19, the horizontal axis indicates pixel numbers on the x-axis of the line segment regions 1708 a and 1709 a in FIG. 17, and the vertical axis indicates the luminance value of the pixel with each pixel number. A graph 1901R is a graph in which the luminance values of the pixels included in the line segment region 1708a are plotted, and a graph 1901L is a graph in which the luminance values of the pixels included in the line segment region 1709a are plotted with circles. A graph 1902R is a graph in which luminance values of pixels included in the line segment region 1708b are plotted, and a graph 1902L is a graph in which the luminance values of pixels included in the line segment region 1709b are plotted with Δ. A graph 1903R is a graph in which the luminance values of the pixels included in the line segment region 1708c are plotted, and a graph 1903L is a graph in which the luminance values of the pixels included in the line segment region 1709c are plotted with +. Since the method of calculating the motion blur width based on each graph has been described in the first embodiment, description thereof is omitted here.

  In step S1501, the vehicle posture determination unit 1406 calculates the first line segment regions 1801L, 1802L,... From the change in luminance value for each pixel calculated for each line segment region. The difference between the range in which the luminance value continues at the maximum value in 1803L (hereinafter referred to as “peak range”) and the peak range of the second line segment regions 1801R, 1802R, and 1803R (hereinafter referred to as “distance between peak ranges”). And determine whether or not the distance between the peak ranges is approximate to the predetermined distance (that is, whether or not the distance is within a range of ± n with reference to the predetermined distance). The distance between peak ranges is a distance between two pixels that are in a symmetrical positional relationship in two line segment regions. In FIG. 18, for example, the two pixels are a pixel at a coordinate value 85 that is the center (middle) of the peak range in the graph 1801R and a pixel at a coordinate value 635 that is the center of the peak range in the graph 1801L. is there. In this case, the difference 550 between the coordinate value 635 and the coordinate value 85 is the distance between the peak ranges. The predetermined distance is 550, for example. Therefore, in the case of the graphs 1802R and 1802L, the two graphs are farther from each other than the graphs 1801R and 1801L, and thus the distance between the peak ranges is longer than the predetermined distance and is not approximate to the predetermined distance. In the case of the graphs 1803R and 1803L, the two graphs are closer to each other than the graphs 1801R and 1801L. Therefore, the distance between the peak ranges is shorter than the predetermined distance and is not approximate to the predetermined distance.

  As a result of the determination in step S1501, when the distance between peak ranges is not approximate to the predetermined distance (step S1501: NO), the flow returns to step S1000. On the other hand, if the result of determination in step S1501 is that the distance between peak ranges is approximate to the predetermined distance (step S1501: YES), the flow proceeds to step S1502.

  In step S1502, the vehicle attitude determination unit 1406 determines whether or not the graph shape of the change in luminance value of the first line segment region and the graph shape of the change in luminance value of the second line segment region are approximate. Determine. For example, in FIG. 19, the median values of the graph 1901L and the graph 1901R are overlapped, and the difference area of the luminance value is smaller than the predetermined area value, so that the shape is approximate. Further, it is assumed that the shape is not approximate because the difference area of the luminance value is larger than the predetermined area value by superimposing the median values of the graph 1902L and the graph 1902R. Similarly, it is assumed that the shape is not approximate since the difference area of the luminance value is larger than the predetermined area value by superimposing the median values of the graphs 1903L and 1903R.

  As a result of the determination, when the shape is not approximate (step S1502: NO), the flow returns to step S1000. On the other hand, if the result of determination is that the shape is approximate (step S1502: YES), the flow proceeds to step S1002. Thus, in the present embodiment, when the distance between the peak ranges is approximate to the predetermined distance (step S1501: YES), and the graph shape of the change in the luminance values of the two line segment regions is approximate (step (S1502: YES), the vehicle posture determination unit 1406 determines that there is no variation in the pitch angle of the vehicle. And only when it determines with there being no pitch angle fluctuation | variation of a vehicle, it progresses to driving state determination after step S1002.

  In step S304, the traveling state determination unit 903 determines the traveling state of the vehicle based on at least one of the two motion blur widths calculated by the motion blur calculation unit 1402 and a threshold value.

  As described above, according to the image processing apparatus 1400 of the present embodiment, in addition to the features of the third embodiment, it is determined whether or not there is a vehicle pitch angle variation, and only when there is no vehicle pitch angle variation. The running state determination is performed. Therefore, the image processing apparatus 1400 can prevent the running state determination based on the width of the motion blur affected by the camera shake caused by the pitch angle variation of the vehicle.

  In the first to fifth embodiments described above, the running state determination units 103, 603, and 903 of the image processing apparatus may operate as follows. That is, the traveling state determination units 103, 603, and 903 also determine (calculate) the degree to which the traveling vehicle deviates from the reference straight line, that is, the angle formed by the vehicle traveling direction and the reference straight line (white line direction). Good. For example, the traveling state determination units 103, 603, and 903 have the motion blur width calculated by the motion blur calculation unit 1402 as d, and the motion blur width when the vehicle travels once shifted as Δd, In step S304, it is determined that the vehicle is deviated by d / Δd degrees from the reference straight line. Specifically, when Δd is 25 pixels, in FIG. 8, the graph 502 starts to decrease from the luminance value 250 to 150 with respect to the x-axis value 175 in which the graph 501 starts to decrease from the luminance value 250 to 150 x. The axis value is 150, and the deviation between the traveling direction and the white line is calculated as 1 degree from the difference 25 between the x-axis values. Thereby, in addition to the determination of whether the vehicle is traveling along the lane marker, the image processing apparatus can also determine how many times the vehicle is traveling with respect to a reference straight line (which may be referred to as a lane marker). .

(Embodiment 6)
An image processing apparatus according to Embodiment 6 of the present invention will be described.

  First, the configuration of the image processing apparatus 2000 according to the present embodiment will be described. FIG. 20 is a block diagram illustrating a configuration example of the image processing apparatus 2000 according to the present embodiment. In FIG. 20, the same components as those in the fifth embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  As described above, the traveling state determination unit 903 determines whether the vehicle is deviated from the reference straight line by n degrees. Then, the traveling state determination unit 903 notifies the lane marker detection unit 2007 of the determined degree of deviation.

  The lane marker detection unit 2007 determines whether the degree of deviation notified from the traveling state determination unit 903 is within a predetermined range. As a result of this determination, if the degree of deviation is within a predetermined range, the lane marker detection unit 2007 determines the exposure time of the camera based on the degree of deviation and notifies the imaging control unit 904 of it. The lane marker detection unit 2007 acquires a road surface image from the image acquisition unit 901 and detects a lane marker.

  In FIG. 20, the lane marker detection unit 2007 is not included in the image processing apparatus 2000, but the present invention is not limited to this. That is, the lane marker detection unit 2007 may be a device provided inside the image processing device 2000.

  Upon receiving the exposure time notification from the lane marker detection unit 2007, the imaging control unit 904 controls the image acquisition unit 901 so as to set the exposure time.

  The image acquisition unit 901 sets the exposure time under the control of the imaging control unit 904. Further, the image acquisition unit 901 notifies the lane marker detection unit 2007 of the road surface image and / or the exposure time being set.

  Next, the operation of the image processing apparatus 2000 according to the present embodiment will be described. FIG. 21 is a flowchart illustrating an operation example of the image processing apparatus 2000 according to the present embodiment. FIG. 21 is a flow executed after the flow of FIG. 15 of the fifth embodiment.

  FIG. 22 is an example of a road surface image for explaining the operation of the image processing apparatus 2000 of the present embodiment. In FIG. 22, reference numeral 2201 denotes an example of a road surface image in normal photographing in a state where a white line as a lane marker is blurred. 2202 is an example of a photographed road surface image acquired by the image acquisition unit 901 of the image processing apparatus 2000 according to the present embodiment. Reference numeral 2203 denotes a road surface image in which the lane marker detection unit 2007 of the image processing apparatus 2000 according to the present embodiment performs contrast and brightness correction processing as preprocessing of processing for detecting a lane marker from the road surface image.

  In step S <b> 2101, the lane marker detection unit 2007 acquires the degree of deviation from the traveling state determination unit 903.

  In step S2102, the lane marker detection unit 2007 determines whether the degree of deviation is within a predetermined range. As a result of the determination, when the degree of deviation is not within the predetermined range (step S2102: NO), the flow returns to step S2101. On the other hand, if the result of determination is that the degree of deviation is within a predetermined range (step S2102: YES), the flow proceeds to step S2103.

  In step S2103, the lane marker detection unit 2007 determines the exposure time of the camera to be shorter than the exposure time notified from the image acquisition unit 901 based on the degree of deviation. For example, the lane marker detection unit 2007 determines the exposure time to be 50 ms. The lane marker detection unit 2007 notifies the imaging control unit 904 of the determined exposure time. The imaging control unit 904 controls the image acquisition unit 901 so as to set the exposure time notified from the lane marker detection unit 2007.

  In step S2104, the image acquisition unit 901 acquires the road surface image 2202 with the exposure time set by the control of the imaging control unit 904. Then, the image acquisition unit 901 notifies the lane marker detection unit 2007 of the acquired road surface image 2202. The road surface image 2202 is an image in which a lane marker is easy to detect by superimposing a motion blur of a faint white line region compared to the original road surface image 2201.

  In step S2105, the lane marker detection unit 2007 generates a road surface image 2203 in which contrast and brightness correction is further performed on the road surface image 2202 acquired from the image acquisition unit 901, and detects a luminance edge from the road surface image 2203. Then, the image area surrounded by the luminance edge is detected as a lane marker.

  As described above, according to the image processing apparatus 2000 of the present embodiment, in addition to the effects of the fifth embodiment, when the degree of deviation is within a predetermined range with respect to the accuracy required for lane marker detection, lane marker detection is performed. A suitable camera exposure time can be set, and lane marker detection can be performed on the road surface image acquired with the exposure time.

(Modification in the embodiment)
As mentioned above, although each embodiment of this invention was described, the said description is an example and various deformation | transformation are possible. Hereinafter, some modifications will be described.

  For example, in the fifth embodiment, the distance between two peak ranges corresponding to two line segment regions is used as a parameter for determining the attitude of the vehicle. However, two or more peak ranges may be used. . Alternatively, the parameter for determining the vehicle posture may use not the distance between peak ranges but a difference in graph shape of a change in luminance value or a difference in area occupied by a graph indicating the luminance value.

  For example, in Embodiment 5, the difference between the areas of the change graphs of two luminance values respectively corresponding to the two line segment regions is used as a parameter for determining the attitude of the vehicle. The difference graph shape may be the difference between the maximum value and the minimum value, or may be other than these.

  For example, in the fifth embodiment, the difference (distance) between two “coordinate values of the center of the x-axis” respectively corresponding to two line segment regions is used as a parameter for determining the attitude of the vehicle. Two or more “coordinate values of the center of the horizontal axis” may be used. Alternatively, the parameter for determining the vehicle posture may be other than the difference between the coordinate values at the center of the horizontal axis.

  For example, in Embodiments 5 and 6, the image processing apparatus may be configured not to include the vehicle speed acquisition unit 905. That is, the image processing apparatus may estimate the vehicle speed based on the width of the motion blur from the motion blur calculation unit 102 in the imaging control unit 1204 as in the image processing apparatus 1200 of the fourth embodiment. . In the fourth, fifth, and sixth embodiments, the approximate expression for estimating the vehicle speed is not limited to the case where the vehicle is shifted by 1 degree with respect to the white line. Or when traveling along a white line, that is, when it is not deviated from the white line. Although the approximate expression is a cubic polynomial, it may be other than this.

  For example, in Embodiments 1 to 5, the exposure time of a normal camera is set to 16 ms and the exposure time set by the image acquisition units 101 and 901 is set to 300 ms. However, the present invention is not limited to these values. In the sixth embodiment, the exposure time set by the image acquisition unit 901 is set to 50 ms, but is not limited to this value.

  For example, in Embodiments 1 to 6, the road surface image captured by the camera is not limited to the images shown in FIGS. 4A, 4B, 17A, 17B, 17C, and 22.

  For example, in the first to sixth embodiments, the line segment region extraction method is not limited to the method described in the above embodiment, and may be a method in which a plurality of methods are combined.

  For example, in Embodiments 1 to 6, the line segment region is not limited to the region described in the above embodiment. For example, the line segment region may be a two-dimensional image region, or may be a spatial region different from the pixel space (for example, a frequency-converted image region).

  For example, in the first to sixth embodiments, the degree of deviation of the vehicle from the reference straight line is 1 degree in the right direction, but the degree of deviation and the direction of deviation are not limited to this.

  For example, in Embodiments 1 to 6, the width of the motion blur is the number of pixels (distance) on the horizontal axis between the maximum steady peak and the minimum steady peak of the luminance value, but is not limited thereto. For example, the width of the motion blur may be the slope between the maximum steady peak and the minimum steady peak of brightness (distance on the vertical axis / distance on the horizontal axis), or when the vehicle is traveling straight It may be the area of the difference area of the graph shape of the change in the luminance value.

  For example, in Embodiments 1 to 6, the motion blur calculation units 102 and 1402 calculate the luminance values of the pixels in order from the smallest coordinate value on the x-axis of the road surface image, but the present invention is not limited to this. For example, the motion blur calculation unit 102 may calculate the luminance values of the pixels in order from the largest coordinate value on the x-axis of the road surface image to the smaller one, or may sequentially calculate from the surrounding pixels of the white line. .

  For example, in the first to fourth embodiments, only the line segment area including a part of the white line area 404 is used for calculating the width of the motion blur. However, the present invention is not limited to this. For example, the motion blur calculation unit 102 may use a line segment area including a part of the white line area 403. Alternatively, the motion blur calculation unit 102 may use both a part of the white line area 403 and a part of the white line area 404.

  For example, in Embodiments 1 to 6, the line segment area is used as an example of the partial area in calculating the motion blur width, but the partial area is not limited to the line segment.

  For example, in the first to sixth embodiments, the threshold value, the expression for deriving the threshold value, the predetermined distance, and the predetermined range are not limited to the values described in the above embodiment.

  For example, in the first to sixth embodiments, the traveling state determination units 103, 603, and 903 determine the traveling state of a vehicle traveling on a straight road as illustrated in FIG. The traveling state of the vehicle traveling on the road can also be determined. Further, the traveling state determination units 103, 603, and 903 may determine the degree of deviation of a vehicle traveling on a curved road having a certain curvature (how many R are deviated from a white line having a curvature).

  For example, in the sixth embodiment, the lane marker detection unit includes a vehicle detection unit, a vehicle recognition unit, a pedestrian detection unit, a pedestrian recognition unit, a traffic sign detection unit, a traffic sign recognition unit, a road surface state detection unit, and other image processing. Part. Further, although the exposure time is 50 ms, it may be an appropriate exposure time in each detection unit and recognition unit. In the lane marker detection unit, the contrast and brightness of the road surface image are corrected. However, other correction processing, conversion processing, and image processing may be performed.

  For example, in the fifth and sixth embodiments, the vehicle attitude determination unit 1406 determines whether or not there is a variation in the pitch angle of the vehicle, but may determine whether or not there is a variation in the roll angle of the vehicle. The presence or absence of both angular variations may be determined.

  For example, in Embodiments 5 and 6, when the distance between peak ranges is approximate to a predetermined distance, and the graph shape of the change in luminance value of two line segment regions is approximate, the vehicle posture determination unit 1406 When it is determined that there is no fluctuation in the pitch angle of the vehicle, either when the distance between peak ranges is approximate to a predetermined distance, or when the graph shape of the change in luminance value of two line segment areas is approximate, either The vehicle posture determination unit 1406 may determine that there is no fluctuation in the pitch angle of the vehicle by only determining the above.

  The present invention can be applied to, for example, a general technique for supporting driving of a vehicle based on a road surface image taken by a vehicle-mounted camera while the vehicle is traveling, a system related to calibration of a vehicle-mounted camera, automatic driving, and platooning.

100, 600, 900, 1200, 1400, 2000 Image processing apparatus 101, 901 Image acquisition unit 102, 1402 Motion blur calculation unit 103, 603, 903 Running state determination unit 904, 1204 Imaging control unit 905 Vehicle speed acquisition unit 1406 Vehicle posture determination 2007 lane marker detector

Claims (9)

An image acquisition unit that acquires a first road surface image that is an image of a road including a lane marker by photographing from a traveling vehicle;
A motion blur calculation unit for calculating a width of a motion blur generated based on the lane marker from the first road surface image;
When the width of the motion blur is smaller than a threshold value, a traveling state determination unit that determines that the vehicle is traveling along the lane marker;
An image processing apparatus. The image acquisition unit notifies the traveling state determination unit of a first exposure time when shooting the first road surface image,
The traveling state determination unit selects the threshold based on the first exposure time notified from the image acquisition unit,
The image processing apparatus according to claim 1. An image capturing control unit that determines a first exposure time when capturing the first road surface image based on a vehicle speed of the vehicle and notifies the image acquisition unit and the traveling state determination unit;
The image acquisition unit acquires the first road surface image based on the first exposure time notified from the imaging control unit,
The traveling state determination unit selects the threshold value based on the first exposure time notified from the imaging control unit.
The image processing apparatus according to claim 1. A vehicle speed acquisition unit that acquires the vehicle speed of the vehicle and notifies the imaging control unit and the traveling state determination unit;
The imaging control unit determines a first exposure time when shooting the first road surface image based on the vehicle speed notified from the vehicle speed acquisition unit,
The traveling state determination unit selects the threshold based on the vehicle speed notified from the vehicle speed acquisition unit and the first exposure time notified from the imaging control unit.
The image processing apparatus according to claim 3. The motion blur calculation unit notifies the imaging control unit of the width of the motion blur,
The imaging control unit estimates a vehicle speed of the vehicle based on the first exposure time and a width of a motion blur notified from the motion blur calculation unit, and a second road surface based on the vehicle speed of the vehicle. Determining a second exposure time when taking an image, notifying the vehicle speed and the second exposure time to the running state determination unit, notifying the image acquisition unit of the second exposure time,
The image acquisition unit acquires the second road surface image based on the second exposure time notified from the imaging control unit,
The motion blur calculation unit notifies the travel state determination unit of the width of the second motion blur,
The traveling state determination unit selects the threshold value based on the vehicle speed and the second exposure time notified from the imaging control unit with respect to the width of the second motion blur.
The image processing apparatus according to claim 3. A vehicle posture determination unit for determining whether there is a pitch variation of the vehicle body of the vehicle;
The motion blur calculation unit calculates a width of a plurality of motion blurs for each of a plurality of partial areas of the road surface image, notifies the vehicle posture determination unit of the widths of the plurality of motion blurs,
The vehicle posture determination unit determines whether or not there is a pitch variation of the vehicle based on the widths of the plurality of motion blurs, and indicates the vehicle posture indicating whether or not the vehicle has a pitch variation. Notify the state determination unit
When the vehicle posture is a posture in which there is no pitch fluctuation of the vehicle, the traveling state determination unit is configured to travel along the lane marker when at least one of the plurality of motion blur widths is smaller than the threshold value. It is determined that
The image processing apparatus according to claim 1. An image processing unit for performing predetermined image processing;
The traveling state determination unit notifies the image processing unit of a traveling state indicating whether the vehicle is traveling along the lane marker,
The image processing unit determines a third exposure time corresponding to the image processing when the traveling state notified from the traveling state determining unit is a state where the vehicle is traveling along the lane marker. And notifying the imaging control unit of the third exposure time,
The image acquisition unit acquires the road surface image based on a third exposure time determined by the image processing unit, and sends the road image to the image processing unit,
The image processing unit performs predetermined image processing on the road surface image received from the image acquisition unit.
The image processing apparatus according to claim 1. Acquiring a road surface image that is an image of a road including a lane marker by photographing from a traveling vehicle; and
Calculating a width of a motion blur generated based on the lane marker from the road surface image;
Determining that the vehicle is traveling along the lane marker when the width of the motion blur is less than a threshold;
An image processing method including: A process of acquiring a road surface image that is an image of a road including a lane marker by photographing from a traveling vehicle;
From the road surface image, processing to calculate the width of the motion blur that occurs based on the lane marker,
When the width of the motion blur is smaller than a threshold value, a process for determining that the vehicle is traveling along the lane marker;
An image processing program for causing a computer to execute.