JP2014106739A - In-vehicle image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase a process speed while suppressing errors detection of a road surface sign.SOLUTION: An in-vehicle image processing device includes an arithmetic processing unit 4 which includes a feature amount extracting part 402, a feature amount correcting part 403, a feature amount grouping part 404, a line extracting part 405 and a road surface sign recognizing part 406. The feature amount extracting part 402 extracts a feature amount in accordance with a line width of a sign line drawn on a traveling surface for the vehicle. The feature amount correcting part 403 corrects each feature amount based on a set condition of a camera. The feature amount grouping part 404 groups each corrected feature amount. The line extracting part 405 extracts a sign line drawn on the traveling surface for the vehicle based on each grouped feature amount. The road surface sign recognizing part 406 recognizes a road surface sign based on the extracted sign line and outputs the recognition result.

Description

  The present invention relates to an apparatus that is mounted on a vehicle and recognizes road markings in a vehicle traveling environment.

  2. Description of the Related Art Conventionally, there has been known an apparatus that supports parking of a vehicle by recognizing a parking frame line from an image that is mounted on the vehicle and obtained by photographing with a camera (Patent Document 1).

JP 2010-12838 A

  In the apparatus disclosed in Patent Document 1, luminance change points (edges) in an image are extracted as feature points, and two parallel pairs of contour lines are parked in a contour line obtained by linearly approximating the point sequence of the feature points. It is detected as a frame line. Therefore, when the vehicle is traveling on a road, there is a possibility that a line other than the parking frame line, such as a roadway center line or a lane boundary line, may be erroneously detected as a parking frame line. In addition, for example, in a situation where the processing load for obtaining the contour line is excessive, such as when there are a large number of lines or complicated shapes in the image, there is a possibility that the processing may not be in time within the predetermined processing time. is there.

  An in-vehicle image processing device according to the present invention is provided on a traveling surface of a vehicle based on a photographing device that captures a photograph of the surroundings of the vehicle to acquire a photographed image, an installation state detection unit that detects an installation state of the photographing device, and the photographed image. Extracted by the feature amount extraction unit based on the installation state of the photographing apparatus detected by the installation amount detection unit and the feature amount extraction unit that extracts the feature amount according to the line width of the drawn marking line at predetermined intervals. A feature amount correcting unit that corrects each feature amount, a feature amount grouping unit that groups the feature amounts corrected by the feature amount correcting unit, and a feature amount grouped by the feature amount grouping unit. A line extraction unit that extracts a marking line, and a road marking recognition unit that recognizes the road marking based on the marking line extracted by the line extraction unit and outputs the recognition result.

  According to the present invention, it is possible to increase the processing speed while suppressing erroneous detection of road markings.

It is a block diagram which shows the structure of the vehicle-mounted image processing apparatus by one Embodiment of this invention. It is a figure which shows the example of the display image which match | combined the bird's-eye view image and the picked-up image of the vehicle forward. It is a figure which shows the example of various road markings. It is a control block diagram of arithmetic processing. It is a figure which shows the specific example of extraction of a feature-value, and grouping. It is a figure which shows the specific example of each extracted feature-value. It is a flowchart of the process performed in a vehicle-mounted image processing apparatus. It is a flowchart of a feature amount extraction process. It is a flowchart of a feature quantity grouping process. It is a flowchart of a line extraction process.

  FIG. 1 is a block diagram showing a configuration of an in-vehicle image processing apparatus 100 according to an embodiment of the present invention. An in-vehicle image processing apparatus 100 shown in FIG. 1 is used by being mounted on a vehicle, and includes cameras 1a, 1b, 1c and 1d, an image composition unit 2, an installation state detection unit 3, and an arithmetic processing unit 4. And an output unit 5.

  The cameras 1a to 1d are electronic cameras that shoot around the vehicle in different shooting ranges, and are installed in various parts such as a vehicle body, a bumper, and a door mirror. The shooting range of each of these cameras is determined so as to cover the entire periphery of the vehicle. In this embodiment, the camera 1a shoots the shooting range in front of the vehicle, the camera 1b shoots the shooting range on the left side of the vehicle, the camera 1c shoots the shooting range on the right side of the vehicle, and the camera 1d shoots the shooting range on the rear side of the vehicle. Will be described. The captured images respectively acquired at predetermined frame rate intervals by the cameras 1 a to 1 d are output to the image composition unit 2.

  In the present embodiment, the four cameras 1a to 1d are described as shooting the respective shooting ranges as described above. However, the number of cameras mounted on the vehicle and the shooting range are not limited thereto. In addition, the shooting range including the cameras may not necessarily cover the entire periphery of the vehicle. If the surroundings of the vehicle can be photographed within an appropriate range, a photographed image can be acquired for an arbitrary photographing range using an arbitrary number of cameras.

  The image composition unit 2 synthesizes an overhead view image (around view image) that shows a state where the entire periphery of the vehicle is seen from the captured images acquired by the cameras 1a to 1d. This bird's-eye view image is synthesized by coordinate-converting the captured images of the cameras 1a to 1d according to the shooting direction and connecting them. At this time, lens distortion correction and luminance conversion processing may be performed as necessary. As brightness conversion processing, for example, correction of contrast according to the surrounding lighting environment and aging of the target object, brightness correction according to decrease in peripheral light reduction that darkens as the amount of light decreases around the lens, etc. Can do. Furthermore, noise reduction processing for reducing noise superimposed on the transmission path may be performed. The bird's-eye view image synthesized by the image synthesis unit 2 is output to the arithmetic processing unit 4.

  The installation state detection unit 3 detects the installation states of the cameras 1 a to 1 d and outputs the detection results to the arithmetic processing unit 4. For example, the camera with respect to the running surface of the vehicle is detected by detecting the expansion / contraction amount of the suspension corresponding to each wheel of the vehicle, the degree of inclination of the vehicle and the cameras 1a to 1d with respect to the horizontal plane, the attachment error of the cameras 1a to 1d to the vehicle, The positions and orientations of 1a to 1d can be detected, and the installation states of the cameras 1a to 1d can be obtained from the detection results. Here, as for the attachment error of the cameras 1a to 1d, for example, a mark for detecting an attachment error is attached to a part of the vehicle surface within the photographing range of each camera, and the mark is displayed on the photographing screen. It can be detected by obtaining the positional deviation. Other methods may be used as long as the installation states of the cameras 1a to 1d can be detected.

  The arithmetic processing unit 4 performs predetermined arithmetic processing on the basis of the overhead image synthesized by the image synthesizing unit 2 and the installation states of the cameras 1a to 1d detected by the installation state detection unit 3, thereby Recognize road markings. The contents of the arithmetic processing performed by the arithmetic processing unit 4 will be described in detail later. The recognition result of the road marking by the arithmetic processing unit 4 is output to the output unit 5.

  The output unit 5 outputs road marking information for road markings around the vehicle based on the recognition result of the road markings by the arithmetic processing unit 4. For example, information indicating the direction of the road marking with respect to the vehicle and the distance to the road marking is output as road marking information. This road marking information is output to a host vehicle control device (not shown) connected to the in-vehicle image processing device 100, and is used for vehicle parking assistance, travel control, and the like.

  For example, the output part 5 outputs the information which shows the direction of the parking frame with respect to a vehicle, and the distance to a parking frame as parking frame information. This parking frame information is output to a host vehicle control device (not shown) connected to the on-vehicle parking frame recognition device, and is used for vehicle parking assistance, travel control, and the like. For example, it is possible to automatically recognize that there is a parking lot around, and for example, in a parking lot environment, it is possible to automatically switch and display a bird's-eye view image around the vehicle on the monitor. As a result, it is possible to suppress a situation in which it is erroneously detected as a parking lot on a public road, and to switch the video presented to the user at an appropriate timing.

  Further, the output unit 5 outputs the position information of the parking frame in the actual environment such as the end point coordinates of the left and right frame lines of the parking frame, the angle and the intercept of the parking frame line. This parking frame information is output to a host vehicle control device (not shown) connected to the on-vehicle parking frame recognition device, and is used for vehicle parking assistance, travel control, and the like. For example, by calculating the travel route to the parking frame based on the relative position and orientation from the own vehicle to the parking frame, by notifying the driver of the timing of brake and shift position change and the amount of steering angle operation, Parking assistance can be performed. As a result, a parking operation can be completed in a short time even for a driver who is unfamiliar with driving operations such as garage entry.

  Furthermore, the travel route to the parking frame is calculated based on the relative position and orientation from the own vehicle to the parking frame, and the control amount of the forward / backward / turning of the vehicle is automatically calculated. The movement may be automatically controlled. As a result, a parking operation can be completed safely and accurately even for a driver unfamiliar with driving operations such as garage entry.

  FIG. 2 is a diagram illustrating an example of a display image in which an overhead image synthesized by the image synthesis unit 2 and a photographed image in front of the vehicle photographed by the camera 1a are combined. In FIG. 2, the left image 21 shows a bird's-eye view image, and the right image 22 shows a photographed image in front of the vehicle.

  The bird's-eye view image 21 is composed of four image areas 21a, 21b, 21c and 21d corresponding to the front, left side, right side and rear of the vehicle, respectively. Each image in these image areas is created based on images taken by the cameras 1a to 1d in FIG. A part of the road marking 23 is displayed in the image area 21a. This road marking 23 is also displayed on the captured image 22.

  The image as described above is displayed on a display monitor (not shown) installed in the vehicle, for example. Note that the image displayed as the captured image 22 is preferably switched according to the traveling direction of the vehicle. For example, when the shift lever of the vehicle is switched to the forward direction, it is determined that the vehicle is moving forward, and a captured image of the front of the vehicle captured by the camera 1a is displayed. On the other hand, when the shift lever of the vehicle is switched to the reverse direction, it is determined that the vehicle is moving backward, and a captured image of the rear of the vehicle captured by the camera 1d is displayed. If it does in this way, the suitable photography picture according to the advancing direction of vehicles can be shown to the driver of vehicles, and driving operation support at the time of parking etc. can be performed.

  FIG. 3 is a diagram showing examples of various road markings. The parking frame line in FIG. 3A shows an example in which only the left and right boundary lines are drawn as parking frame lines for the parking frames for two vehicles. The parking frame line in FIG. 3B shows an example in which either the front or rear boundary line is drawn as a parking frame line in addition to the parking frame line in FIG. The parking frame line in FIG. 3C shows an example in which both the front and rear boundary lines are drawn as parking frame lines in addition to the parking frame line in FIG. The parking frame line of FIG.3 (d) has shown the example which drew the parking frame line of Fig.3 (a) diagonally.

  The parking frame line of FIG.3 (e) has shown the example which drawn the left-right boundary line with the U-shaped parking frame line with respect to the parking frame for one vehicle. The parking frame line in FIG. 3 (f) shows an example in which either the front or rear boundary line is drawn as a parking frame line in addition to the parking frame line in FIG. 3 (e). The parking frame line in FIG. 3G shows an example in which the parking frame line as shown in FIG. 3E is drawn obliquely with respect to the parking frame lines for two vehicles.

  Next, the contents of the arithmetic processing performed by the arithmetic processing unit 4 will be described. FIG. 4 is a control block diagram of the arithmetic processing performed by the arithmetic processing unit 4. As shown in FIG. 4, the arithmetic processing unit 4 includes an edge search direction determination unit 401, a feature amount extraction unit 402, a feature amount correction unit 403, a feature amount grouping unit 404, a line extraction unit 405, and a road marking recognition unit 406. Each control block is functionally included. In the arithmetic processing unit 4, for example, each control block of FIG. 4 is realized by executing a program recorded in the memory corresponding to each of these control blocks by a microcomputer.

  The edge search direction determination unit 401 determines an edge search direction for extracting a feature amount from the overhead image synthesized by the image synthesis unit 2 in the feature amount extraction unit 402. As will be described later, the feature amount extraction unit 402 extracts a feature amount according to the line width of the marking line on the traveling surface of the vehicle by performing an edge search on the overhead image synthesized by the image synthesis unit 2. . The edge search direction at this time is preferably orthogonal to the marking line. Here, when the processing cycle of the arithmetic processing unit 4 is sufficiently short, when the vehicle is traveling at a low speed in the parking lot, the angle of the marking line found in the previous processing cycle does not change significantly. It is done. Therefore, the edge search direction determination unit 401 can determine the orthogonal direction with respect to the marking line from which the feature amount is extracted using such conditions. Specifically, the position and angle of the marking line found in the previous processing cycle in the overhead image is recorded, and this is used in the current processing cycle to obtain the orthogonal direction to the marking line. The edge search direction can be determined. In addition, the direction of the parking frame line is obtained from the parking lot information recorded in the map data used in the car navigation system (not shown), and the direction of the vehicle is obtained from the GPS or the geomagnetic sensor. It is also possible to calculate the orthogonal direction of the marking line to be searched and determine the edge search direction based on the direction.

  The overhead image synthesized by the image synthesis unit 2 is input to the feature amount extraction unit 402 in the arithmetic processing unit 4. The feature amount extraction unit 402 performs edge search in the edge search direction determined by the edge search direction determination unit 401 on the input overhead image, and thereby features according to the line width of the marking line drawn on the traveling surface of the vehicle. Extract the amount. Specifically, the bird's-eye view image is scanned at a predetermined interval in the edge search direction, and when a pair of luminance change points (edges) exceeding a predetermined threshold is found, they are extracted as feature points, and the distance between both feature points, That is, the distance between edges is recorded as a feature amount. By doing in this way, each feature-value according to the line | wire width of a marking line can be extracted for every predetermined interval. Here, in the bird's-eye view image, since the thickness of the road marking can be expected to be constant, it is possible to extract a constant feature amount with respect to the road marking. When a plurality of combinations of edges are found in the search range in the edge search direction, it is preferable to store all the distances between the edges as feature amounts. In this case, a feature amount used for recognition of road markings can be determined in a feature amount grouping unit 404 described later.

  In addition, the marking lines from which the feature quantity is extracted by the feature quantity extraction unit 402 include various road marking lines drawn on the road (roadway center line, lane boundary line, roadway outer line, pedestrian crossing, etc.) Includes parking frame lines drawn on the parking lot. Information on the feature amount extracted by the feature amount extraction unit 402 is output to the feature amount correction unit 403.

  The installation states of the cameras 1 a to 1 d detected by the installation state detection unit 3 are input to the feature amount correction unit 403 in the arithmetic processing unit 4. The feature amount correcting unit 403 corrects each feature amount extracted by the feature amount extracting unit 402 based on the input installation states of the cameras 1a to 1d. Here, if the positions and orientations of the cameras 1a to 1d are deviated from their original states, the photographing range changes according to the deviation. For this reason, when the overhead image is synthesized by the image synthesis unit 2 from the acquired captured image, the overhead image is distorted in accordance with the position and orientation shift of the cameras 1a to 1d. Such distortion of the overhead image increases as the distance from the vehicle increases. As a result, the line width of the marking line changes from the original one, and the feature quantity extraction unit 402 cannot correctly extract the feature quantity.

  Therefore, in the in-vehicle image processing apparatus 100 according to the present invention, the feature amount correction unit 403 predicts each feature amount from which the magnitude of distortion in the overhead image is extracted from the detection result of the installation state of the cameras 1a to 1d. Correction corresponding to the amount is performed on each feature amount. At this time, it is preferable to adjust the correction amount for each feature amount according to the position of the luminance conversion point corresponding to each feature amount on the overhead image. Information about each feature amount corrected by the feature amount correction unit 403 is output to the feature amount grouping unit 404.

  The feature amount grouping unit 404 groups the feature amounts corrected by the feature amount correction unit 403 according to their similarity, the continuity of the positions of the feature points on the overhead image, and the like. For example, feature quantities whose corresponding feature points are adjacent to each other on the overhead image and whose difference is within a predetermined range are grouped together and classified into the same group. Details of the grouping of feature amounts performed by the feature amount grouping unit 404 will be described later in detail with reference to FIG. Information of each feature quantity grouped by the feature quantity grouping unit 404 is output to the line extraction unit 405.

  The line extraction unit 405 extracts a marking line drawn on the traveling surface of the vehicle based on the feature amounts grouped by the feature amount grouping unit 404. Here, feature points corresponding to each feature amount of the same group are extracted as one marking line, and when there is a condition that satisfies a predetermined condition with another group, one marking is also performed for the group. Extract as a line. Thereby, various marking lines as described above are extracted from the overhead image. The line extraction result by the line extraction unit 405 is output to the road marking recognition unit 406.

  The road marking recognition unit 406 determines whether each marking line is a road marking based on the result of the marking line extraction by the line extraction unit 405, and if it is a road marking, the recognition result of the corresponding road marking. Is output to the output unit 5 of FIG. At this time, it is determined whether or not the vehicle is traveling in the parking lot, and only when it is determined that the vehicle is traveling in the parking lot, it is determined whether or not each marking line is a road marking. Is preferred. The determination as to whether or not the vehicle is traveling in a parking lot can use information such as vehicle speed and steering angle indicating the behavior of the vehicle, current position information and map information from the navigation device, and the like. Furthermore, whether the vehicle is traveling in the parking lot by detecting characteristic marking lines, structures, etc. in the parking lot or road based on the captured images of the cameras 1a to 1d and the synthesized overhead view image. You may determine whether or not.

  Next, details of feature quantity extraction performed by the feature quantity extraction unit 402 and feature quantity grouping performed by the feature quantity grouping unit 404 will be described. FIG. 5 is a diagram illustrating a specific example of feature amount extraction and grouping. Here, a specific example will be described in which feature amounts are extracted and grouped with respect to a marking line (road surface paint) as shown in FIG.

  In the case of extracting feature amounts for road surface paint as shown in FIG. 5A, the feature amount extracting unit 402 is indicated by a dotted line in FIG. 5B according to the edge search direction set by the edge search direction determining unit 401. Such a sampling line is set for each predetermined pixel interval (sampling interval). Then, the bird's-eye view image is sequentially scanned along the set sampling line, and the rise and fall of the luminance are detected as a set of feature points (edges), and the distance between them, that is, the line width is characterized. Extract as a quantity. As a result, for each sampling straight line, for example, feature quantities as shown in FIG. 5C are extracted. In FIG. 5C, the size of each feature is schematically shown by the width of a strip-shaped figure corresponding to each part of the original road surface paint. As described above, the feature amount extraction unit 402 can perform sampling in a straight line without considering the influence of lens distortion by using the overhead image.

  Each feature quantity extracted as described above is corrected by the feature quantity correction unit 403. The feature quantity grouping unit 404 groups the corrected feature quantities as shown in FIG. 5D in consideration of their similarity and continuity. Note that in FIG. 5D, each feature amount included in the same group is surrounded by a broken line.

  Here, for example, an identity condition and a monotonicity condition can be used as conditions for determining the similarity of feature amounts. The identity condition is that the absolute value of the difference between the two feature quantities to be compared falls within a predetermined first threshold range. On the other hand, the monotonicity condition means that when three or more feature quantities are compared, each feature quantity monotonously increases or decreases in the order of the positional relationship, and the increase or decrease quantity is within the second threshold range. It is in the range. In addition, it is preferable to make this 2nd threshold value larger than said 1st threshold value.

  FIG. 6 is a diagram illustrating a specific example of each feature amount extracted by the feature amount extraction unit 402. FIG. 6A shows a positional relationship with a predetermined reference point in the real space by enlarging a part of each feature amount exemplified in FIG. 5C. In FIG. 6A, the left end point is used as the reference point, but a point at another position, for example, the right end point or the midpoint of the left and right end points may be used as the reference point.

  FIG. 6B shows a list of the results of grouping based on the positional relationship between each feature amount shown in FIG. 6A and the reference point. In this list, for each feature quantity, the world coordinates, that is, the position in real space, the distance between edges indicating the size, and the group number indicating which group is classified by grouping are recorded. Yes.

  The world coordinates in FIG. 6B are expressed in a coordinate system on the traveling surface of the vehicle indicated by the overhead view image. That is, the first coordinate axis is set along the traveling direction (depth direction) of the vehicle, the second coordinate axis is set along the direction crossing the traveling direction of the vehicle, and the coordinate values on the coordinate plane defined by these coordinate axes are set. , The world coordinates of each feature. In FIG. 6A, the first coordinate axis is a coordinate axis with the upward direction in the figure as the positive direction, and the second coordinate axis is a coordinate axis with the right direction in the figure as the positive direction. The unit system of these coordinate axes is meter.

  FIG. 7 is a flowchart of processing executed in the in-vehicle image processing apparatus 100. In step S <b> 10, the installation state detection unit 3 detects the installation states of the cameras 1 a to 1 d and outputs the detection results to the arithmetic processing unit 4.

  In step S <b> 20, captured images are acquired by the cameras 1 a to 1 d and output to the image composition unit 2. In subsequent step S <b> 30, the overhead image is generated by the image composition unit 2 and output to the arithmetic processing unit 4.

  In step S40, the arithmetic processing unit 4 uses the edge search direction determination unit 401 to determine the edge search direction by the method described above.

  In step S <b> 50, the arithmetic processing unit 4 performs a feature amount extraction process using the feature amount extraction unit 402. Here, the feature quantity is extracted from the overhead image generated in step S20 by executing the feature quantity extraction process according to the flowchart of FIG. 8 described later.

  In step S60, the arithmetic processing unit 4 corrects each feature amount extracted in step S50 by the feature amount correction unit 403. Here, the correction amount of each feature amount is determined based on the installation state of the cameras 1a to 1d detected in step S10, and each feature amount is corrected according to the correction amount. Thereby, the error of each feature amount according to the distortion of the overhead image caused by the deviation of the positions and postures of the cameras 1a to 1d is corrected.

  In step S <b> 70, the arithmetic processing unit 4 performs a feature amount grouping process using the feature amount grouping unit 404. Here, the feature amount grouping process is executed according to the flowchart of FIG. 9 described later, thereby grouping the feature amounts corrected in step S60.

  In step S <b> 80, the arithmetic processing unit 4 performs line extraction processing by the line extraction unit 405. Here, by performing a line extraction process according to the flowchart of FIG. 10 described later, a marking line drawn on the traveling surface of the vehicle is extracted based on each feature amount grouped in step S70.

  In step S <b> 90, the arithmetic processing unit 4 recognizes the road marking by the road marking recognition unit 406. Here, it is determined whether or not each marking line extracted in step S80 is a road marking. If it is a road marking, the position of the corresponding road marking is recognized and output to the output unit 5.

  In step S100, the output unit 5 outputs road marking information based on the road marking recognition result output from the arithmetic processing unit 4 in step S90. If it is determined in step S90 that the vehicle is traveling on a road instead of a parking lot, or if the road marking cannot be recognized, information indicating that is preferably output as road marking information. . If step S100 is performed, the process shown in the flowchart of FIG. 7 will be complete | finished.

  Next, the feature amount extraction process in step S50 will be described. FIG. 8 is a flowchart of the feature amount extraction process.

  In step S51, the feature amount extraction unit 402 sets a sampling line for performing edge search for the overhead image synthesized by the image synthesis unit 2 in step S20 of FIG. Here, sampling lines are set at predetermined intervals according to the edge search direction determined in step S40 in FIG.

  The following processes in steps S52 to S55 are performed by sequentially selecting each pixel of the overhead image located between the start point and the end point of each sampling line set in step S51 as a processing target. In step S52, the feature amount extraction unit 402 determines whether or not the rising luminance difference in the selected pixel is equal to or greater than a predetermined threshold value. As a result, if it is less than the threshold, the next pixel is selected as a processing target, and if it is greater than or equal to the threshold, the pixel is detected as a luminance rising point, and the process proceeds to step S53. For example, when the luminance of a pixel adjacent to the pixel in the edge search direction is higher than the luminance of the pixel and the luminance difference is equal to or greater than a predetermined threshold, the pixel is detected as a luminance rising point. be able to.

  In the processing of the following steps S53 to S55, each pixel of the overhead image located between the luminance rising point detected in step S52 and the luminance falling point detected in step S53 or S54 described later is sequentially selected as a processing target. Done. In step S53, the feature amount extraction unit 402 determines whether the falling luminance difference in the selected pixel is equal to or greater than a predetermined threshold value. As a result, if it is less than the threshold value, the process proceeds to step S54. If it is equal to or greater than the threshold value, the pixel is detected as a luminance falling point, and the process proceeds to step S55. For example, when the luminance of a pixel adjacent to the pixel in the edge search direction is lower than the luminance of the pixel and the luminance difference is equal to or greater than a predetermined threshold, the pixel is detected as a luminance falling point. Can do.

  In step S54, the feature amount extraction unit 402 determines whether the selected pixel has reached the end point of the sampling line. As a result, if the end point has not been reached, the process proceeds to the next pixel. If the end point has been reached, the pixel is detected as a luminance falling point, and the process proceeds to step S55.

  In step S55, the feature amount extraction unit 402 calculates an edge interval as a feature amount from the distance between the luminance rising point detected in step S52 and the luminance falling point detected in step S53 or S54.

  In step S56, the feature quantity extraction unit 402 records the edge interval of each sampling line calculated in step S55, that is, the feature quantity information as feature point information in a memory (not shown). When step S56 is executed, the feature amount extraction process is completed, and the process proceeds to step S60 in FIG.

  Next, the feature amount grouping process in step S70 will be described. FIG. 9 is a flowchart of the feature amount grouping process.

  In step S71, the feature amount grouping unit 404 initializes a group number.

  The following steps S72 to S76 are performed by sequentially selecting each feature point as a processing target for all the feature points whose feature amounts have been corrected by the feature amount correcting unit 403 in step S60 of FIG. In step S72, the feature quantity grouping unit 404 determines whether the size of the corrected feature quantity at the selected feature point is within a predetermined range. As a result, if it is outside the predetermined range, the next feature point is selected as the processing target, and if it is within the predetermined range, the feature point is selected as the processing target, and the process proceeds to step S73.

  With the above processing, the feature quantity grouping unit 404 excludes those feature quantities corrected by the feature quantity correction unit 403 that are not within the predetermined target line width range, and falls within the target line width range. Only the feature quantities belonging to can be grouped. Note that an arbitrary value can be set as the target line width according to the type of the marking line to be extracted. For example, when 10 to 20 cm is set as the first target line width corresponding to the parking frame line and 50 to 70 cm is set as the second target line width corresponding to the pedestrian crossing, in the processing of the subsequent steps S73 to S76, Only the feature amounts within the range of the target line width can be grouped separately as grouping targets.

  In step S73, the feature amount grouping unit 404 determines whether there is a feature point to which a group number has already been assigned in the vicinity of the selected feature point. If such a feature point exists in the vicinity, the process proceeds to step S74, and if not, the process proceeds to step S75.

  In step S74, the feature quantity grouping unit 404 determines whether the corrected feature quantity at the selected feature point is similar to the feature quantity at the neighboring feature point determined in step S74. If the difference between these feature amounts is small, it is determined that they are similar, and the process proceeds to step S76. If the difference is large, it is determined that they are not similar, and the process proceeds to step S75.

  In step S75, the feature amount grouping unit 404 assigns a new group number to the selected feature point. As a result, the feature points are classified into a group different from the feature points already grouped.

  In step S76, the feature quantity grouping unit 404 registers the selected feature points in the same group as the neighboring feature points determined to have similar feature quantities in step S74, and assigns the same group number to these. Give. Thus, the feature point is added to any of already classified groups.

  In step S77, the feature quantity grouping unit 404 records the group number information assigned to each feature point as group information in a memory (not shown). When step S77 is executed, the feature amount grouping process is completed, and the process proceeds to step S80 in FIG.

  Next, the line extraction process in step S80 will be described. FIG. 10 is a flowchart of the line extraction process.

  In step S81, the line extraction unit 405 initializes the group number.

  The processes in steps S82 to S86 below are performed by sequentially selecting each group as one of a pair of processing target groups for all the groups classified by the feature amount grouping unit 404 in step S70 of FIG. . In step S82, the line extraction unit 405 determines whether the selected group has already been registered as a white line. If already registered, the next group is selected as a processing target, and if not registered, that group is selected as a processing target and the process proceeds to step S83.

  The following steps S83 to S86 are performed by sequentially selecting each group as the other of the pair of processing target groups for all other groups except the group currently selected. In step S83, the line extraction unit 405 determines whether or not a predetermined identity condition is satisfied between the selected pair of groups. As a result, if the identity condition is satisfied, the process proceeds to step S84, and if not, the next group is selected.

  In step S84, the line extraction unit 405 determines whether or not a predetermined angle condition is satisfied between the selected pair of groups. As a result, if the angle condition is satisfied, the process proceeds to step S85, and if not satisfied, the next group is selected.

  In step S85, the line extraction unit 405 determines whether or not a predetermined length condition is satisfied between the selected pair of groups. As a result, if the length condition is satisfied, the process proceeds to step S86, and if not, the next group is selected.

  In step S86, the line extraction unit 405 registers the selected pair of groups as the same white line.

  In step S87, the line extraction unit 405 outputs the information on each white line registered in step S86 to the road marking recognition unit 406 as a line extraction result. When step S87 is executed, the line extraction process is completed, and the process proceeds to step S90 in FIG.

  According to the embodiment described above, the following operational effects can be obtained.

(1) The in-vehicle image processing apparatus 100 includes cameras 1a to 1d that shoot around the vehicle and acquire captured images, an installation state detection unit 3 that detects installation states of the cameras 1a to 1d, and an arithmetic processing unit 4. Is provided. The arithmetic processing unit 4 includes a feature amount extraction unit 402, a feature amount correction unit 403, a feature amount grouping unit 404, a line extraction unit 405, and a road marking recognition unit 406. The feature amount extraction unit 402 extracts a feature amount corresponding to the line width of the marking line drawn on the traveling surface of the vehicle based on the captured image (step S50). The feature amount correction unit 403 corrects each feature amount extracted by the feature amount extraction unit 402 based on the installation state of the cameras 1a to 1d detected by the installation state detection unit 3 in step S10 (step S60). The feature quantity grouping unit 404 groups the feature quantities corrected by the feature quantity correction unit 403 (step S70). The line extraction unit 405 extracts a marking line drawn on the traveling surface of the vehicle based on the feature amounts grouped by the feature amount grouping unit 404 (step S80). The road marking recognition unit 406 recognizes the road marking based on the marking line extracted by the line extraction unit 405, and outputs the recognition result (step S90). Since it did in this way, speeding-up of a process can be achieved, suppressing the erroneous detection of a road marking.

(2) The feature quantity grouping unit 404 groups only the feature quantities that fall within a predetermined target line width among the feature quantities corrected by the feature quantity correction unit 403 (step S72). Since it did in this way, the feature-value corresponding to the marking line which is not related to a process can be excluded from the grouping object, a subsequent processing load can be reduced, and the further processing speed can be achieved.

(3) The target line width can include, for example, a first target line width corresponding to a parking frame line and a second target line width corresponding to a pedestrian crossing. In this case, the feature quantity grouping unit 404 separately groups each feature quantity belonging to the first target line width range and each feature quantity belonging to the second target line width range. (Steps S73 to S76). In this way, the parking frame line and the pedestrian crossing can be reliably extracted as separate lines.

(4) The installation state detection unit 3 detects the positions and orientations of the cameras 1a to 1d with respect to the traveling surface of the vehicle as the installation states of the cameras 1a to 1d. Since it did in this way, the installation state of the cameras 1a-1d which affect a feature-value can be detected reliably.

(5) The in-vehicle image processing apparatus 100 includes a plurality of cameras 1a to 1d and an image composition unit 2 that synthesizes an overhead image based on each captured image acquired by the plurality of cameras 1a to 1d. . The feature amount extraction unit 402 extracts a feature amount from this overhead image. Since it did in this way, the feature-value of the marking line drawn on the running surface around a vehicle can be extracted reliably.

  In the embodiment described above, the image composition unit 2 synthesizes the bird's-eye view image based on the captured images taken by the cameras 1a to 1d, and the feature amount extraction unit 402 extracts the feature amount from the bird's-eye view image. It was decided to. However, the feature amount may be directly extracted from the captured image of the camera without synthesizing the overhead view image.

(Modification 1)
A self-calibration that is a known technique may be used as an installation state detection unit that detects the installation state of the photographing apparatus. Self-calibration is also called targetless calibration, and the vehicle posture can be calculated by capturing images at a plurality of points while the moving body is traveling and tracking the same feature points appearing in the images. Furthermore, you may detect the installation state of an imaging device with the relative positional relationship from the own vehicle attitude | position. By doing so, an additional sensor device is not required, and the cost of the device can be reduced.

(Modification 2)
As an installation state detection unit that detects the installation state of the imaging device, the vehicle posture is calculated by observing road markings with known shapes such as the parallelism of the white road pairs that are known in the art and pedestrian crossing notices. The installation state of the imaging device may be detected based on the relative positional relationship from the vehicle attitude. By doing so, an additional sensor device is not required, and the cost of the device can be reduced.

  The embodiment and various modifications described above are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired.

DESCRIPTION OF SYMBOLS 1a, 1b, 1c, 1d Camera 2 Image composition part 3 Installation state detection part 4 Arithmetic processing part 5 Output part 100 Car-mounted image processing apparatus 401 Edge search direction determination part 402 Feature quantity extraction part 403 Feature quantity correction part 404 Feature quantity grouping 405 Line extraction unit 406 Road marking recognition unit

Claims (6)

An imaging device that captures an image of the surrounding area of the vehicle;
An installation state detection unit for detecting an installation state of the imaging device;
A feature amount extraction unit that extracts a feature amount corresponding to a line width of a marking line drawn on the traveling surface of the vehicle based on the captured image;
A feature amount correction unit that corrects each feature amount extracted by the feature amount extraction unit based on the installation state of the imaging device detected by the installation state detection unit;
A feature amount grouping unit that groups the feature amounts corrected by the feature amount correction unit;
A line extraction unit that extracts the marking line based on the feature amounts grouped by the feature amount grouping unit;
An in-vehicle image processing apparatus comprising: a road marking recognition unit that recognizes a road marking based on the marking line extracted by the line extraction unit and outputs the recognition result. The in-vehicle image processing apparatus according to claim 1,
The feature amount grouping unit groups only feature amounts belonging to a predetermined target line width among the feature amounts corrected by the feature amount correction unit. . The in-vehicle image processing apparatus according to claim 2,
The target line width includes a first target line width corresponding to a road marking and a second target line width corresponding to a pedestrian crossing,
The feature amount grouping unit separately groups each feature amount belonging to the range of the first target line width and each feature amount belonging to the range of the second target line width. An in-vehicle image processing apparatus characterized by the above. The in-vehicle image processing device according to any one of claims 1 to 3,
The in-vehicle image processing apparatus, wherein the installation state detection unit detects a position and an attitude of the imaging device with respect to a traveling surface of the vehicle as an installation state of the imaging device. The in-vehicle image processing device according to any one of claims 1 to 3,
A plurality of the photographing devices;
An image compositing unit that combines the overhead images based on the captured images acquired by the plurality of imaging devices;
The in-vehicle image processing apparatus, wherein the feature amount extraction unit extracts the feature amount from the overhead image. The in-vehicle image processing device according to any one of claims 1 to 5,
The in-vehicle image processing apparatus, wherein the road marking recognition unit recognizes a parking frame line as the road marking.

Images