JP6086424B2 - In-vehicle image processing device - Google Patents

Description

  The present invention relates to an in-vehicle image processing apparatus.

  2. Description of the Related Art Conventionally, a parking assistance device that captures an image around a vehicle and recognizes a frame line of a parking space from the captured image is known (for example, Patent Document 1).

JP 2011-16405 A

  Patent Document 1 has a problem in that when a white line having a U-shaped double line shape is present instead of a simple white line, the shape of the white line cannot be detected correctly.

The in-vehicle image processing apparatus according to claim 1, wherein the processing target image output unit outputs a processing target image based on a captured image obtained by capturing the periphery of the vehicle including the road surface, and the processing output by the processing target image output unit. from the target image, and recognizing the white line recognition unit each of a plurality of white lines existing in the road surface as an approximate straight line, from the white line detection unit recognized the plurality of white lines, the the two constituting the U-shaped double line shape A white line is specified, and for each of the two specified white lines, one of the two end points included in the approximate straight line of the white line is recalculated based on the processing target image, thereby obtaining one of the end points. When the two white lines specified by the correction unit for correcting coordinates and the correction by the correction unit are closer to parallel than before the correction, the correction result is adopted, and the specified And if the two white lines is not more parallel close than before the correction and a discarding correction result sifting unit results of the correction.

  According to the present invention, a white line having a U-shaped double line shape can be detected with high accuracy.

It is a block diagram which shows the structure of the vehicle-mounted image processing apparatus 100 by one Embodiment of this invention. It is a figure which shows typically the relationship between a vehicle and the rear camera. It is a figure for demonstrating the image composition process by the image composition part. It is a figure for demonstrating a white line recognition process. It is a flowchart of the white line recognition process which the white line recognition part 140 performs. 6 is a diagram for explaining the operation of a double line correction unit 150. 6 is a flowchart of a correction process executed by a double line correction unit 150. It is a figure for demonstrating operation | movement of the parking frame recognition part. It is a flowchart of the parking frame recognition process which the parking frame recognition part 160 performs. It is a figure for demonstrating the modification 4. FIG.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of an in-vehicle image processing apparatus 100 according to an embodiment of the present invention. An in-vehicle image processing apparatus 100 shown in FIG. 1 is used by being mounted on a vehicle, and includes a video acquisition unit 110, an image synthesis unit 120, a processed image selection unit 130, a white line recognition unit 140, and a double line correction unit 150. The parking frame recognition unit 160 and the output unit 170 are provided. Further, the rear camera 200 is connected to the video acquisition unit 110.

  FIG. 2 is a diagram schematically showing the relationship between the vehicle and the rear camera 200. FIG. 2A shows the vehicle 10 viewed from above, and FIG. Each side view is shown. The rear camera 200 is an electronic camera that is installed, for example, in the vicinity of a bumper or license plate of the vehicle 10 and photographs a predetermined range 20 behind the vehicle 10. The rear camera 200 is preferably configured to be able to photograph a wide range including the road surface behind the vehicle 10. The rear camera 200 captures the rear of the vehicle every predetermined period (for example, 1/30 second).

  The video acquisition unit 110 acquires, as image data (captured image data), a video behind the vehicle that is captured by the rear camera 200 at predetermined intervals. The image composition unit 120 performs image processing in accordance with a well-known algorithm to synthesize planar image (overhead image) data viewed from right above the vehicle from the image data acquired by the video acquisition unit 110.

  Since the rear camera 200 captures the rear of the vehicle every predetermined period, the image composition unit 120 synthesizes the overhead image data every predetermined period. The processed image selection unit 130 selects the overhead image data to be subjected to the parking frame recognition process from the plurality of overhead image data synthesized by the image synthesis unit 120 and delivers it to the white line recognition unit 140. There are various methods for selecting the overhead image data. For example, the overhead image data can be delivered to the white line recognition unit 140 only when a vehicle occupant performs a predetermined operation on an operation member (not shown), or a part of a predetermined number of overhead image data is thinned out. The white line recognition unit 140 may be handed over. Alternatively, the current location of the vehicle and the map data can be collated, and the overhead image data can be delivered to the white line recognition unit 140 only when the vehicle exists in the parking lot. That is, the processed image selection unit 130 outputs a processing target image based on a captured image obtained by capturing the periphery of the vehicle, and the processing target image is an overhead image converted from the captured image data by the image composition unit 120. It is data.

  The white line recognition unit 140 recognizes a white line existing on the road surface from the overhead image data delivered by the processed image selection unit 130. In the description of the present invention, a line constituting the parking frame drawn on the road surface is referred to as a “white line” for convenience, but the color of such a line is not necessarily white. That is, the “white line” in the description of the present invention is a concept including a line that is not white. The double line correction unit 150 applies a double line correction process to be described later to the white line recognition result by the white line recognition unit 140. The parking frame recognition unit 160 recognizes the parking frame based on the white line recognition result after correction by the double line correction unit 150.

  The output unit 170 outputs parking frame information for parking frames around the vehicle based on the recognition result of the parking frame by the arithmetic processing unit. For example, information indicating the direction of the parking frame relative to the vehicle and the distance to the parking frame is output as parking frame information. The parking frame information is output to a host vehicle control device (not shown) connected to the in-vehicle image processing device 100, and is used for vehicle parking assistance, travel control, and the like. For example, it is possible to automatically recognize that there is a parking lot around, and for example, in a parking lot environment, it is possible to automatically switch and display a bird's-eye view image around the vehicle on the monitor. As a result, it is possible to suppress a situation in which it is erroneously detected as a parking lot on a public road, and to switch the video presented to the user at an appropriate timing.

  Moreover, the output part 170 outputs parking frame information as relative positional information from the own vehicle based on the recognition result of the parking frame by an arithmetic processing part. For example, the position information of the parking frame in the actual environment, such as the end point coordinates of the left and right frame lines of the parking frame, the angle and intercept of the parking frame line, is output. The parking frame information is output to a host vehicle control device (not shown) connected to the in-vehicle image processing device 100, and is used for vehicle parking assistance, travel control, and the like. For example, by calculating the travel route to the parking frame based on the relative position and orientation from the own vehicle to the parking frame, by notifying the driver of the timing of brake and shift position change and the amount of steering angle operation, Parking assistance can be performed. As a result, a parking operation can be completed in a short time even for a driver who is unfamiliar with driving operations such as garage entry.

  Furthermore, the travel route to the parking frame is calculated based on the relative position and orientation from the own vehicle to the parking frame, and the control amount of the forward / backward / turning of the vehicle is automatically calculated. The movement may be automatically controlled. As a result, a parking operation can be completed safely and accurately even for a driver unfamiliar with driving operations such as garage entry.

  Hereinafter, the operation of each unit constituting the in-vehicle image processing apparatus 100 will be described in order.

(Description of operation of image composition unit 120)
FIG. 3 is a diagram for explaining image composition processing by the image composition unit 120. In the following description, as shown in the overhead view of FIG. 3A, it is assumed that two white lines 30 and 31 constituting a parking frame are present on the road surface behind the vehicle 10. The white line 30 is a U-shaped double line and separates two adjacent parking frames. The same applies to the white line 31.

  At this time, the video acquisition unit 110 acquires the video 40 shown in FIG. 3B from the rear camera 200. The image composition unit 120 synthesizes the overhead image data 41 shown in FIG. 3C by applying known image processing to the video 40.

(Description of the operation of the white line recognition unit 140)
The white line recognition unit 140 performs white line recognition processing described below on the bird's-eye view image data selected by the processing image selection unit 130 out of the bird's-eye view image data synthesized by the image synthesis unit 120.

  FIG. 4 is a diagram for explaining white line recognition processing. In the following description, it is assumed that the overhead image data 42 shown in FIG. 4A is delivered from the processed image selection unit 130. In this overhead view image data 42, in addition to the white lines 50 and 51 constituting the parking frame, there are other signs 52 present on the road surface, noise 53 which is dust reflected on the photographing screen, objects present on the road surface, and the like. include.

  In the white line recognition process, the white line recognition unit 140 first performs edge extraction in the horizontal direction for each horizontal line of the overhead image data 42. As an example, edge extraction in a line indicated by a broken line L1 in FIG. The luminance value 60 of each pixel constituting this line is shown in FIG. When the differential value of the luminance value 60 of each pixel is calculated, the result is as shown in FIG. The white line recognition unit 140 determines that rising edges and falling edges exist at the maximum position and the minimum position of the differential value 61.

  For example, in the case of FIG. 4C, the white line recognition unit 140 determines that rising edges exist at positions 62 and 64 and falling edges exist at positions 63 and 65. FIG. 4D shows the result of the above edge extraction performed on all the horizontal lines in FIG. The edge extraction result 43 shown in FIG. 4D includes eight edge sets 66 to 73, the edge sets 66, 68, 70, and 72 are rising edges, and the edge sets 67, 69, 71, and 73 are falling edges. It is an edge.

  Next, in the edge extraction result 43 in FIG. 4D, the white line recognizing unit 140 leaves only the edge where the pair of the rising edge and the falling edge is established, and excludes the other edges from the subsequent processing. This is because, when a white line exists, it is considered that the rising edge and the falling edge always appear as a pair. In addition, a pair whose interval between the rising edge and the falling edge is smaller than a predetermined lower limit value (for example, 3 cm) and a pair larger than a predetermined upper limit value (for example, 30 cm) are also excluded from the subsequent processing. The predetermined upper limit value and lower limit value are determined based on the thickness of the white line used as a parking frame.

  For example, in FIG. 4D, since the edge 66 is paired with the edge 67, these two edges remain. Similarly, a pair of edge 68 and edge 69 and a pair of edge 70 and edge 71 are also left. On the other hand, the pair of the edge 72 and the edge 73 is excluded from the subsequent processing because the distance between the two is too close.

  Next, the white line recognizing unit 140 groups (groups) the continuous edges in the vertical direction among the remaining edges. Then, pairs in which at least one edge is shorter than a predetermined threshold in each group are excluded from the subsequent processing. This is because a white line used as a parking frame has a certain length or more, and an extremely short edge is considered not to be a white line. For example, in FIG. 4D, the pair of the rising edge 70 and the falling edge 71 are both too short and are excluded from the subsequent processing. Accordingly, in the subsequent processing, among the edges shown in FIG. 4D, only four groups of the group consisting of the edge 66, the group consisting of the edge 67, the group consisting of the edge 68, and the group consisting of the edge 69 are included. It becomes a processing target.

  Finally, the white line recognition unit 140 determines the upper and lower ends of the edges included in each remaining group, and approximates the edges with a straight line connecting the upper and lower ends. FIG. 4E shows a state in which the edge included in the remaining group in FIG. 4D is approximated by a straight line. A straight line approximating the edge 66 is a straight line 74, and a straight line approximating the edge 67 is a straight line 75. Similarly, a straight line approximating the edge 68 is a straight line 76, and a straight line approximating the edge 69 is a straight line 77. A white line is considered to exist in a space 78 between the straight line 74 and the straight line 75 and a space 79 between the straight line 76 and the straight line 77 that are paired with each other.

  FIG. 5 is a flowchart of white line recognition processing executed by the white line recognition unit 140. First, in step S10, the white line recognition unit 140 performs edge extraction by applying a lateral edge filter to all horizontal lines of the overhead image data. In step S20, the peak positions of the rise and fall of the luminance value are extracted.

  In subsequent step S30, the white line recognition unit 140 determines that the edge where the pair of the rising edge and the falling edge is not established, and the edge where the pair is established but the interval is outside the predetermined range (the edge is too large or too small). ) Are excluded from the subsequent processing, and only other edges are left. In the next step S40, the consecutively existing edges are grouped, and only the group whose length is equal to or greater than a predetermined threshold is left. In step S50, the coordinates of the upper and lower ends of the rising edge and the falling edge are extracted, and each edge is approximated by a straight line connecting the upper end and the lower end.

  The white line recognition unit 140 recognizes a white line existing in the overhead image data by executing the white line recognition process described above. Edges whose pair interval is too narrow or edges whose length is too short are excluded. For example, the marking 52 and the noise 53 other than the white line shown in FIG. 4A are recognized as white lines by the white line recognition unit 140. There is no.

(Description of operation of double line correction unit 150)
FIG. 6A is a diagram illustrating an example of a bird's-eye view image data of a road surface and an approximate straight line of a white line recognized from the bird's-eye view image data. As shown in FIG. 6A, when the white line 32 that is a U-shaped double line is recognized by the white line recognition unit 140, the U-shaped end portion (curved portion) is an approximate straight line. Recognized as the top or bottom edge. Therefore, as shown in FIG. 6A, it is recognized that one end of the approximate line exists at the bottom of the U shape (curved part), and the approximate line of the two white lines constituting the double line is a U character. Tapered toward the bottom (curved part) of the mold. As a result, as shown in FIG. 6B, the shapes of the spaces 84 and 85 between the two approximate straight lines forming a pair are greatly deviated from the actual white line.

  Therefore, in the present embodiment, the double line correction unit 150 corrects the white line recognition result (approximate straight line position) by the white line recognition unit 140 to an approximate straight line that is closer to the actual white line. Hereinafter, correction processing by the double line correction unit 150 will be described.

  First, the double line correction unit 150 selects two white lines that are considered to form a U-shaped double line from a plurality of white lines recognized by the white line recognition unit 140. When the upper ends and the lower ends of the two white lines are greatly separated from each other, the two white lines are considered not to form a U-shaped double line, and thus no correction processing is performed. In the following description, two white lines (a white line constituted by the approximate straight lines 80 and 81 and a white line constituted by the approximate straight lines 82 and 83) shown in FIG. Assume a state.

  In the correction process, the double line correction unit 150 first selects the narrower end point between the approximate lines corresponding to the rising edges of the two white lines (the approximate line 80 and the approximate line 82), and the other end point is selected from that end point. In the line in which the Y coordinate is shifted by a predetermined length toward, an edge is searched in the horizontal direction.

  In FIG. 6C, when the approximate straight line 80 and the approximate straight line 82 are viewed, the lower end interval is narrower than the upper end interval, so that the Y coordinate is set to the upper end by a predetermined length H from the horizontal line L2 where the lower end exists. In the horizontal line L3 shifted toward, a rising edge is searched in the horizontal direction. As a result, a rising edge is found at the position 86 on the approximate straight line 80 side, and a rising edge is found at the position 89 on the approximate straight line 82 side.

  The double line correction unit 150 considers a straight line passing through the edge (position 86, 89) found here and the upper end of the approximate straight lines 80, 82, and obtains positions 87, 90 where the straight line intersects the horizontal line L2. If the new straight lines 88 and 91 connecting the positions 87 and 90 obtained here and the upper ends of the approximate straight lines 80 and 82 are parallel to the approximate straight lines 80 and 82, the approximate straight lines at the positions 87 and 90 are obtained. The lower ends of 80 and 82 are corrected.

  That is, if the straight line 88 and the straight line 91 are more parallel than the approximate straight line 80 and the approximate straight line 82, the approximate straight lines 80 and 82 are corrected by replacing the approximate straight line 80 with the straight line 88 and replacing the approximate straight line 82 with the straight line 91. To do. In the example of FIG. 6C, since the straight line 88 and the straight line 91 are more parallel than the approximate straight line 80 and the approximate straight line 82, the double line correction unit 150 replaces the approximate straight line 80 with the straight line 88 and replaces the approximate straight line 82. Replace with a straight line 91. Therefore, the final straight lines constituting the two white lines are a straight line 88 and a straight line 91 shown in FIG.

  The double line correction unit 150 repeats the processing described above for the approximate straight lines corresponding to the falling edges of the two white lines (the approximate straight line 81 and the approximate straight line 83). At this time, what is searched for in the horizontal line L3 is not the rising edge but the falling edge. The above is the correction processing.

  FIG. 7 is a flowchart of the correction process executed by the double line correction unit 150. First, in step S100, the double line correction unit 150 selects two arbitrary white lines from all white lines recognized by the white line recognition unit 140. In step S110, it is determined whether the distance (interval) between the upper ends of the two selected white lines is equal to or less than a predetermined value. If the interval is equal to or smaller than the predetermined value, the process proceeds to step S120, and if not, the process proceeds to step S170.

  Similarly, in step S120, it is determined whether the distance (interval) between the lower ends of the two selected white lines is equal to or less than a predetermined value. If the interval is equal to or smaller than the predetermined value, the process proceeds to step S130, and if not, the process proceeds to step S170. In step S130, an edge is searched for in the horizontal direction on a line in which the Y coordinate is shifted by a predetermined length from the end point having the smaller interval between the upper end and the lower end to the other end point. In step S140, the position of the upper end or the lower end is recalculated based on the straight line passing through the searched edge.

  In subsequent step S150, the double line correction unit 150 determines whether the two white lines recalculated in step S140 are more parallel than before the recalculation. As a result of the recalculation, the process proceeds to step S160 if it is more parallel than before the recalculation, and if not, the process proceeds to step S170. In step S160, the upper or lower ends of the two white lines selected in step S100 are corrected (replaced) with the coordinates recalculated in step S140.

  In step S170, it is determined whether or not the processing in step S110 and subsequent steps has been executed for all white line combinations. If there is a combination of two white lines that have not yet been processed, the process returns to step S100, and if the processing has been completed for all the combinations of white lines, the process of FIG. 7 ends. That is, the double line correction unit 150 executes the processing of steps S110 to S160 for all combinations that select two white lines among the plurality of white lines recognized by the white line recognition unit 140.

(Description of operation of parking frame recognition unit 160)
FIG. 8 is a diagram for explaining the operation of the parking frame recognition unit 160. The parking frame recognition unit 160 recognizes the parking frame by specifying two white lines existing on the left and right sides thereof. For example, as shown in FIG. 8 (a), if the parking lot is a parking lot in which parking frames are separated by a U-shaped double line, the parking frame recognition unit 160 is included in the U-shaped double line. The parking frame 96 is recognized by specifying two white lines 34 and 35 (white lines directly adjacent to the parking frame) on the left and right sides. On the other hand, as shown in FIG. 8B, if the parking lot is a parking lot where the parking frames are separated by a single white line 36, 37, the parking frame recognition unit 160 specifies each white line 36, 37. Thus, the parking frame 97 is recognized. That is, the parking frame recognition unit 160 of the present embodiment can recognize the parking frame for both the parking lot shown in FIG. 8A and the parking lot shown in FIG. 8B. .

  The parking frame recognizing unit 160 selects two white lines from all the recognized white lines for all combinations as in the correction processing by the double line correcting unit 150, and the two white lines have a relationship unique to the parking frame. A parking frame is recognized by determining whether it is satisfied. Hereinafter, the parking frame recognition process by the parking frame recognition unit 160 will be described assuming that the two white lines 92 and 93 shown in FIG. 8C are selected.

  The parking frame recognition unit 160 first calculates angles θ1 and θ2 formed by the two white lines 92 and 93 with respect to the vehicle 10. When the difference between θ1 and θ2 exceeds a predetermined threshold (for example, 3 degrees), it is determined that the two white lines 92 and 93 do not constitute a parking frame. This is because the two white lines constituting the parking frame are generally parallel.

  Next, the parking frame recognition unit 160 determines whether or not the interval W1 between the two white lines 92 and 93 is out of a predetermined range (for example, 2.4 m to 2.7 m). And when the space | interval W1 has remove | deviated from this range, it determines that the two white lines 92 and 93 do not comprise a parking frame. This predetermined range is determined based on the size of a general parking lot. That is, this determination is for confirming that the interval between the two white lines does not deviate from the size expected for a parking lot.

  Next, the parking frame recognition unit 160 determines whether or not the amount of deviation W2 at the lower ends of the two white lines 92 and 93 exceeds a predetermined value (for example, 30 cm). And when the deviation | shift amount W2 exceeds this predetermined value, it determines that the two white lines 92 and 93 do not comprise a parking frame. If it is a white line constituting the parking frame, it is expected that the end points are aligned. Therefore, when the displacement amount W2 is too large, it is considered that the two white lines 92 and 93 do not constitute a parking frame.

  When all the above determinations are cleared, the parking frame recognition unit 160 determines that the two white lines 92 and 93 constitute a parking frame. And as shown in FIG.8 (d), the area | region 94 enclosed by the two white lines 92 and 93 is recognized as a parking frame.

  FIG. 9 is a flowchart of parking frame recognition processing executed by the parking frame recognition unit 160. First, in step S200, the parking frame recognition unit 160 selects two arbitrary white lines from all the white lines recognized by the white line recognition unit 140 (including the white lines corrected by the double line correction unit 150). In step S210, it is determined whether or not the angle difference between the two white lines is equal to or smaller than a predetermined value Thθmax. If the angle difference is Thθmax, the process proceeds to step S220, and if not, the process proceeds to step S250.

  In the next step S220, it is determined whether the interval between the two white lines falls within a predetermined range ThWmin to ThWmax, that is, whether the interval is greater than or equal to ThWmin and less than or equal to ThWmax. If the distance between the two white lines falls within the predetermined range, the process proceeds to step S230, and if not, the process proceeds to step S250.

  In subsequent step S230, the parking frame recognition unit 160 determines whether or not the amount of deviation between the lower ends of the two white lines is equal to or less than a predetermined value ThBmax. If the amount of deviation is equal to or less than ThBmax, the process proceeds to step S240, and if not, the process proceeds to step S250. In step S240, the coordinates of the upper and lower ends of the two white lines, that is, the coordinates of the four corners of the parking frame are registered as parking frames. The output unit 170 outputs the aforementioned parking frame information based on the coordinates of the four corners registered here.

  In step S250, it is determined whether or not the processing after step S210 has been executed for all combinations of white lines. If there are two white line combinations that have not yet been processed, the process returns to step S200, and if all the white line combinations have been processed, the process of FIG. 9 ends. That is, the parking frame recognizing unit 160 executes the processes in steps S210 to S240 for all combinations that select two white lines among the plurality of white lines recognized by the white line recognizing unit 140.

According to the vehicle-mounted image processing apparatus according to the first embodiment described above, the following operational effects are obtained.
(1) The image composition unit 120 converts a captured image obtained by photographing the periphery of the vehicle 10 into an overhead image from a viewpoint above the vehicle 10. The white line recognition unit 140 recognizes each of a plurality of white lines existing on the road surface as an approximate straight line from the overhead image. The double line correction unit 150 identifies two white lines constituting the U-shaped double line shape from the recognized white lines, and each of the identified two white lines is included in an approximate straight line of the white line. One coordinate of the end point is corrected by recalculating one coordinate of the two end points based on the overhead image. The double line correction unit 150 adopts the result of the correction when the two white lines become more parallel due to the correction, and discards the result of the correction when the two white lines do not become more parallel. Since it did in this way, the white line which has U-shaped double line shape can be detected accurately.

(2) The double line correction unit 150 has a distance between one end point of a plurality of recognized white lines that is equal to or less than a predetermined threshold value and a distance between the other end points is equal to or less than a predetermined threshold value. Two white lines are specified as two white lines constituting a U-shaped double line shape. As a result, two white lines that do not actually form a U-shaped double line shape are mistakenly recognized as two white lines that form a U-shaped double line shape. There is nothing. In addition, as shown in FIG. 6, the recognition result before correction of the two white lines that actually configure the U-shaped double line shape tends to bring the end points closer to each other than the correct position. Even in this case, the two white lines that actually constitute the U-shaped double line shape are not erroneously recognized as the two white lines that are not.

(3) For each of the two specified white lines, the double line correction unit 150 calculates the coordinates of the end point that is closer to the other white line among the two end points included in the approximate line of the white line. Recalculate based on the overhead image. Since it did in this way, the position of the two white lines which comprise a U-shaped double line | wire shape can be correct | amended exactly with a small amount of calculations.

(4) The white line recognition unit 140 recognizes a plurality of white lines by performing an edge detection process on the overhead image, and the double line correction unit 150 performs an approximate straight line of the white line for each of the two specified white lines. In the position shifted from one of the two end points to the other of the two end points, edge detection processing is performed on the bird's-eye view image to recalculate one coordinate of the two end points. Since it did in this way, the position of the two white lines which comprise a U-shaped double line | wire shape can be correct | amended exactly with a small amount of calculations.

(5) The rear camera 200 outputs a photographed image by photographing the periphery of the vehicle 10. The parking frame recognition unit 160 selects a pair of white lines constituting the parking frame from the plurality of recognized white lines, and calculates the coordinates of the parking frame based on the selected pair of white lines. Since it did in this way, the position of the parking frame divided | segmented by the white line which has U-shaped double line shape can be pinpointed correctly.

  The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.

(Modification 1)
In the first embodiment described above, the parking frame is recognized using the rear camera 200 that captures the rear of the vehicle. The present invention is not limited to such an embodiment. For example, it is possible to recognize the parking frame based on the result of photographing the front of the vehicle. Further, it is also possible to use in combination with the present invention a technique for creating a planar image viewed from directly above a vehicle based on images taken by four cameras provided at the front, rear, and left and right sides of the vehicle. . That is, the in-vehicle image processing apparatus 100 may be configured so that the image composition unit 120 synthesizes the above-described planar image from images captured by four cameras.

  Further, when a front camera for photographing the front of the vehicle 10 is provided in addition to the rear camera 200 for photographing the rear of the vehicle 10, these two cameras can be properly used. For example, when the vehicle 10 is in the forward state, the parking frame is specified using a plurality of white lines recognized in the captured image captured by the front camera, and when the vehicle 10 is in the reverse state, the image is captured by the rear camera 200. The parking frame recognition unit 160 can be configured to specify a parking frame using a plurality of white lines recognized in the image.

(Modification 2)
In the first embodiment described above, one white line is constituted by two approximate lines corresponding to the rising edge and the falling edge. The present invention is not limited to such an embodiment, and one white line may be constituted by one approximate straight line corresponding to either the rising edge or the falling edge.

(Modification 3)
In the first embodiment described above, a white line is detected based on an image obtained by converting an image obtained from the rear camera 200 that captures the rear of the vehicle into an overhead image. The present invention is not limited to such an embodiment, and a double line may be corrected based on a white line detected from an image before conversion into an overhead image. For example, the white line is detected as an approximate line directly from the captured image obtained from the rear camera 200, and the parallelism and distance between the two lines are calculated based on the result of converting the end points of the approximate line to world coordinates. It is also possible to perform the correction by calculating and re-searching the edge of the double line on the captured image by the double line correction unit 150. However, when using a bird's-eye view image, there is an advantage that the algorithm can be constructed simply because the parallelism and distance between the two straight lines can be easily calculated as compared with the processing in the captured image.

(Modification 4)
The correction process by the double line correction unit 150 can correct not only the U-shaped double line that constitutes the parking frame but also misrecognition caused by the occurrence of disturbance in two adjacent lines. it can. For example, as shown in FIG. 10, in the recognition of a running lane mark (consisting of white lines 98a and 98b), when detecting whether the lane mark is a double line, a disturbance 99 such as dust or dirt occurs. Then, as in the case of the U-shaped double line, it may be determined that the white line 98a and the white line 98b are not parallel. Even in such a case, by using the above-described correction method, the recognition result of the white line can be corrected in parallel without taking a processing time, and stable double line detection can be performed.

  As long as the characteristics of the present invention are not impaired, the present invention is not limited to the above-described embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. .

DESCRIPTION OF SYMBOLS 100 ... Car-mounted image processing apparatus, 110 ... Image | video acquisition part, 120 ... Image composition part, 130 ... Processed image selection part, 140 ... White line recognition part, 150 ... Double line correction part, 160 ... Parking frame recognition part, 170 ... Output 200, rear camera

Claims (7)

A processing target image output unit that outputs a processing target image based on a photographed image obtained by photographing the periphery of the vehicle including the road surface ;
From outputted the processing target image by the processing target image output unit, and recognizes the white line recognition unit each of a plurality of white lines existing in the road surface as an approximate straight line,
Two white lines constituting a U-shaped double line shape are identified from the plurality of white lines recognized by the white line recognition unit, and each of the identified two white lines is included in the approximate straight line of the white line 2 A correction unit for correcting one coordinate of the end point by recalculating one coordinate of the two end points based on the processing target image;
When the two white lines specified by the correction by the correction unit become more parallel than before the correction, the result of the correction is adopted, and the two specified white lines are compared with those before the correction. A correction result discarding unit that discards the correction result when it is not nearly parallel,
A vehicle-mounted image processing apparatus comprising: The in-vehicle image processing apparatus according to claim 1,
The processed image output unit, the vehicle-mounted image processing apparatus for outputting an overhead image obtained by converting the captured image to the image from the upper viewpoint of the vehicle as the processing target image. The in-vehicle image processing apparatus according to claim 2 ,
The correction unit has a distance between one end point of the plurality of white lines recognized by the white line recognition unit being a predetermined threshold value or less and a distance between the other end points being a predetermined threshold value or less. An in-vehicle image processing apparatus that identifies two white lines as two white lines constituting the U-shaped double line shape. The in-vehicle image processing device according to claim 2 or 3 ,
The correction unit, for each of the identified two white lines, based on the overhead image, the coordinates of the end point that is closer to the other white line among the two end points included in the approximate line of the white line In-vehicle image processing device that recalculates. The in-vehicle image processing device according to any one of claims 2 to 4,
The white line recognition unit recognizes the plurality of white lines by performing an edge detection process on the overhead image.
The correction unit detects, for each of the two specified white lines, edge detection for the overhead image at a position shifted from one of two end points included in the approximate line of the white line toward the other of the two end points. An in-vehicle image processing apparatus that recalculates one coordinate of the two end points by performing processing. The in-vehicle image processing device according to any one of claims 2 to 5,
A selection unit that selects a pair of white lines constituting a parking frame from the plurality of recognized white lines;
A vehicle-mounted image processing apparatus comprising: a calculation unit that calculates coordinates of the parking frame based on a pair of white lines selected by the selection unit. The in-vehicle image processing device according to claim 6,
The selection unit selects the pair of white lines based on the captured image captured by a front camera that captures the front of the vehicle when the vehicle is in a forward state, and the vehicle when the vehicle is in a reverse state. A vehicle-mounted image processing apparatus that selects the pair of white lines based on the captured image captured by a rear camera that captures the rear of the camera.

Images