JP2007310591A - Image processor and parking space determination method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and a parking space determination method for determining whether a parking space having a solid object such as a car stop between two white straight lines exists near a vehicle. <P>SOLUTION: An image picked up by a camera is obtained, and converted to a top-view image by Topview conversion to extract a white line by binarization. The binarized image is Hough-transformed. Points crossing many Hough curves corresponding to each point of the image Hough transformed are extracted to extract straight lines. The straight lines are extracted and obtained by inverse Hough transform. A distance to an object in the image obtained by the camera is calculated to detect the solid object based on the calculation result. When the solid object exists between the extracted two straight lines, it is determined that a parking space exists near the vehicle. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a parking lot determination method that can determine whether or not a parking lot exists in the vicinity of a vehicle from an image captured by a camera mounted on the vehicle.

  2. Description of the Related Art Conventionally, a system in which a camera is mounted on the front side, rear side, or left / right side of a vehicle and various information is provided to a driver based on an image captured by the camera has been put into practical use. For example, by mounting a camera that captures a blind spot from the driver's seat of the vehicle, a portion that cannot be directly seen by the driver can be displayed on the display inside the vehicle. In addition, by recognizing a white straight line from an image captured by the camera, it is possible to realize a function of issuing a warning when the vehicle deviates from the traveling lane, a parking assist function for guiding the vehicle to the parking lot, or the like. .

  As a method for recognizing a white straight line from an image captured by a camera, Hough transform (Hough transform) has been conventionally used. The Hough transform converts each point (x, y) of the image from the xy plane to a Hough curve of ρ = x cos θ + ysin θ on the ρ θ plane, and on the xy plane corresponding to the point (ρ, θ) where these curves intersect most frequently. This is a method of detecting a straight line at. Note that ρ corresponds to the length of a perpendicular line drawn from the origin to a straight line, and θ corresponds to the angle formed by the perpendicular line and the x axis. The Hough transform can detect straight lines regardless of whether the straight line is a solid line or a broken line, and can detect a straight line even if the thickness of the straight line varies. is there.

  In Patent Document 1, the type of road surface is determined from an image captured by a camera, and when a white line is provided on the road surface, the target parking area of the vehicle is set based on the white line, and the white line is displayed on the road surface. There has been proposed an image processing apparatus that can detect a mark when a vehicle is guided to a parking lot with high accuracy by setting a target parking area of the vehicle based on an obstacle when it is not provided. In this image processing apparatus, the Hough transform is used when determining whether or not a white line is provided on the road surface.

In Patent Document 2, a camera that images the front of the vehicle, a camera that images the front and left and right sides of the vehicle, and an infrared laser camera that acquires distance information of the left and right sides and the rear of the vehicle for each pixel are provided. In addition, a parking support system has been proposed that can automatically detect a parking position based on image information from each camera and distance information from an infrared laser camera and provide parking support information.
JP 2004-34946 A JP 2006-96312 A

  However, when the white line is provided on the road surface, the image processing apparatus of Patent Document 1 sets an area surrounded by the white line as a target parking section, and a white line such as a lane or a pedestrian crossing and a parking lot Cannot be distinguished from the white line. Therefore, this image processing apparatus is premised on the use of the driver after moving the vehicle to the nearest parking lot, and can only be used for limited applications. Further, the parking support system described in Patent Document 2 has a problem of high cost because it is necessary to mount an infrared laser camera on the vehicle in order to acquire distance information.

  In recent years, various functions for suppressing accidents during traveling such as a function of automatically adjusting the distance between other vehicles traveling in front of and behind the vehicle or a function of detecting a pedestrian at night and issuing a warning to the driver. Efforts have been made to increase the safety when the vehicle is running by mounting the function on the vehicle. Since there is a high possibility that there are pedestrians in the vicinity of the parking lot and it is dangerous for the vehicle to travel at high speed in the vicinity of the parking lot, it is desirable that the vehicle speed can be automatically adjusted in the vicinity of the parking lot. . For this purpose, it is necessary to determine whether or not a parking lot exists in the vicinity of the vehicle even after the vehicle has not been moved to the immediate vicinity of the parking lot.

  The present invention has been made in view of such circumstances, and an object of the present invention is to extract two straight lines from an image picked up by the image pickup means and detect a three-dimensional object between the two straight lines. By determining whether or not there is a parking lot based on whether or not a three-dimensional object is present, determining whether or not there is a parking lot in which a three-dimensional object such as a ring stop is provided between two white straight lines An object of the present invention is to provide an image processing apparatus capable of

  Another object of the present invention is to take an image of the periphery of the vehicle by an imaging means having a degree of curvature of an image to be imaged at the end portion rather than the center portion of the field of view, and correct the curved portion of the image so as to look down. Therefore, the present invention provides an image processing apparatus that can image the periphery of a vehicle over a wide range and determine the presence or absence of a parking lot.

  Another object of the present invention is to extract two white straight lines from the captured image by Hough transform, so that the white line is less likely to be mistakenly extracted, and the presence or absence of a parking lot is surely detected. An object of the present invention is to provide an image processing apparatus capable of determining.

  Another object of the present invention is to calculate the distance to the object captured in each pixel of the captured image, and according to the difference in distance to the object captured in the adjacent pixel. By providing a configuration for detecting a three-dimensional object from an image, an image processing apparatus capable of reliably detecting a three-dimensional three-dimensional object from a two-dimensional image and reliably determining the presence or absence of a parking lot is provided. There is to do.

  Another object of the present invention is to mount a plurality of image pickup means on a vehicle, and when the plurality of image pickup means can pick up the same object from different positions, the plurality of image pickup means respectively By adopting a configuration that calculates the distance from the vehicle to the object based on the imaging position of the same object in the captured image, the distance from the vehicle to the object can be accurately calculated, and the image is captured. An object of the present invention is to provide an image processing apparatus capable of reliably detecting a three-dimensional object from an image.

  Another object of the present invention is to calculate the distance from the vehicle to the object based on the imaging positions of the same object in a plurality of images taken at different times by the imaging means. Thus, an object of the present invention is to provide an image processing apparatus that can accurately and inexpensively calculate the distance from a vehicle to an object.

  Another object of the present invention is to use an image processing apparatus having an imaging means to extract two straight lines from a captured image and detect a three-dimensional object, and a three-dimensional object exists between the two straight lines. A parking lot determination method capable of determining the presence or absence of a parking lot in which a three-dimensional object such as a ring stop is provided between two white straight lines by determining the presence or absence of a parking lot based on whether or not It is to provide.

  An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that processes an image obtained by an imaging unit that images the outside of a vehicle. An extraction unit that extracts a straight line from the image, and a detection unit that detects a three-dimensional object from the image. And determining means for determining the presence or absence of a parking lot based on whether or not the three-dimensional object detected by the detecting means exists between the two straight lines extracted by the extracting means.

  In the present invention, two straight lines are extracted from the image picked up by the image pickup means and a three-dimensional object is detected, and whether or not there is a parking lot is determined based on whether a three-dimensional object exists between the two straight lines. Parking lots are often configured with one or two rectangular parallelepiped rings between two white straight lines. Do two straight lines and solid objects between the straight lines exist in the image? It is possible to determine the presence or absence of a parking lot depending on whether or not. In addition, since there are few solid objects between white lines such as lanes or pedestrian crossings, these white lines are not mistakenly identified as parking lots, and the vehicle moves close to the parking lot. Even if it is not, the presence or absence of a parking lot can be determined.

  In the image processing apparatus according to the second aspect of the invention, the imaging unit increases the degree of curvature of the image captured at the end rather than the center of the field of view, and the curved portion of the image captured by the imaging unit is detected. The image processing apparatus includes a conversion unit that corrects and converts the image into a bird's-eye view image, and the extraction unit extracts two straight lines from the image converted by the conversion unit.

  In the present invention, the periphery of the vehicle is imaged in a wider range by capturing an image of the periphery of the vehicle with an imaging means having a greater degree of curvature of the image captured at the end than the center of the field of view, such as a camera having a fisheye lens or a wide-angle lens. The image is taken over. In addition, by correcting the curved portion of the captured image and converting it into an image in the bird's-eye view, two straight lines for determining the presence or absence of a parking lot are reliably extracted.

  The image processing apparatus according to the third invention is characterized in that the extraction means extracts a white straight line from the image by Hough transform.

  In the present invention, a white straight line is extracted from the captured image by Hough transform. In the Hough transform, a straight line can be detected whether it is a solid line or a broken line, and a straight line can be detected even if there is a variation in the thickness of the straight line. Even if it is partially damaged, a white straight line can be reliably extracted, and erroneous extraction is unlikely to occur.

  The image processing apparatus according to a fourth aspect of the present invention includes a calculation unit that calculates a distance to an object captured in each pixel for each pixel of the image obtained by the imaging unit, and the detection unit includes: The three-dimensional object is detected from the image according to the difference in the distance calculated by the calculation unit for adjacent pixels.

  In the present invention, the distance to the object captured in each pixel of the captured image is calculated, and the three-dimensional object is detected from the image according to the difference in distance to the object captured in the adjacent pixel. To do. For example, when there is no three-dimensional object on the road surface, the imaged road surface and the vehicle gradually move away at a certain distance, but when there is a three-dimensional object on the road surface, at a certain distance at the position where the three-dimensional object exists. Since the road surface and the vehicle are not far away, the presence of a three-dimensional object can be detected.

  In the image processing apparatus according to the fifth aspect of the present invention, the imaging unit includes a plurality of imaging units, the plurality of imaging units can capture the same object from different positions, and the calculation unit includes a plurality of the imaging units. The distance to the object is calculated based on the imaging position of the same object in the images captured by the imaging means.

  In the present invention, a plurality of imaging means are mounted on the vehicle so that the same object can be imaged from different positions. When the same object is imaged from different positions, the imaging position of the same object on the images acquired by the respective imaging means is different, so that the distance to the object can be calculated based on the difference in imaging position. Since the distance to the object can be calculated only by the imaging means, it is not necessary to provide a device such as a radar that calculates the distance by emitting an electromagnetic wave or an ultrasonic wave toward the object and measuring the reflected wave.

  In the image processing apparatus according to the sixth aspect of the invention, the calculating means calculates a distance to the target object based on imaging positions of the same target object in a plurality of images taken at different times by the imaging means. It is made to do so.

  In the present invention, the distance from the vehicle to the object is calculated based on the imaging positions of the same object in a plurality of images taken at different times by the imaging means. When a plurality of images obtained by imaging at different time points are compared when the vehicle is traveling, the same object is gradually moved at the imaging position in each image. By considering the above, it is possible to calculate the distance to the object based on the difference in the imaging position, as in the case of mounting a plurality of imaging means. Since it is not necessary to mount a plurality of imaging means, the cost of the image processing apparatus can be reduced.

  Moreover, the parking lot determination method according to the seventh aspect of the present invention is to extract a straight line from an image using an image processing device that processes an image obtained by an imaging unit that captures the outside of the vehicle, detect a three-dimensional object from the image, The presence or absence of a parking lot is determined based on whether or not a detected three-dimensional object exists between two extracted straight lines.

  In the present invention, a parking lot is extracted based on whether a three-dimensional object exists between two straight lines by extracting a straight line from the captured image and detecting a three-dimensional object using an image processing device having an imaging means. The presence or absence of is determined. Parking lots are often configured with one or two rectangular parallelepiped rings between two white straight lines. Do two straight lines and solid objects between the straight lines exist in the image? Whether or not there is a parking lot can be determined by determining whether or not the parking lot is present even if the vehicle has not moved to the immediate vicinity of the parking lot.

  In the case of the first invention, two straight lines are extracted from the image picked up by the image pickup means and a three-dimensional object is detected, and whether or not there is a parking lot is determined based on whether or not a three-dimensional object exists between the two straight lines. Since it can be determined whether or not there is a parking lot provided with a three-dimensional object such as a ring stopper between two white straight lines, even after the vehicle has moved to the nearest parking lot The presence or absence of a parking lot can be determined without erroneous determination. This makes it possible to perform control such as adjusting the speed of the vehicle in the vicinity of the parking lot, so that it is possible to improve safety when the vehicle is traveling.

  Further, in the case of the second invention, the periphery of the vehicle is covered over a wide range by adopting a configuration in which the periphery of the vehicle is imaged by the imaging means in which the degree of curvature of the image captured at the end portion is larger than the center portion of the visual field. An image can be taken. In addition, by adopting a configuration in which the curved portion of the captured image is corrected and converted into an image in the bird's-eye view direction, even when using an imaging means having a large degree of curvature at the end, it is possible to reliably extract two straight lines. It can be carried out. Therefore, the presence or absence of a parking lot can be more reliably determined, and when control such as adjusting the speed of the vehicle in the vicinity of the parking lot using the determination result is performed, the safety during driving of the vehicle Can be further increased.

  In the case of the third invention, by adopting a configuration in which two white straight lines are extracted from the captured image by Hough transform, for example, even when the white straight line deteriorates and is partially damaged. Since a white straight line can be reliably extracted and erroneous extraction is unlikely to occur, the presence or absence of a parking lot can be more reliably determined. Therefore, when control such as adjusting the speed of the vehicle in the vicinity of the parking lot using the determination result is performed, it is possible to further increase the safety during traveling of the vehicle.

  Further, in the case of the fourth invention, the distance to the object captured in each pixel of the captured image is calculated, and the distance from the image is determined according to the difference in distance to the object captured in the adjacent pixel. By adopting a configuration for detecting a three-dimensional object, a three-dimensional solid object can be reliably detected from a two-dimensional image when the three-dimensional object exists on the road surface. Therefore, the presence or absence of a parking lot can be reliably determined, and when control such as adjusting the speed of the vehicle in the vicinity of the parking lot using the determination result is performed, the safety during driving of the vehicle is reduced. It becomes possible to raise.

  In the case of the fifth invention, the vehicle is equipped with a plurality of image pickup means so that the same object can be picked up from different positions, and the same object in the images picked up by the plurality of image pickup means respectively. The distance from the vehicle to the target can be accurately calculated by calculating the distance from the vehicle to the target based on the imaging position. Therefore, it is possible to reliably detect a three-dimensional object from the captured image, and to reliably determine the presence or absence of a parking lot. In addition, since it is possible to calculate the distance to the object only with the imaging means, it is not necessary to have a device such as a radar that calculates the distance by emitting electromagnetic waves or ultrasonic waves toward the object and measuring the reflected wave. An increase in the cost of the image processing apparatus due to the function of determining the presence or absence of a parking lot can be suppressed.

  In the case of the sixth invention, the vehicle is configured to calculate the distance from the vehicle to the object based on the imaging positions of the same object in the plurality of images captured at different times by the imaging unit. Since the distance from the object to the object can be accurately calculated, the three-dimensional object can be reliably detected, and the presence or absence of the parking lot can be reliably determined. In addition, since there is no need to mount a plurality of imaging means, it is possible to further suppress an increase in the cost of the image processing apparatus due to the function of determining the presence or absence of a parking lot.

  Further, in the case of the seventh invention, using an image processing apparatus having an imaging means, a straight line is extracted from the captured image and a three-dimensional object is detected, and whether or not a three-dimensional object exists between two straight lines. By determining the presence or absence of a parking lot based on this, it is possible to determine the presence or absence of a parking lot with a three-dimensional object such as a ring stop between two white straight lines, so the vehicle moves to the nearest parking lot. Even if it is not done, the presence or absence of a parking lot can be determined without erroneous determination. This makes it possible to perform control such as adjusting the speed of the vehicle in the vicinity of the parking lot using the determination result obtained by the parking lot determination method, thereby improving the safety when the vehicle is traveling. Is possible.

(Embodiment 1)
Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof. FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. In the figure, reference numeral 1 denotes an ECU (Electronic Control Unit) mounted on the vehicle, which mainly performs processing of acquiring images captured by the cameras 20R and 20L, converting them to display images, and giving them to the display 25. The cameras 20R and 20L and the ECU 1, and the ECU 1 and the display 25 are connected by a connection cable or a network cable, respectively.

  Each of the cameras 20R and 20L includes an image sensor such as a CCD or a CMOS, and converts an analog signal output from the image sensor into an image of digital data and supplies it to the ECU 1 respectively. Further, the camera 20R is configured so that the optical axis of the camera 20R and the optical axis of the camera 20L are substantially parallel, and the distance from the road surface to the camera 20R is substantially equal to the distance from the road surface to the camera 20L. It is mounted on the right front part of the vehicle, and the camera 20L is mounted on the left front part. The lenses included in the cameras 20R and 20L are lenses having a field angle of 90 ° or more, such as wide-angle lenses or fish-eye lenses, and part or all of the imaging ranges of the cameras 20R and 20L arranged at a certain distance are provided. It is made to overlap.

  The ECU 1 includes an image processing unit 10 that performs various types of image processing based on images provided from the cameras 20R and 20L. The image processing unit 10 includes an image memory 11 for temporarily storing images. Also connected is a conversion table memory 12 for storing a conversion table used for Topview conversion (described later) for converting an image captured from the front into an image in the bird's-eye view. In addition, a display 25 is connected to the image processing unit 10, and image processing is performed on images given from the cameras 20 </ b> R and 20 </ b> L so that the image can be displayed on the display 25.

  Since the cameras 20R and 20L capture an image using a lens such as a wide-angle lens or a fisheye lens, the degree of curvature of the captured image gradually increases from the center toward the end. Therefore, when displaying the image captured by the camera 20R or 20L on the display 25, the image processing unit 10 corrects and displays the curved portion of the image. Further, the image processing unit 10 can convert the images captured by the cameras 20R and 20L into an image viewed from above, that is, an image viewed from above and display the image on the display 25, for example, when parking.

  The ECU 1 also includes an electronic device and an image processing unit mounted on the vehicle based on a detection result obtained from a sensor such as a vehicle speed sensor or an acceleration sensor (not shown) mounted on the vehicle and a processing result of the image processing unit 10. A CPU 14 that performs control such as 10 is provided, and a ROM 13 and a RAM 15 are connected to the CPU 14. The ROM 13 stores various control programs to be executed by the CPU 14 and is configured by a nonvolatile memory element such as a mask ROM, an EEPROM, or a flash memory. The RAM 15 stores temporary data generated when the CPU 14 performs control processing, arithmetic processing, or the like, and includes a rewritable memory element such as SRAM, DRAM, or flash memory. The CPU 14 reads the control program from the ROM 13 and executes it to perform electrical control of the vehicle, such as ABS (Antirock Bracket System) or TCS (Traction Control System), and travels ahead. When a radar that measures the distance between the vehicle and the vehicle is mounted, it is possible to perform processing such as adjusting the vehicle speed according to the distance between the vehicles.

  Furthermore, the image processing apparatus according to the present embodiment has a function of determining whether or not there is a parking lot around the vehicle based on images obtained by the cameras 20R and 20L. With this function, for example, the position of the parking lot can be displayed on the display 25 to notify the driver, and when there is a parking lot around the vehicle, the CPU 14 can adjust the vehicle speed. Thus, it is possible to realize processing that assists the driver in driving.

  In order to determine whether or not there is a parking lot, the image processing unit 10 extracts a white straight line that may form a parking lot frame from the images given from the cameras 20R and 20L, and from the given image. A process for detecting the presence or absence of a three-dimensional object is performed, and the presence or absence of a parking lot is determined based on the results of these two processes. Further, in order to detect a three-dimensional object, a process of measuring the distance between the object captured in the given image and the vehicle (or the cameras 20R and 20L) by image processing without using a radar is performed. It is.

  The parking lot where the image processing apparatus according to this embodiment determines whether or not there is a wheel between two white straight lines drawn substantially parallel to the road surface or between two white straight line extensions. It is assumed that the parking lot has a substantially rectangular configuration with three-dimensional objects such as stops and two white lines and a three-dimensional object surrounded on three sides, but there are walls on the extension lines of the two white lines. It is possible to determine an existing parking lot or the like. Details of each process will be described below.

  2 to 4 are schematic diagrams for explaining a method of extracting a white straight line performed by the image processing apparatus according to the present invention, and (a) to (e) illustrate processes in alphabetical order in time series. It is. The case where a white straight line is extracted from an image captured by the camera 20R will be described as an example, but the same applies to the case where a white straight line is extracted from an image captured by the camera 20L.

  The camera 20R has an angle of view of 90 ° or more because it uses a lens such as a wide-angle lens or a fish-eye lens, and can capture the periphery of the vehicle over a wide range. Becomes an image whose end is greatly curved (see FIG. 2A). For example, in the image shown in FIG. 2A, the white line on the road surface captured at the end of the image, the roof of the building, and the like are curved. In such an image, even though it is originally a straight line, the image is curved at the end of the image and cannot be extracted as a straight line. Therefore, the image processing unit 10 firstly curves the image given from the camera 20R. Topview conversion is performed to correct the portion and convert it to an image in the overhead view direction.

  The Topview conversion is performed using a conversion table stored in advance in the conversion table memory 12. The image processing unit 10 moves each pixel of the image given from the camera 20R to a predetermined position, thereby converting an image picked up from the front and having a curved end portion into an image in an overhead view direction. In the conversion table, the position of the movement destination of each pixel is stored. The image processing unit 10 reads out the conversion table from the conversion table memory 12 and performs Topview conversion, whereby the image shown in FIG. 2A is viewed from above the road surface shown in FIG. 2B. That is, it can be converted into an image in the bird's-eye view direction. For example, FIG. 2B shows an image in which six white straight lines 51, 51,... And two rectangular solid objects (ring stoppers) 55, 55 arranged on the road surface are arranged on the road surface. This is an example acquired by Topview conversion.

  Next, the image processing unit 10 binarizes the image after Topview conversion. Binarization is a process of determining whether or not the luminance of each pixel of the image is greater than a predetermined threshold and converting the luminance of each pixel to the highest luminance or the lowest luminance. That is, when the luminance of the pixel is determined by 256 gradations, the luminance of this pixel is set to 255 when the luminance of the pixel to be processed is higher than the threshold, and the luminance of this pixel is set to 0 when the luminance is lower than the threshold. Thus, binarization can be performed. FIG. 3C is a binarized version of FIG. By setting the threshold appropriately, only white lines can be extracted from the image by binarization.

  After binarizing the image, the image processing unit 10 performs straight line extraction processing from the image obtained by the Hough transform. The Hough transform is a process of transforming each point (x, y) of the image from the xy plane to a Hough curve of ρ = x cos θ + ysin θ on the ρ θ plane. After performing this process for all points in the image, a straight line can be extracted by extracting points (ρ, θ) where many Hough curves intersect on the ρθ plane. FIG. 3D shows the ρθ plane in which the image of FIG. 3C is subjected to the Hough transform and six points 52, 52... Where many Hough curves intersect are extracted. Since the distance between each point extracted on the ρθ plane corresponds to the distance between straight lines on the xy plane, it is not appropriate compared to the distance between white straight lines assumed as a parking frame. Can be excluded from the target of subsequent processing.

  Next, each point extracted on the ρθ plane is converted into a straight line on the xy plane by inverse Hough transform. Inverse Hough transform is a process of converting each point on the ρθ plane into a straight line with the value of ρ at each point as the length of a perpendicular drawn from the origin of the xy plane, and the value of θ as the angle between this perpendicular and the x axis. It is. FIG. 4E shows six points 52, 52... On the .rho..theta. Plane shown in FIG. 3D converted into six straight lines 53, 53 ... on the xy plane by inverse Hough transform. Six white straight lines 51, 51... c) are extracted. In addition, although the termination | terminus part exists in six white straight lines 51, 51 ... of FIG.3 (c), the straight lines 53, 53 ... extracted using the Hough transform become a straight line without a termination | terminus part.

  Through the processes shown in FIGS. 2 to 4, a white straight line can be extracted from the image captured by the camera 20R. The image processing unit 10 performs a three-dimensional object detection process from images captured by the cameras 20R and 20L before or after or in parallel with the straight line extraction process. The three-dimensional object detection process calculates the distance to the captured object based on the two images captured by the cameras 20R and 20L, and detects the presence or absence of the three-dimensional object based on the distance calculation result. It is.

  FIG. 5 is a schematic diagram for explaining a distance calculation method performed by the image processing apparatus according to Embodiment 1 of the present invention. The image processing unit 10 determines the difference in position where the same object captured simultaneously by the two cameras 20R and 20L appears on the images obtained by the respective cameras 20R and 20L, that is, up to the object according to the parallax. The distance is calculated, and so-called stereo distance measuring distance is calculated.

The two cameras 20R and 20L are mounted on the vehicle at a predetermined distance b so that the respective imaging directions, that is, the optical axes are substantially parallel and the distances from the road surface are substantially equal. . In FIG. 5, the mounting position of the camera 20R is the origin O, the imaging direction of the camera 20R is the Z direction, the direction from the camera 20L to the camera 20R is the X direction, and the direction perpendicular to the Z direction and the X direction is the Y direction. In this case, the position of the object P to be imaged with respect to the origin O is represented by (x _P , y _P , z _P ). The cameras 20R and 20L are mounted on the vehicle so that the Z direction and the X direction are orthogonal to each other. The imaging surface of the camera 20R is illustrated as 21R, and the imaging surface 21R illustrates a two-dimensional coordinate system (x ₁ -y ₁ coordinate system) with the intersection point with the optical axis of the camera 20R as the origin. . The imaging surface 21R is located at a distance away from the camera 20R by the focal length f. Similarly, the imaging surface of the camera 20L is illustrated as 21L, and a two-dimensional coordinate system (x ₂ -y ₂ coordinate system) with the intersection point with the optical axis of the camera 20L as the origin is illustrated on the imaging surface 21L. is there. Note that the imaging surface 21L is located at a distance away from the camera 20L by the focal length f, and the focal lengths f of the cameras 20R and 20L are substantially equal. The imaging surfaces 21R and 21L are equivalent to images obtained by imaging with the cameras 20R and 20L.

When imaging is performed by the cameras 20R and 20L, and the object P is imaged on m _R (x _R , y _R ) on the imaging surface 21R and imaged on m _L (x _L , y _L ) on the imaging surface 21L (Where x _R and y _R are values in the x ₁ -y ₁ coordinate system, and x _L and y _L are values in the x ₂ -y ₂ coordinate system), the position of the object P (x _P , y _P, z _P) can be calculated from the following (1) to (3).

  However, since the solid object detection process performed by the image processing unit 10 requires only the distance of the object P in the Z direction, the calculation according to the above expression (1) is performed, and the expressions (2) and (3) Calculation by is not performed. Note that the cameras 20R and 20L perform imaging using a lens such as a wide-angle lens or a fish-eye lens, and an image obtained by imaging is an image having a large curved end as shown in FIG. Therefore, it is necessary to correct the curved portion of the image before performing the above distance calculation. Although detailed description of the process of correcting the curved portion is omitted, it can be performed by moving each pixel of the image using the conversion table stored in advance in the conversion table memory 12 as in the Topview conversion. .

  6 to 8 are schematic diagrams for explaining a method of detecting a three-dimensional object performed by the image processing apparatus according to the present invention. FIG. 6A shows an example of an image when there is no three-dimensional object on the road surface. FIG. 6B shows an example of an image in the case where the stop 55 is present as a three-dimensional object on the road surface. FIG. 7 illustrates the distance calculation result in the case of FIG. 6A, and FIG. 8 illustrates the distance calculation result in the case of FIG. 6B. In FIGS. 7 and 8, the calculated distance is schematically shown by color shading, and the dark portion indicates that the distance is close and the light portion indicates that the distance is long.

  The image processing unit 10 calculates the distance by the above equation (1) for all the objects appearing in the images acquired by the cameras 20R and 20L, that is, for all the pixels of the obtained image. Repeatedly. For example, when there is no solid object on the road surface as shown in FIG. 6A, the distance to the object shown in the image obtained as shown in FIG. 7 gradually increases from the bottom to the top of the image. The distance calculation result changes continuously. On the other hand, as shown in FIG. 6B, in the case where there is a stop 55 on the road surface, as shown in FIG. 8, the area 55a (the area surrounded by the black line in FIG. ) Exists. For this reason, the image processing unit 10 compares the distances between adjacent pixels for each pixel of the obtained image, and searches for an area on which the distance does not change or the amount of change is smaller than other parts. 3D objects can be detected. In addition, when detecting a three-dimensional object, the image processing unit 10 specifies the position of the three-dimensional object based on the distance calculation result.

  FIG. 9 is a flowchart showing a procedure of parking lot determination processing performed by the image processing apparatus according to the present invention. First, the image processing unit 10 acquires images captured by the cameras 20R and 20L (Step S1), reads the conversion table from the conversion table memory 12, and performs Topview conversion on the acquired image (Step S2). Next, the image subjected to Topview conversion is binarized (step S3), the binarized image is subjected to Hough transform (step S4), and each point of the image is converted from an xy plane to a Hough curve on the ρθ plane. By extracting points where many Hough curves obtained by transforming each point of the image are extracted (step S5), a straight line is extracted, each extracted point is subjected to inverse Hough transform (step S6), and each extracted point is converted into a straight line. Convert. The processing in steps S2 to S6 is the processing shown in FIGS. 2 to 4 described above, and by performing this processing, a white straight line drawn on the road surface can be extracted, and the position of the white straight line is acquired. can do.

  After extracting the white straight line, the image processing unit 10 calculates the distance of the object captured in the image based on the images acquired from the cameras 20R and 20L in step S1 (step S7). The calculation of the distance is a calculation process according to the equation (1), whereby the calculation result of the distance as shown in FIG. 7 or FIG. 8 can be acquired. Next, a solid object on the road surface is detected based on the distance calculation result (step S8). As described above, the detection of the three-dimensional object is performed by examining the amount of change in the distance to the object captured in the images acquired from the cameras 20R and 20L.

  Based on the extraction result of the white line performed in steps S2 to S6 and the detection result of the three-dimensional object performed in steps S7 to S8, the image processing unit 10 has a three-dimensional object between the two straight lines. It is checked whether or not to perform (step S9). When there is a solid object between two straight lines (S9: YES), the image processing unit 10 determines that a parking lot exists in the vicinity of the vehicle (step S10), and ends the parking lot determination process. If there is no solid object between the two straight lines (S9: NO), it is determined that there is no parking lot in the vicinity of the vehicle (step S11), and the parking lot determination process is terminated.

  The image processing unit 10 always repeats the processing of steps S1 to S11 while the vehicle is traveling, and the determination result of the presence or absence of the parking lot is given to the CPU 14 and used for controlling the vehicle. 10 switches the video displayed on the display 25 according to the determination result.

  In the image processing apparatus configured as described above, a white straight line is extracted from the images captured by the cameras 20R and 20L, a solid object is detected, and parking is performed depending on whether a solid object exists between the two straight lines. By adopting a configuration for determining whether or not a parking lot exists, the parking lot can be determined even after the vehicle has not moved to the immediate vicinity of the parking lot. In addition, since it is possible to easily determine the presence or absence of a parking lot using only the images captured by the cameras 20R and 20L, if the vehicle is already equipped with the cameras 20R and 20L, the parking lot can be determined without adding other hardware. This can be performed, and an increase in cost due to the provision of this function can be suppressed. In addition, it is possible to perform control such as automatically adjusting the speed of the vehicle around the parking lot using the determination result of the presence or absence of the parking lot, and by performing such control, Safety can be increased.

  In the present embodiment, the cameras 20R and 20L are cameras that capture the front of the vehicle. However, the present invention is not limited to this, and may be cameras that capture the rear or side of the vehicle. Moreover, although the cameras 20R and 20L are configured to include a wide-angle lens or a fisheye lens, the present invention is not limited to this, and a configuration including other lenses may be used. Moreover, although the description has been given by taking the ring stop as an example of the three-dimensional object existing on the road surface, the present invention is not limited to this, and other three-dimensional objects can be similarly detected. Further, the shape of the ring stopper is not limited to the illustrated one. In order to improve the detection accuracy of the wheel stopper, a shape pattern of the wheel stopper may be stored in advance, and a process of comparing with the detected shape of the three-dimensional object may be further performed.

  Moreover, in the flowchart shown in FIG. 9, after performing the straight line extraction process of step S2-S6, it was set as the structure which performs the detection process of the solid object of step S7-S8, but it is not restricted to this, Three-dimensional object A configuration in which a straight line extraction process is performed after the above detection processing is performed may be employed, or two processes may be performed simultaneously. In addition, the image processing unit 10 provides the CPU 14 with the determination result of the presence / absence of the parking lot and uses it for vehicle control, and the image processing unit 10 switches the video displayed on the display 25 according to the determination result. However, it is not limited to what kind of processing the determination result is used.

(Embodiment 2)
FIG. 10 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 2 of the present invention. Although the image processing apparatus according to the first embodiment includes two cameras 20R and 20L, the image processing apparatus according to the second embodiment includes only one camera 120. Therefore, the image processing unit 10 of the image processing apparatus according to the second embodiment uses a method for calculating the distance to the object captured in the image acquired from the camera 120 as in the case of the first embodiment shown in FIG. Although different, other processes are the same as those in the first embodiment.

FIG. 11 is a schematic diagram for explaining a distance calculation method performed by the image processing apparatus according to the second embodiment of the present invention. The image processing apparatus according to the second embodiment has two images: an image captured by the camera 120 at time t ₁ and an image captured by the camera 120 at time t ₂ after a predetermined time has elapsed from time t _1. The distance is calculated based on the above. If the vehicle during the period from the time t ₁ the time t ₂ is moved at a velocity v, the imaging position L by the camera 120 can be calculated by the following equation (4).
L = v * (t ₂ −t ₁ ) (4)

Since the image picked up at time t ₁ and the image picked up at time t ₂ can be handled as images picked up by two cameras in a pseudo manner, the distance b between the two cameras in equation (1) is expressed as (4 The distance to the object P can be calculated by the equations (1) and (4) by approximating b = L with the movement distance L of the equation (1).

  Compared with the image processing apparatus according to the first embodiment, the image processing apparatus according to the second embodiment having the above-described configuration has an advantage that the number of cameras mounted on the vehicle may be small, so that the cost can be reduced.

  Since the other configuration of the image processing apparatus according to the second embodiment is the same as that of the image processing apparatus according to the first embodiment, the corresponding portions are denoted by the same reference numerals and detailed description thereof is omitted. To do.

1 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. It is a schematic diagram for demonstrating the method of white line extraction which the image processing apparatus which concerns on this invention performs. It is a schematic diagram for demonstrating the method of white line extraction which the image processing apparatus which concerns on this invention performs. It is a schematic diagram for demonstrating the method of white line extraction which the image processing apparatus which concerns on this invention performs. It is a schematic diagram for demonstrating the method of distance calculation which the image processing apparatus which concerns on Embodiment 1 of this invention performs. It is a schematic diagram for demonstrating the method of the solid-object detection which the image processing apparatus which concerns on this invention performs. It is a schematic diagram for demonstrating the method of the solid-object detection which the image processing apparatus which concerns on this invention performs. It is a schematic diagram for demonstrating the method of the solid-object detection which the image processing apparatus which concerns on this invention performs. It is a flowchart which shows the procedure of the parking lot determination process which the image processing apparatus which concerns on this invention performs. It is a block diagram which shows the structure of the image processing apparatus which concerns on Embodiment 2 of this invention. It is a schematic diagram for demonstrating the method of distance calculation which the image processing apparatus which concerns on Embodiment 2 of this invention performs.

Explanation of symbols

1 ECU
10 Image processing unit (extraction means, detection means, determination means, conversion means, calculation means)
11 Image memory 12 Conversion table memory 13 ROM
14 CPU
15 RAM
20R, 20L camera (imaging means)
25 Display 120 Camera (Imaging means)

Claims (7)

In an image processing apparatus that processes an image obtained by an imaging unit that images the outside of a vehicle,
Extraction means for extracting a straight line from the image;
Detecting means for detecting a three-dimensional object from an image;
And a determination unit that determines whether or not there is a parking lot based on whether or not the three-dimensional object detected by the detection unit exists between the two straight lines extracted by the extraction unit. apparatus. The imaging means has a greater degree of curvature of the image captured at the end than at the center of the field of view,
A conversion unit that corrects a curved portion of the image captured by the imaging unit and converts the image to an image in the overhead direction,
The image processing apparatus according to claim 1, wherein the extraction unit extracts two straight lines from the image converted by the conversion unit.   The image processing apparatus according to claim 1, wherein the extraction unit extracts a white straight line from the image by Hough transform. For each pixel of the image obtained by the imaging means, a calculation means for calculating the distance to the object copied to each pixel,
The detection unit according to any one of claims 1 to 3, wherein the detection unit detects a three-dimensional object from an image in accordance with the difference in the distance calculated by the calculation unit for adjacent pixels. Image processing device. The imaging means is a plurality, and the plurality of imaging means are capable of imaging the same object from different positions,
The image processing apparatus according to claim 4, wherein the calculating unit calculates a distance to the object based on an imaging position of the same object in an image captured by each of the plurality of imaging units. .   5. The image according to claim 4, wherein the calculation unit calculates a distance to the object based on imaging positions of the same object in a plurality of images captured at different times by the imaging unit. Processing equipment. Using an image processing device that processes an image obtained by an imaging means that images the outside of the vehicle,
Extract a straight line from the image,
Detect solid objects from images,
A parking lot determination method, wherein the presence or absence of a parking lot is determined based on whether or not a detected three-dimensional object exists between two extracted straight lines.

Images