WO2013062401A1

WO2013062401A1 - A machine vision based obstacle detection system and a method thereof

Info

Publication number: WO2013062401A1
Application number: PCT/MY2012/000147
Authority: WO
Inventors: Yahya Ratnam DAWSON
Original assignee: Dawson Yahya Ratnam
Priority date: 2011-10-24
Filing date: 2012-06-26
Publication date: 2013-05-02

Abstract

A machine vision based obstacle detection system (100) on a planar surface and a method thereof is provided, characterized in that, the system (100) includes a reflecting means (103) positionable such that an obstacle is distortably reflectable on said reflecting means (103), at least one image capture device (105) positionable such that the image from the reflecting means (103) is within field of view (FOV) of the image capture device (105), wherein a lens system (107) and an optical filter (109) is fixable to the image capture device (105), a processing unit (111) connectable to the image capture device (105).

Description

A MACHINE VISION BASED OBSTACLE DETECTION SYSTEM AND A METHOD

THEREOF

FIELD OF INVENTION

The present invention relates to a machine vision based obstacle detection system and a method thereof.

BACKGROUND OF INVENTION

Vision systems are known to use optical apparatus. These apparatus have objectives such as to increase the field of view of the vision system. Spherical mirrors and wide angle lenses are used to obtain a panoramic view to enable obstacle detection/avoidance. However, these systems have very limited depth range such as that described by WO 2011/087354 A2 though the provided solution is by means of an imaging device capturing video signals of an area surrounding a vehicle.

Image analysis software is generally used in robotics and more recently by UAV's (unmanned autonomous vehicles). Many vision systems use image recognition for obstacle detection and therefore still rely on RADAR or LIDAR to detect obstacles. However, this methods of detection may not be as accurate as required.

US 20110050883 A1 describes a machine vision based obstacle avoidance system. However, in this document, normalized image and dynamic masking is used in object detection. The solution requires using histograms for formed variation of intensities. This could overly complicate calculations. Therefore, there is a need for a solution that detects obstacles without using image recognition that is not compromised when depth range is not adhered to.

SUMMARY OF INVENTION

Accordingly, there is provided a machine vision based obstacle detection system on a planar surface, characterized in that, the system includes a reflecting means positionable such that an obstacle is distortably reflectable on said reflecting means, at least one image capture device positionable such that the image from the reflecting means is within field of view (FOV) of the image capture device, wherein a lens system and an optical filter is fixable to the image capture device, a processing unit connectable to the image capture device.

There is also provided a method of detecting obstacles on a planar surface, characterized in that, the method includes the steps of conditioning an image of an object on the planar surface, capturing the image from the conditioned image within field of view, processing the captured image wherein the image is cropped, edges are detected, pixels are expanded, blobs are filtered and data is extracted and producing an output from the processed image.

The present invention consists of several novel features and a combination of parts hereinafter fully described and illustrated in the accompanying description and drawings, it being understood that various changes in the details may be made without departing from the scope of the invention or sacrificing any of the advantages of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be fully understood from the detailed description given herein below and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present invention, wherein:

Figure 1 shows a block diagram of a preferred embodiment of a machine vision based obstacle detection system;

Figure 2 shows a front view of mirror reflection exhibiting cylindrical distortion in the preferred embodiment of a machine vision based obstacle detection system; Figure 3 shows a graphical representation of a captured image after analysis in the preferred embodiment of a method of detecting obstacles;

Figure 4 shows a graphical representation of a captured image when an object is too close to mirror;

Figure 5 shows a side view of an illuminator in the preferred embodiment of the machine vision based obstacle detection system;

Figure 6 shows a graphical representation of captured images of separate apparatus views arranged to form a single image for analysis in the preferred embodiment of the invention;

Figure 7 shows a side view of possible positioning of the system in applications;

Figure 8 shows a second embodiment of a machine vision based obstacle detection system; Figure 9 shows a flowchart showing a method of detecting obstacles in the preferred embodiment of the invention; Figure 10 shows a flowchart of the image processing and analysis of said processed image in the preferred embodiment of the method; and

Figure 11 shows a block diagram of the preferred embodiment of the system when there is no planar surface.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The present invention relates to a machine vision based obstacle detection system and a method thereof. Hereinafter, this specification will describe the present invention according to the preferred embodiment of the present invention. However, it is to be understood that limiting the description to the preferred embodiment of the invention is merely to facilitate discussion of the present invention and it is envisioned that those skilled in the art may devise various modifications and equivalents without departing from the scope of the appended claims.

The following detailed description of the preferred embodiment will now be described in accordance with the attached drawings, either individually or in combination. Figure 1 is a block diagram that illustrates the preferred embodiment of a machine vision based obstacle detection system (100) on a planar surface. The system (100) includes a reflecting means (103) positionable such that an obstacle is distortably reflectable on said reflecting means (103), an image capture device (105) positionable such that the image from the reflecting means (103) is within field of view (FOV) of the image capture device (105), wherein a lens system (107) and an optical filter (109) is fixable to the image capture device (105), a processing unit (111) connectable to the image capture device (105).

The reflecting means (103) includes a mirror reflector which reflects a cylindrically distorted image of an object before the mirror reflector. It is to be understood that a curve profile of the mirror reflector can be wholly or partially circular or hyperbolic, but not restricted to the above mentioned curve profile. The curve profile is positionable parallel to the planar surface. However, it is to be appreciated that the curve profile may also be positionable at an angle to the planar surface depending on a required application. The image capture device (105) is selected from, but not restricted to, charge coupled device (CCD) or Complementary metal- oxide-semiconductor (CMOS) camera. Depending on the application wherein if continuous detection is required, a camera optimized for motion picture acquisition is required. If single instance detection is required a camera optimized for still picture acquisition may be utilized. Existing digital cameras have dual capability in this regard.

In a second embodiment of the system (100) as seen in Figure 8, a custom lens such as a cylindrical lens (203) attached in front of the image capture device (205) to achieve this same distortion effect is used. In this situation a camera points directly to area of interest without need of a mirror.

As seen in Figure 2, a reflection of objects in a scene of interest appears on a mirror curve apex (which is an area of interest) such that a vertical component of the reflection (perpendicular to the plane surface) seems stretched and a horizontal component (parallel to the plane surface) is compressed.

A lens system (107) includes a convex lens in front of the camera as seen in Figure 1 and 8. An optical filter (109) further includes a plurality of optical components such as neutral density filters, polarizing filters and band pass filters. The image capture device (105) such as the CMOS or CCD camera is positionable towards the mirror such that the reflection of the area of interest is within field of view (FOV).

The processing unit (111 ) performs a function of extracting live video onto an image frame sequence if required, implementing an obstacle detection process, extracting data from a processed image and produces a corresponding output or passes extracted data to a control application. The processing unit (111) can be implemented either via a computer system or via custom electronics. A further improvement to the problem of an obstacle being too close to apparatus is by using an illuminator as seen in Figure 5. An illuminator is used to project a pattern onto the obstacle surface. The difference in light intensity between lighter and darker areas in relation to each other which form the pattern projected onto the obstacle surface will enable the edge detection process to correctly identify the presence of an obstacle.

In yet another embodiment, a laser display system is used to create the above mentioned pattern as the laser display system has the advantage of always being able to form a focused image onto the obstacle surface regardless of obstacle distance. Another option is to use a collimated light source behind a mask such that the mask forms a silhouette of the required pattern.

Depending on the application, the illuminator may be made to operate in the visible or invisible spectrum. The above explanation relates to a single camera used in the system (100) wherein camera sensitivity is made to overlap with illuminator spectrum.

It is to be noted that multiple cameras are positionable to monitor different areas of interest as only a small portion of the total field of view of the camera is actually utilized, images from multiple apparatus can be arranged to form an image collage. This collage of images will form a single image that can be processed by a single instance of the analysis application enabling simultaneous processing of multiple apparatus views. This is seen in Figure 6.

In another example of the system (100), it is possible to use two cameras in the same system (100). The image is splittable by an image splitter such as a pentaprism such that two separate cameras see the same image. These cameras may be sensitive to specific light bandwidths enabling images from both cameras to be combined via a mathematical process such as , but not restricted to, image addition or image subtraction before further processing. Further, the system (100) is mountable onto a rotating platform to enable a wide view or panoramic view of the area to be analyzed. Alternatively only the camera and its attachments (105, 107 and 109) may be made to rotate around a fixed mirror system (103) to achieve the same effect.

A method of detecting obstacles on a planar surface is described herein as seen in Figure 9. The method includes the steps of conditioning an image of an object on the planar surface, capturing the image from the conditioned image within field of view, processing the captured image wherein the image is cropped, edges are detected, pixels are expanded, blobs are filtered and data is extracted and producing an output from the processed image.

Optical conditioning is done by reflecting a cylindrically distorted image wherein the reflection is parallel to the planar surface or at an angle depending on the intended application. This reflection is as seen on Figure 2 wherein the reflection on the mirror curve apex appears distorted such that its vertical component (perpendicular to the plane surface) seems stretched and its horizontal component (parallel to the plane surface) is compressed. There is also provided necessary optical magnification depending on the required range or application. The live video is captured via a CCD or CMOS video camera wherein each image frame is processed. Depending on the application, the device sensitivity may be in the visible or invisible spectrum. Image processing and analysis is performed as seen in Figure 10. The captured image is cropped such that only the required area is processed. In some applications, color manipulation such as grayscale conversion may be applied before the edge detection step so as to improve detection accuracy. Edge detection is then performed wherein there are several outcomes. Objects with less or no vertical component are less pronounced and thus have a relatively smaller edge area and can be filtered out in blob filter stage, objects with sufficient vertical component are more pronounced and thus have a bigger edge area, objects with less or no vertical component have edge areas that increase substantially as the object gets closer to the mirror however it only occupies the lower part of the captured image or objects with sufficient vertical component (depending on apparatus elevation) occupy an area where a resultant blob centre of gravity (COG) remains within a predetermined "y" range. This is depicted in Figure 3. Figure 4 shows an instance where the object is too close to the mirror and that no blob can be formed.

It is to be appreciated that it is possible to obtain the required distorted image artificially by applying an image transform process to the captured image. This way neither the mirror nor the cylindrical lens would be required.

Pixel expansion expands the edge detected areas such that the objects left and right vertical edges join to form a single blob. Once blobs are formed, the blob's characteristics can be extracted for the next process. The processes used for this stage are dilate, fill and population threshold.

Referring to Figure 3, a plurality of blobs are formed from the above process. The blobs are filtered based on blob size, vertical orientation, Centre of gravity (COG) being within an acceptable region (based on minimum detectable height), horizontal centre, as well as an additional step of clustering the filtered blobs to form larger ones (blobs).

Following this, data extraction is performed using a largest blob to extract possible object distance. The "y" coordinates of the isolated blobs lowest point is extracted and a suitable conversion factor is applied in order to correlate pixel coordinate to actual distance. Edge probe detection is useful in detecting obstacles at further distances from the apparatus. Thus it may be necessary to use a combination of lowest blob point and edge probe detection based on object distance to increase the system accuracy. Distance data (over time) could also be used to derive the rate of approach of an object. The extracted data is then processed to produce an output or passed to another application (control application) for further processing or to produce an output. Apart from distance data and rate of approach data, other blob related characteristics such as, but not restricted to, blob area and blob COG is extracted. Other image processing steps which may be included such as image stabilization, noise reduction, and flicker reduction are not discussed as these are obvious to those familiar with the art. The processed image may be displayed via an appropriate display device if required. An isolated blob indicating the detected obstacle may be superimposed onto the pre-processed video for display purposes.

In another example of the method, detection of obstacle is done using a line laser when there is no planar surface as seen in Figure 11. In this example, the detection method is slightly varied. The following processes are used wherein the steps include detecting a laser line, performing a side fill from base of field of view to laser line and erosion. This will leave an area above which an obstacle was detected. Further steps of the method are then performed as discussed earlier.

In order to calculate distance of the object, angle of the laser output is required. If the laser line is detected, known angle 'x' corresponds to a known height 'h'. Thus the object distance d can be derived from: d = h / tan x

This invention is adapted for use as part of a camera based vehicle collision warning system as seen in Figure 7. The system (100) detects any sufficiently large obstacle (such as another vehicle) and warns the driver (via audio and visual alerts) if such obstacle was below a maximum distance threshold. The system (100) would also warn the driver if the rate of approach of such obstacle was above the minimum threshold indicating possible impact. The disclosed invention is suitable, but not restricted to, for use in surveillance, navigation, surface defect detection and proximity detection.

Claims

1. A machine vision based obstacle detection system (100) on a planar surface, characterized in that, the system (100) includes:

a reflecting means (103) positionable such that an obstacle is distortably reflectable on said reflecting means ( 03);

at least one image capture device (105) positionable such that the image from the reflecting means (103) is within field of view (FOV) of the image capture device (105); wherein a lens system (107) and an optical filter (109) is fixable to the image capture device (105);

a processing unit (111) connectable to the image capture device (105).

2. The system (100) as claimed in claim 1 , wherein the reflecting means is a mirror.

3. The system (100) as claimed in claim 1 , wherein the reflecting means is a cylindrical lens.

4. The system (100) as claimed in claim 1 , wherein the image capture device (105) is a camera.

5. The system (100) as claimed in claim 1 , wherein an illuminator is used to project a pattern onto an obstacle surface.

6. The system (100) as claimed in claim 1 , wherein a laser display system is able to form a focused image onto the obstacle surface regardless of obstacle distance.

7. The system (100) as claimed in claim 1 , wherein multiple cameras are positionable in the system (100) to monitor different areas of interest.

8. The system (100) as claimed in claim 1 , wherein two cameras are used in the same system (100) wherein the image is splittable by an image splitter.

9. The system (100) as claimed in claim 1 , wherein the system (100) is mountable onto a rotating platform.

10. A method of detecting obstacles on a planar surface, characterized in that, the method includes the steps of:

i. conditioning an image of an object on the planar surface; ii. capturing the image from the conditioned image within field of view; iii. processing the captured image wherein the image is cropped, edges are detected, pixels are expanded, blobs are filtered and data is extracted; and iv. producing an output from the processed image.

11. The method as claimed in claim 10, wherein the step of conditioning the image includes reflecting a cylindrically distorted image wherein the reflection is parallel to the planar surface.

12. The method as claimed in claim 10, wherein the step of conditioning the image includes reflecting a cylindrically distorted image wherein the reflection is positionable at an angle to the planar surface.

13. The method as claimed in claim 11 , wherein the planar surface is a virtual planar surface.

14. The method as claimed in claim 10, wherein the method includes obtaining a required distorted image artificially by applying an image transform process to the captured image.

15. The method as claimed in claim 10, wherein the method further includes detecting a laser line, performing a side fill from base of field of view to laser line and erosion, when there is no planar surface.