Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/23—Clustering techniques
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle

Definitions

• the invention relates to a method for creating at least one cluster of image regions of an image taken from a surrounding area of a motor vehicle, which cluster is usable for detecting at least one target object in said surrounding area.
• the method comprises the following steps: (i) identifying a given number of characteristic features in the image; (ii) dividing the image into a plurality of image regions; (iii) determining at least one statistical parameter of the characteristic features to each image region; and (iv) clustering adjacent image regions with their characteristic features having statistical parameters within a common parameter area.
• the invention further relates to a corresponding use of the method for detecting target objects, computer program product, a corresponding camera system and motor vehicle with such kind of camera system.
• Camera systems for motor vehicles are already known from the prior art.
• a camera system includes at least one camera, which is attached to the motor vehicle and captures a surrounding area of the motor vehicle.
• Several such cameras can also be employed, which capture the entire surrounding of the motor vehicle.
• the camera mounted on the motor vehicle provides a temporal sequence of images of the surrounding area, namely a plurality of images per second.
• This image sequence is then communicated to an electronic evaluation device, which processes the captured images and is able to provide very different functionalities in the motor vehicle based on the images.
• the interest applies to the detection of target objects located in the depicted surrounding area of the vehicle. If a target object is detected in the images, thus, this target object can be tracked in the sequence of images.
• the optical flow method is usually used, in which characteristic features such as for example edges and/or corners are detected in the images and a flow vector is calculated to each characteristic feature, which specifies the direction of movement and the speed of movement of the characteristic feature in the sequence of images.
• Document WO 2015/014882 A1 describes a method for detecting at least one target object in a surrounding area of a motor vehicle based on an image of the surrounding area, wherein a given number of characteristic features are identified in the image, at least one part of the image is divided into a plurality of image regions connected to image nodes, and a local feature density of the characteristic features is determined to each image node or image region respectively.
• the detection of the target object includes that several of the image nodes/image regions are combined to a cluster representing the target object depending on the respective feature density.
• the statistical parameter of the characteristic features used in this method is a feature density of the characteristic features in a corresponding image region.
• the recognition of characteristic features out of an image taken by a camera can be very CPU intensive even if no characteristic features at all are present in some regions of the image.
• the parameter of the characteristic features or at least one of the parameters of the characteristic features is a feature saliency.
• Feature saliency of characteristic features in an image is a measure of feature relevance. This measure is known to the skilled person from different kind of analysis of image features for identification of important image regions of an image (keyword: saliency map).
• the saliency of a characteristic feature (or of a simple pixel) is based on its contrast with its surroundings.
• the method makes sure that no characteristic features are missed on key objects due to a low statistical parameter with respect to high texture regions (i.e. regions with high local feature density of the characteristic features) in the image.
• parameter score e.g. feature saliency score
• the image regions can be tiles or other rectangular image cells.
• another parameter of the characteristic features in an image region is a feature density of the characteristic features in the respective image region.
• the method comprises the steps of (a) determining both, the feature saliency and the feature density of the characteristic features to each image region of the image and (b) clustering adjacent image regions with their characteristic features having a common parameter range with respect to feature saliency score and/or feature density of the characteristic features.
• the image is divided into the plurality of image regions by means of an iterative dividing scheme allowing division of the image into of image regions of different size.
• the iterative dividing scheme uses intermediate image regions to end up with the final image regions the image is divided into. These final regions do not have uniform size. Because at least some of the final regions are relatively large, the method using this iterative scheme does not need a time-consuming and/or expensive merging procedure at the end.
• At least one of the parameters of the features in the respective region, especially the feature saliency, is determined to each region in each iterative step, wherein in each iterative step the size of possible sub-regions of the region generated by division in the previous iterative step is based on the parameter assigned to the corresponding region generated in the previous iterative step.
• the (final) image regions with highest feature density and the image regions with lowest feature density are combined to clusters separately.
• the clustering process ends up with a number of clusters C x of different cluster-types X. These clusters C are formed by thresholding against features density and feature sali- ency of the characteristic features. The number of characteristic features assign to each of the clusters C is N fea tures-
• the invention further relates to the use of the aforementioned creating method for detecting at least one target object in said surrounding area of the motor vehicle.
• the invention further relates to an inventive method using the
• the computer program product comprises computer- executable program code portions having program code instructions configured to execute the aforementioned method.
• the camera system for a motor vehicle includes a camera for providing an image of a surrounding area of the motor vehicle and includes an evaluation device adapted to perform the aforementioned method.
• the invention relates to a motor vehicle with the aforementioned camera system.
• Fig. 1 shows a motor vehicle including a camera system according to a
• Fig. 2 shows a first step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 3 shows a second step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 4 shows a third step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 5 shows a forth step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 6 shows a fifth step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 7 shows a sixth step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 8 shows a final step of an iterative dividing scheme for dividing an image of a surrounding area of the vehicle into a plurality of image regions
• Fig. 9 shows a clustering step for clustering the final image regions shown in
• Fig. 10 shows a flow chart of a corresponding method for detecting target objects according to a preferred embodiment of the invention.
• a motor vehicle 10 according to an embodiment is shown in schematic illustration.
• the motor vehicle 10 is for example a passenger car. It includes a camera system 12 having a camera 14.
• the motor vehicle 10 is on a road 16.
• the camera system 12 is for example a collision warning system and serves for warning the driver of the presence of the target object 18 in a surrounding area 20 of the motor vehicle 10.
• the camera system 12 thus serves for detecting the target object 18 and preferably also for tracking the target object 18 in the surrounding area 20.
• images A are taken by means of the camera 14, which are then processed by means of an electronic evaluation device (signal processor) not illustrated in more detail.
• the evaluation device receives the images from the camera 14 and processes them.
• the evaluation device can be integrated in the camera 14 or it can be a component of the camera system 12 separate from the camera 14.
• the images can also be displayed on a display in the motor vehicle 10 (not shown), wherein the detection of the target object 18 can for example be effected to the effect that the target object 18 is provided with a border in the images.
• the camera 14 can be a CMOS camera or a CCD camera or any image capturing device, which is formed for detecting light in the visible spectral range.
• the camera 14 is preferably a video camera continuously providing a sequence of images.
• the electronic evaluation device then processes the image sequence in real time and can detect and track the target object 18 based on this image sequence.
• the method for creating clusters C comprises three main steps:
• characteristic features to each image region, namely (a) the feature saliency (Feature V ariance) of the characteristic features in a respective image region and (b) the feature density (Feature de nsit y ) of the characteristic features in a respective image region (shown in Figs. 2 to 8); and
• Feature detection refers to methods that aim at computing abstractions of image information and making local decisions at every image point whether there is an image feature of a given type at that point or not.
• Figs. 2 to 8 show seven steps of an iterative dividing scheme for dividing the image A into a plurality of final image regions T1 A, T1 B, T1 C, T2A, T3 as shown in Fig. 8.
• the image or ROI of the image is iteratively subdivided into a number of tiles or other kind of image regions based on the statistical parameter (a) feature saliencies score (feature variance score) and (b) feature density of the characteristic features within each tile.
• the tiles size is not fixed.
• a scene can, e.g., have blocks with multiple sized rectangular tiles. Only final minimum block size width and height is configurable, i.e. sub divided block width and height should be more than configured minimum block width and height.
• the feature saliencies score is a measure for a parameter area of similar feature saliencies.
• Fig. 2 shows a first step of the iterative dividing scheme for dividing the image A into a plurality of image regions.
• these image regions are called "tiles" T.
• the image A is divided into three (intermediate) tiles T1 , T2, T3 and local statistical analysis of feature density and feature saliency (spatial feature variance) is performed in each tile.
• Tiles having higher feature densities and higher feature saliencies variance (T2 in the shown example) are further subdivided equally in to smaller tiles.
• This step is required to further differentiate the regions into smaller regions based on feature densities and feature saliencies. So that smaller regions will represents higher feature densities. And the regions which do not have higher features densities will clustered together and adapt saliency threshold as per tile sizes. The smaller tile size will have higher saliency threshold due to high feature density.
• Fig. 4 shows a sub-divisions of tile T2A as per feature density and higher feature saliencies variance. Further smaller tiles are analysed to extracts the features densities and feature saliencies. Again these smaller tiles are subdivided based on their local statistical analysis of feature densities and feature saliencies.
• Figs. 5 to 7 show a further division of tiles as per feature densities (Feature de nsit y ) and higher feature saliency
• Fig. 9 shows the result of the clustering of adjacent image regions with their
• characteristic features having statistical parameters within a common parameter area.
• the tiles with highest densities and lowest density are clustered separately.
• Clusters C M with mid(-sized) feature saliency
• the image A is finally divided into ten clusters C with three different feature saliencies scores (low, mid, high), namely four clusters C L with a low saliency score, three clusters C M with a mid-sized saliency score and three clusters C H with a high saliency score.
• This whole process is fully adaptive and automatic in nature and will insure the distributions of the features extracted/detected using various computer vision based feature detector approach will be uniform in nature.
• the proposed method will make sure that features are not missed on key objects due to low saliency score with respect to high texture region in the scene.
• Fig. 10 shows a corresponding flow chart of the method for efficient feature distributions based on feature clustering by analysing global and local statistical properties of scene.
• the method comprises the following step:
• Tiles having higher feature densities and higher feature saliencies variance further subdivided equally in to smaller tiles. This step is required to further differentiate the regions into smaller regions based on feature densities and feature saliencies. So that smaller regions will represents higher feature densities. And the regions which do not have higher features densities will clustered together and adapt saliency threshold as per tile sizes. The smaller tile size will have higher saliency threshold due to high feature density.
• Step 5 This process of subdividing smaller tiles will continue until the process reached to the minimum configurable tiles size, which is requested in Request R ("all Tiles processed"). In case the process did not reach to the minimum configurable tiles size (path N), steps 3 and 4 will be performed again on the next iterative (sub-)level. Otherwise (path Y) the method will continue with Step 5.
• the steps S3, S4 and R are forming a loop of an iterative dividing scheme.
• the smallest tile will represent an image region with high feature density and variance. This till will be considered as a starting point of a cluster which has high saliency score.
• another tile with similar feature density and variance to the cluster C H 1 has to be found.
• This tile is a starting point for another cluster C H 2 with high saliency score.
• neighbour tiles with similar high feature density and variance will be merged to create a cluster until all neighbour- tiles are merged. This process has to be repeated to create all high saliency score clusters C H 1"3 .
• Remaining unlabeled tiles belongs to mid saliency clusters C M .
• Find first unlabeled tile This tile will be considered as start of cluster C M 1 with mid-sized saliency. Start merging neighbour tiles which are unlabeled to create cluster C M 1 until all neighbour-tiles are merged. In the following another tile with similar feature density and variance to the cluster C M 1 has to be found. This tile is a starting point for another cluster C M 2 with midsized saliency score. Again, neighbour tiles with similar sized feature density and variance will be merged to create a cluster until all neighbour-tiles are merged. This process has to be repeated to create all mid-sized saliency score clusters C M 1"3 .
• These clusters are formed by thresholding against features density and features variance.
• the number of characteristic features assign to each cluster C is N fea tures-
• the number n of different clusters for the "low" type L is 4.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Data Mining & Analysis (AREA)
• Life Sciences & Earth Sciences (AREA)
• Artificial Intelligence (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Bioinformatics & Computational Biology (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Evolutionary Biology (AREA)
• Evolutionary Computation (AREA)
• General Engineering & Computer Science (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=60484383

Family Applications (1)

Country Status (3)

Families Citing this family (2)

Family Cites Families (2)

• 2016
  • 2016-12-13 DE DE102016124123.7A patent/DE102016124123A1/de active Pending
• 2017
  • 2017-11-29 WO PCT/EP2017/080745 patent/WO2018108524A1/en not_active Ceased
  • 2017-11-29 EP EP17805197.5A patent/EP3555801A1/de not_active Withdrawn

Also Published As

Similar Documents

Legal Events