WO2018234109A1 - CLASSIFICATION OF STATIC AND DYNAMIC IMAGE SEGMENTS IN A DEVICE FOR DRIVING A MOTOR VEHICLE - Google Patents

CLASSIFICATION OF STATIC AND DYNAMIC IMAGE SEGMENTS IN A DEVICE FOR DRIVING A MOTOR VEHICLE Download PDF

Info

Publication number
WO2018234109A1
WO2018234109A1 PCT/EP2018/065584 EP2018065584W WO2018234109A1 WO 2018234109 A1 WO2018234109 A1 WO 2018234109A1 EP 2018065584 W EP2018065584 W EP 2018065584W WO 2018234109 A1 WO2018234109 A1 WO 2018234109A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
segment
motion vector
segments
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2018/065584
Other languages
French (fr)
Inventor
Swaroop Kaggere Shivamurthy
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Connaught Electronics Ltd
Original Assignee
Connaught Electronics Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Connaught Electronics Ltd filed Critical Connaught Electronics Ltd
Publication of WO2018234109A1 publication Critical patent/WO2018234109A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/215Motion-based segmentation
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior
    • G06T2207/30252Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261Obstacle

Definitions

  • the invention relates to a method for operating a computing device of a driver assistance device of a motor vehicle, which serves for classifying an image segment as a static or dynamic image segment.
  • the invention relates to a driver assistance device for a motor vehicle including a computing device, which is formed to segment a provided frame into a plurality of image segments depending on at least one image characteristic and to respectively detect at least one image feature in the image segments of the segmented frame as well as to ascertain a respective formation vector for the detected image features.
  • an object can be partially covered such that only a part of a static object is detected.
  • a dynamic object or a dynamic image feature can erroneously be recognized.
  • one and the same object can be recognized partially as a static object and partially as a dynamic, moving object by for example differently classifying different image features, which are associated with the same object.
  • the focus of expansion in a frame is that point, in which the motion vectors of all of the features converge or from which they all point away.
  • the length of a motion vector of a corresponding image feature will basically tend to zero if the corresponding image feature is closer to the focus of expansion. Since the motion vectors close to the point of expansion, which can also be referred to as vanishing point, have a low length, thus, a small error there already results in great consequential errors in a corresponding three-dimensional reconstruction. Therefore, some detection algorithms for objects have an area around the focus of expansion, within which motion vectors are filtered out. Here, omitted detection of the objects near the focus of expansion therefore occurs.
  • static objects are recognized as dynamic objects and vice versa.
  • a dynamic object like a vehicle moving parallel to the own motor vehicle is static compared to the own motor vehicle.
  • the optical flow or image flow that is the motion vectors used for determining the image flow, is scale-variant in the usual approaches.
  • image features which are detected on a corresponding object and associated with it, are not necessarily well traceable image features, which are suitable for calculating the image flow and/or determining image segments or objects as static or dynamic. This is the principal reason that object recognition algorithms often fail in recognizing static objects in large distance as such.
  • an image feature is not recognized for some objects, that is an image feature is not associated with an object, can be solved by a so-called dense optical flow, that is an increase of the number of the image features with corresponding motion vectors (or trajectories), as it is possible in future driver assistance devices as a result of hardware.
  • dense optical flow is as scale-variant as a less dense optical flow.
  • better and worse traceable image features are difficult to differentiate.
  • an optical flow inconsistent and inhomogeneous to greater extent is even to be expected. Namely, due to the finer sampling of respective random textures such as for example road or vegetation in the environment of the motor vehicle, image features are also erroneously recognized in the random textures. Thereby, the number of the false positives mentioned above under secondly will increase such that a more expensive classification of the image features is again required.
  • the object to improve the object recognition of a driver assistance device of a motor vehicle, in particular improve classifying image features and/or image segments.
  • the invention relates to a method for operating a computing device of a driver assistance device of a motor vehicle including a series of methods steps.
  • one method step can be capturing an environment of the motor vehicle by a camera device of the driver assistance device with providing a captured frame on the part of the camera device.
  • a method step is segmenting a provided frame, for example the captured camera frame of the camera device, into a plurality of image segments depending on at least one image characteristic by the computing device.
  • the frame can for example also be a sensor card of a sensor device of the driver assistance device.
  • Segmenting or clustering the frame into image segments or image clusters can for example be effected based on a respective color as the image characteristic such that for example image areas with the same color or preferably contiguous image areas with the same color are treated as one image segment.
  • each segmentation algorithm can basically be used.
  • one possibility is dividing the frame into respective sectors, which is advantageous upon lighting changes.
  • another possibility is for example a semantic classification in image segmentation, which can for example be performed by dividing an image into image blocks and for example dividing the image block with the respectively most complicated texture into further image blocks.
  • a next method step is detecting at least on image feature in the respective image segments, preferably all of the image segments, of the segmented frame by the computing device.
  • a number N of image features can for example be divided among the image segments according to a preset algorithm, that is at least one image feature can be associated or a corresponding processing place in the computing device can be associated with each image segment and then the corresponding image feature or features can be recognized or detected in the respective image segment.
  • a further method step is ascertaining (in the sense of determining or evaluating) a respective motion vector for the detected image features by the computing device. This can be effected based on an image feature associated with the respective image feature in the or a (preceding) further provided and segmented frame.
  • a motion vector can include a length and an angle, which correspondingly describes a flow magnitude or flow speed of the image feature as well as a flow orientation of the image feature, that is a size and/or orientation of the optical flow induced or caused by the image feature.
  • Another method step is ascertaining (again in the above-mentioned sense) a respective overall motion vector for the image segments in total, thus for the respective image segments as a whole, by the computing device.
  • This can be effected based on at least one image segment of a further provided and segmented (preceding) frame.
  • a contour of the respective image segment can for example be approximated with a polygon in the frame and the preceding frame and then the corresponding positions of the polygon can be compared to be able to describe an overall motion of the respective image segment with the corresponding overall motion vector.
  • a further method step is then defining an image feature of a respective image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide or match except for a presettable tolerance.
  • This method step is carried out by the computing device, namely in particular for all of the image segments and therein preferably for all of the respective image features.
  • the respective motion vectors resemble each other except for a preset or presettable deviation, for example except for 10 percent or preferably 5 percent, in an angle and/or a length.
  • the presettable tolerance is preset to 10 percent and 5 percent, respectively.
  • the image features are characterized as reference image features or well traceable image features, the motion of which represented by the motion vector coincides or is consistent with the overall motion represented by the overall motion vector of the associated image segment. Thereby, it is ensured that the image features, which move consistently with the image segment, are further used and poorly traceable image features, for example the above described randomly generated false positives, are sorted out.
  • features which are arranged on a static object of the environment of the driver assistance device, generally have a higher consistency than image features on dynamic objects.
  • the reference image features consistently move with the image segment and also consistently in one direction, thus sudden motion jumps, which are recognized, but which are not plausible for an image feature, are for example sorted out.
  • a predetermined number of X image features, which consistently move can for example respectively be retained such that the number N - X of features is to be newly detected in the frame in a respective new step of the iterative method.
  • a further method step is defining at least one or more image segments, which represent a subfloor or ground of the environment of the motor vehicle in the frame, as a ground image segment by the computing device. This can for example occur in that an image area immediately in front of the camera, which, due to the orientation of the camera used for generating the evaluated frame, always represents the ground according to
  • a further method step is classifying at least one non-ground image segment, preferably of the remaining non-ground image segments, as a static or dynamic image segment, wherein the or a respective image segment is in particular only classified as a dynamic image segment if the motion vector or a motion vector distribution of at least one, preferably all, of the reference image features of the image segment to be classified differs from the motion vector or a motion vector distribution of at least one reference image feature or preferably of all of the reference image features of one or more ground image segments by more than a preset or presettable value.
  • an individual value can be correspondingly preset for the length of the motion vector, thus a magnitude of the flow or the motion, and/or a corresponding individual value can be preset for the angle of the motion vector or the orientation.
  • a difference of the respective motion vector can for example be formed for ascertaining the disparity.
  • the corresponding disparity between respective motion vector distributions can for example be ascertained by difference formation between the average values of the motion vector distribution or else by a distance of the respective distributions, which can for example be ascertained by the minimum of the differences between all of the motion vectors of the respective one distribution with all of the motion vectors of the respective other distribution.
  • the motion vector of the at least one reference image feature of the image segment to be classified does not differ from the motion vector of the reference image feature of the ground image segment by more than the preset value, in classification, at least one, in particular exactly one, reference image feature closest to the image segment to be classified, thus a reference image feature closest to the image segment to be classified or a reference image feature with the lowest distance to the image segment to be classified, of a ground image segment is selected.
  • the corresponding ground image segment is in particular a ground image segment closest to the image segment to be classified.
  • the image segment to be classified is classified as a dynamic image segment if the motion vector does not coincide with the motion vector of the at least one selected reference image feature of the ground image segment for more than a preset or presettable portion of the reference image features of the image segment to be classified except for the tolerance.
  • a preset portion can for example be 80 percent.
  • a length interval and/or an angle interval is preset as the tolerance for the ground image segment selected in classification based on the motion vectors of multiple reference image features of the ground image segment depending on a minimum value and a maximum value of the lengths and/or angles of the motion vectors such that the motion vector of the reference image feature of the image segment to be classified coincides with the motion vector of the selected reference image feature of the ground image segment if length and/or angle of the motion vector of the image segment to be classified are in the corresponding interval, thus the angle interval and/or the length interval.
  • the interval boundaries can in particular respectively be preset by the minimum value and the maximum value of the lengths and/or angles of the motion vectors of the ground image segment.
  • ground image segments are particularly accurately characterized and correspondingly the image segments can also be particularly accurately classified as moving relative to the ground image segment, thus dynamic image segments, or as standing relative to the ground image segment, thus static image segments.
  • the method is performed for a series of time steps with consecutive frames, wherein the overall motion vectors are ascertained based on a comparison of image segments corresponding to each other in consecutive frames and the motion vectors of the image features are ascertained based on a comparison of image features corresponding to each other in consecutive frames.
  • the reference image features are buffered and a coincidence of the respective motion vector with the overall motion vector is monitored for the reference image features over multiple consecutive time steps, for example via a suitable measure, and only a preset portion, for example 80 percent, of the reference image features with the greatest coincidence, thus the reference image features with the greatest consistency with the overall motion over a preset period of time and/or only reference image features with a coincidence exceeding a preset limit value continue to be defined as a reference image feature.
  • reference image feature or features for the ground image segment or the ground image segments are buffered with the associated motion vector or the associated motion vectors over a preset number of time steps.
  • one or more characteristics of the image segments are buffered over a preset number of time steps, in particular respective positions of a contour for the respective image segments, for example the above mentioned polygons as an approximation for the contour, and/or a size of the image segments.
  • a motion of the corresponding object associated with the image segment in the environment of the motor vehicle towards the camera from a uniform increase of the size of the image segment and vice versa a motion away from the camera from a constant reduction of the size of the image segment.
  • the overall motion vectors of the image segments are buffered over a preset number of time steps and the image segments are categorized in a respective motion class, in particular in a rotation class and/or a translation class and/or a scaling class, based on the buffered motion vectors.
  • image segments and in particular correspondingly image features, which are associated with a scaling class can be subjected to further analyzing steps by the computing device to simplify the classification of the image segment as a static or dynamic image segment.
  • the environment of the motor vehicle is recognized in the frame, in which a first safety distance to image segments classified as dynamic is ascertained and a further, second safety distance different from the first one to image segments classified as static is ascertained.
  • the environment of the motor vehicle can be divided into sectors in the frame, which are then respectively associated with dynamic or static image segments or dynamic or static objects, which each are composed of one or more image segments.
  • the respective safety distances can be made according to different specifications depending on the class of the image segments in the respective sector or corresponding motion classes of the image segments in the respective sector.
  • the respective safety distances can also be set depending on further factors such as for example an activated parking mode, a vehicle speed or further motor vehicle parameters.
  • the image characteristic includes a luminance and/or a color and/or a texture characteristic, for example a contrast structure.
  • the frame is provided by a camera device of the driver assistance device and represents an environment of the motor vehicle.
  • the invention also relates to a computing device for a driver assistance device of a motor vehicle, which is formed to segment a provided frame into a plurality of image segments depending on at least one image characteristic and respectively to detect at least one image feature in the image segments of the segmented frame as well as to ascertain a respective motion vector for the detected image features.
  • the computing device is formed to ascertain a respective overall motion vector for the image segments and to define an image feature of an image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance, as well as to define image segments, which represent a subfloor or ground in the frame, as a ground image segment and finally to classify at least one image segment as a static or dynamic image segment.
  • the respective image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of one or more ground image segments by more than a presettable value.
  • advantages and advantageous embodiments of the driver assistance device correspond to advantages and advantageous embodiments of the described method.
  • the invention also relates to a motor vehicle with such a computing device or a driver assistance device with such a computing device.
  • Fig. 1 a motor vehicle with an exemplary embodiment of a computing device, which is part of a driver assistance device;
  • Fig. 2 a schematic representation of an exemplary embodiment of a method for operating a computing device of a driver assistance device of a motor vehicle
  • Fig. 3 an exemplary frame, for which a free space recognition was performed. Therein, identical or functionally identical elements are provided with the same reference characters in the figures.
  • FIG. 1 an exemplary embodiment of a computing device 1 of a driver assistance device 2 in a motor vehicle 3 is illustrated.
  • the computing device 1 is presently coupled to a camera device 4, which is formed to provide a frame, presently a series of frames, to the computing device 1 .
  • the frame or frames each represent an environment 5 of the motor vehicle 3.
  • the driver assistance device 2 is presently also coupled to a brake device 6 such that a brake intervention can for example be performed by the driver assistance device 2.
  • the computing device 1 is formed to segment the provided frame into a plurality of image segments depending on at least one image characteristic and to respectively detect at least one image feature in the image segments of the segmented frame as well as to ascertain a respective motion vector for the detected image features.
  • the computing device 1 is also formed to ascertain a respective overall motion vector for the image segments and to define an image feature of an image segment as a reference image feature if the respective motion change of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance. Further, the computing device 1 is formed to define image segments representing a subfloor in the frame as a ground image segment and to classify an image segment as a static or dynamic image segment.
  • the image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of one or more ground image segments by more than a preset value.
  • the computing device 1 is in particular formed to perform a method as it is explained based on Fig. 2.
  • a method step is segmenting 8 the provided frame into a plurality of image segments. This occurs depending on at least one image characteristic, for example a color or a brightness, such that image areas with the same color and/or the same brightness, preferably contiguous image areas respectively with the same characteristic, are segmented to an image segment.
  • a further method step is detecting 9 respectively at least one image feature in the image segments of the segmented frame.
  • an image feature for example an edge or other shape, is detected as an image feature in each image segment.
  • a respective motion vector for the detected image features is effected.
  • a motion vector magnitude and thereby a flow speed of an optical flow and a motion vector orientation and thereby a flow orientation are presently determined for each image feature, which is quantified by the motion vector.
  • an optical flow can be determined from the motion vectors in the frame for the respective image features.
  • a further step is ascertaining 1 1 a respective overall motion vector for the individual image segments.
  • the image features of the respective image segments as the reference image features, for which the respective motion vectors coincide with the overall motion vector of the image segment associated with the image feature except for a presettable tolerance, is then effected.
  • the tolerance can in particular be individually preset for each image segment.
  • an individual selection criterion for the reference image features thus the image features, the motion of which is consistent with the motion of the associated image segment, is yet achieved by defining 12.
  • a further step is defining 13 image segments, which represent a subfloor in the frame, as a ground image segment based on the segmenting 8.
  • classifying 14 the respective image segments as either static or dynamic image segment is effected.
  • the image segments are classified as dynamic image segments, for which the motion vector of at least one reference image feature associated with the image segment to be classified differs from the motion vector of at least one reference image feature of at least one ground image segment by more than a preset value.
  • a feedback loop can be realized from defining 12 to detecting 9 in that a coincidence of the respective motion vector with the overall motion vector is verified for the reference image features over multiple consecutive time steps and only a preset portion of the reference image features with the greatest coincidence and/or only reference image features with a coincidence exceeding a preset limit value continue to be defined as a reference image feature.
  • the method is performed for a series of time steps with consecutive frames and reference image features once recognized and defined as such are further used as an image feature, however, they are therein checked for their persistent suitability.
  • recognizing 15 a free space is performed for the frame based on classifying 14, thus the environment 5 (Fig. 1 ) is analyzed with respect to present free spaces and there is associated a safety distance with the environment for objects, which correspond to an image segment classified as dynamic, and a further safety distance with objects, which correspond to an image segment classified as static.
  • outputting 16 a brake and/or acceleration signal or a direction signal for safe drive is presently effected based on the ascertained safety distances.
  • a ground area 18 of the frame is presently divided into a plurality of sectors 19a to 19v.
  • respective first safety distances df to dk and dn to dq, respectively were ascertained for the sectors 19f to 19k as well as 19n to 19q.
  • second safety distances da to de and dl and dm were presently ascertained.
  • the distances dq to dv which are correspondingly associated with the vectors 19q to 19v, were ascertained based on the free space recognition and correspondingly also based on image segments classified as static.
  • the described procedure for classifying image segments as static or dynamic thus results in extremely useful free space recognition, in which outputting 16 corresponding brake and/or steering and/or acceleration signals can be effected based on the ascertained distances da to dv and for example further information such as a steering angle of the motor vehicle.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention relates to a method for operating a computing device (1) of a driver assistance device (2) of a motor vehicle (3), including a) segmenting (8) a provided frame (17) into a plurality of image segments depending on at least one image characteristic; b) detecting (9) respectively at least one image feature in the image segments of the segmented frame (17); c) ascertaining (10) a respective motion vector for the detected image features; d) ascertaining (11) a respective overall motion vector for the image segments; e) defining (12) an image feature of an image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance; f) defining (13) image segments, which represent a ground in the frame (17), as a ground image segment; and g) classifying (14) an image segment as a static or dynamic image segment, wherein the image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of at least one ground image segment by more than a preset value to improve the object recognition of the driver assistance device (2).

Description

Classification of static and dynamic image segments in a driver assistance device of a motor vehicle
The invention relates to a method for operating a computing device of a driver assistance device of a motor vehicle, which serves for classifying an image segment as a static or dynamic image segment. Correspondingly, the invention relates to a driver assistance device for a motor vehicle including a computing device, which is formed to segment a provided frame into a plurality of image segments depending on at least one image characteristic and to respectively detect at least one image feature in the image segments of the segmented frame as well as to ascertain a respective formation vector for the detected image features.
The detection of image features (feature detection) and the prediction of an image flow (flow prediction), that is ascertaining respective motion vectors for image features, are two core parts of each method for detecting and classifying objects. Due to restrictions on the part of hardware, therein, the number of the respectively tracked image features is restricted to a preset number N, for example N = 800 features per frame. Therein, it is important to recognize and distribute this limited number of (image) features within the frame, respectively, such that at least one such image feature is associated with each object in the frame, that is each object in the environment of a corresponding motor vehicle represented by the frame. Only in this manner, each object can also be recognized and taken into account in corresponding downstream method steps, which is required for safe operation of a driver assistance device of a motor vehicle.
Therein, a series of problems is known in the area of detecting the objects:
Firstly, an object can be partially covered such that only a part of a static object is detected.
Secondly, accidentally, for example as a result of an image texture, a dynamic object or a dynamic image feature can erroneously be recognized. Thirdly, one and the same object can be recognized partially as a static object and partially as a dynamic, moving object by for example differently classifying different image features, which are associated with the same object.
Fourthly, it occurs that objects are not detected at all, for example if they are located close to a focus of expansion of the frame. The focus of expansion in a frame is that point, in which the motion vectors of all of the features converge or from which they all point away. Here, the length of a motion vector of a corresponding image feature will basically tend to zero if the corresponding image feature is closer to the focus of expansion. Since the motion vectors close to the point of expansion, which can also be referred to as vanishing point, have a low length, thus, a small error there already results in great consequential errors in a corresponding three-dimensional reconstruction. Therefore, some detection algorithms for objects have an area around the focus of expansion, within which motion vectors are filtered out. Here, omitted detection of the objects near the focus of expansion therefore occurs.
Fifthly, it occurs again and again that static objects are recognized as dynamic objects and vice versa. For example, a dynamic object like a vehicle moving parallel to the own motor vehicle is static compared to the own motor vehicle.
Sixthly, finally, the optical flow or image flow, that is the motion vectors used for determining the image flow, is scale-variant in the usual approaches.
Even image features, which are detected on a corresponding object and associated with it, are not necessarily well traceable image features, which are suitable for calculating the image flow and/or determining image segments or objects as static or dynamic. This is the principal reason that object recognition algorithms often fail in recognizing static objects in large distance as such.
At least the problem that an image feature is not recognized for some objects, that is an image feature is not associated with an object, can be solved by a so-called dense optical flow, that is an increase of the number of the image features with corresponding motion vectors (or trajectories), as it is possible in future driver assistance devices as a result of hardware. However, therein, a dense optical flow is as scale-variant as a less dense optical flow. Again, there is also the problem that better and worse traceable image features are difficult to differentiate. In fact, an optical flow inconsistent and inhomogeneous to greater extent is even to be expected. Namely, due to the finer sampling of respective random textures such as for example road or vegetation in the environment of the motor vehicle, image features are also erroneously recognized in the random textures. Thereby, the number of the false positives mentioned above under secondly will increase such that a more expensive classification of the image features is again required.
Thus, there is set the object to improve the object recognition of a driver assistance device of a motor vehicle, in particular improve classifying image features and/or image segments.
This object is solved by the subject matters of the independent claims. Advantageous embodiments are apparent from the dependent claims, the description and the figures.
As already indicated, there are better and worse traceable and trackable image features in each image area of a frame (individual image). Thus, the image features are differently well suited for ascertaining motions and representative motion vectors. Here, in the detection of objects, the same criteria are set for all of the image features in known methods to recognize well and poorly traceable image features as such. With current approaches, it is either not ensured that a well traceable image feature is present for each object in the image.
Such a uniform criterion for all of the image features cannot ensure the detection of well traceable image features in each image segment or each image area due to the number of the image areas with different characteristics, for example different texture
characteristics. Here, it is very important to individually detect well traceable image features respectively consistently behaving for each image segment and/or each object, namely independently of whether it is a static and dynamic object and correspondingly a static or dynamic image segment.
Thus, it is crucial to find image features as well traceable as possible in each image segment of the frame. These well traceable image features also referred to as reference image features in the following are then used to rank or to categorize or to classify an image area, that is an image segment, as static or dynamic. Therein, the static
characteristics of the well traceable image features are exploited in the proposed method. The approach described in the following ensures that well traceable image features are present in each image segment by for example separately analyzing the flow
characteristics of the image features in each image segment.
The invention relates to a method for operating a computing device of a driver assistance device of a motor vehicle including a series of methods steps. Here, one method step can be capturing an environment of the motor vehicle by a camera device of the driver assistance device with providing a captured frame on the part of the camera device. A method step is segmenting a provided frame, for example the captured camera frame of the camera device, into a plurality of image segments depending on at least one image characteristic by the computing device. However, the frame can for example also be a sensor card of a sensor device of the driver assistance device.
Segmenting or clustering the frame into image segments or image clusters can for example be effected based on a respective color as the image characteristic such that for example image areas with the same color or preferably contiguous image areas with the same color are treated as one image segment. Therein, each segmentation algorithm can basically be used. Here, one possibility is dividing the frame into respective sectors, which is advantageous upon lighting changes. Here, another possibility is for example a semantic classification in image segmentation, which can for example be performed by dividing an image into image blocks and for example dividing the image block with the respectively most complicated texture into further image blocks.
A next method step is detecting at least on image feature in the respective image segments, preferably all of the image segments, of the segmented frame by the computing device. Here, a number N of image features can for example be divided among the image segments according to a preset algorithm, that is at least one image feature can be associated or a corresponding processing place in the computing device can be associated with each image segment and then the corresponding image feature or features can be recognized or detected in the respective image segment.
A further method step is ascertaining (in the sense of determining or evaluating) a respective motion vector for the detected image features by the computing device. This can be effected based on an image feature associated with the respective image feature in the or a (preceding) further provided and segmented frame. Here and in the following, a motion vector can include a length and an angle, which correspondingly describes a flow magnitude or flow speed of the image feature as well as a flow orientation of the image feature, that is a size and/or orientation of the optical flow induced or caused by the image feature.
Another method step is ascertaining (again in the above-mentioned sense) a respective overall motion vector for the image segments in total, thus for the respective image segments as a whole, by the computing device. This can be effected based on at least one image segment of a further provided and segmented (preceding) frame. Hereto, a contour of the respective image segment can for example be approximated with a polygon in the frame and the preceding frame and then the corresponding positions of the polygon can be compared to be able to describe an overall motion of the respective image segment with the corresponding overall motion vector. Thus, there are recorded respective motion vectors for all of the detected image features of an image segment on the one hand as well as the overall motion for the image segment in total on the other hand.
A further method step is then defining an image feature of a respective image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide or match except for a presettable tolerance. This method step is carried out by the computing device, namely in particular for all of the image segments and therein preferably for all of the respective image features.
Therein, by coincidence or matching except for a presettable tolerance, it can be understood that the respective motion vectors resemble each other except for a preset or presettable deviation, for example except for 10 percent or preferably 5 percent, in an angle and/or a length. Thus, in this case, the presettable tolerance is preset to 10 percent and 5 percent, respectively. Thus, the image features are characterized as reference image features or well traceable image features, the motion of which represented by the motion vector coincides or is consistent with the overall motion represented by the overall motion vector of the associated image segment. Thereby, it is ensured that the image features, which move consistently with the image segment, are further used and poorly traceable image features, for example the above described randomly generated false positives, are sorted out.
Here, features, which are arranged on a static object of the environment of the driver assistance device, generally have a higher consistency than image features on dynamic objects. Thus, it is ensured via the described definition that the reference image features consistently move with the image segment and also consistently in one direction, thus sudden motion jumps, which are recognized, but which are not plausible for an image feature, are for example sorted out. Therein, as explained below, in iteratively performing the method, a predetermined number of X image features, which consistently move, can for example respectively be retained such that the number N - X of features is to be newly detected in the frame in a respective new step of the iterative method.
A further method step is defining at least one or more image segments, which represent a subfloor or ground of the environment of the motor vehicle in the frame, as a ground image segment by the computing device. This can for example occur in that an image area immediately in front of the camera, which, due to the orientation of the camera used for generating the evaluated frame, always represents the ground according to
experience.
Then, a further method step is classifying at least one non-ground image segment, preferably of the remaining non-ground image segments, as a static or dynamic image segment, wherein the or a respective image segment is in particular only classified as a dynamic image segment if the motion vector or a motion vector distribution of at least one, preferably all, of the reference image features of the image segment to be classified differs from the motion vector or a motion vector distribution of at least one reference image feature or preferably of all of the reference image features of one or more ground image segments by more than a preset or presettable value.
Therein, an individual value can be correspondingly preset for the length of the motion vector, thus a magnitude of the flow or the motion, and/or a corresponding individual value can be preset for the angle of the motion vector or the orientation. Here, a difference of the respective motion vector can for example be formed for ascertaining the disparity. The corresponding disparity between respective motion vector distributions can for example be ascertained by difference formation between the average values of the motion vector distribution or else by a distance of the respective distributions, which can for example be ascertained by the minimum of the differences between all of the motion vectors of the respective one distribution with all of the motion vectors of the respective other distribution.
This has the advantage that the respective individual characteristics of the individual image segments are taken into account in classifying the image segments as either static or dynamic image segment and therein all of the initially mentioned problems of the known approaches for detecting objects are overcome. Therein, the proposed solution is simply constructed and can be easily implemented in existing computing devices of driver assistance devices. At the same time, it also opens stepwise more accurate classification as is described in the following based on further advantageous embodiments.
Namely, in such a further advantageous embodiment, it is provided that if the motion vector of the at least one reference image feature of the image segment to be classified does not differ from the motion vector of the reference image feature of the ground image segment by more than the preset value, in classification, at least one, in particular exactly one, reference image feature closest to the image segment to be classified, thus a reference image feature closest to the image segment to be classified or a reference image feature with the lowest distance to the image segment to be classified, of a ground image segment is selected. Therein, the corresponding ground image segment is in particular a ground image segment closest to the image segment to be classified. The image segment to be classified is classified as a dynamic image segment if the motion vector does not coincide with the motion vector of the at least one selected reference image feature of the ground image segment for more than a preset or presettable portion of the reference image features of the image segment to be classified except for the tolerance. Such a preset portion can for example be 80 percent. In case of a single reference image feature, the coincidence between the one reference image feature of the image segment to be classified and the reference image feature of a ground image segment closest to it decides on the result of classification, thus if the image segment is classified as static or dynamic.
This has the advantage that if the image segment is not recognized as a dynamic image segment, the result is again checked and thus accuracy of the method is increased. At the same time, this then slightly more expensive step is only effected in the really initially obscure cases, whereby computing capacity is saved. In that especially the reference image feature closest to the image segment to be classified is used for classification, an influence of geometric distortions is also reduced in the method.
Therein, it is provided in a further advantageous embodiment that a length interval and/or an angle interval is preset as the tolerance for the ground image segment selected in classification based on the motion vectors of multiple reference image features of the ground image segment depending on a minimum value and a maximum value of the lengths and/or angles of the motion vectors such that the motion vector of the reference image feature of the image segment to be classified coincides with the motion vector of the selected reference image feature of the ground image segment if length and/or angle of the motion vector of the image segment to be classified are in the corresponding interval, thus the angle interval and/or the length interval. Therein, the interval boundaries can in particular respectively be preset by the minimum value and the maximum value of the lengths and/or angles of the motion vectors of the ground image segment.
This has the advantage that the respective ground image segments are particularly accurately characterized and correspondingly the image segments can also be particularly accurately classified as moving relative to the ground image segment, thus dynamic image segments, or as standing relative to the ground image segment, thus static image segments.
In a further advantageous embodiment, it is provided that the method is performed for a series of time steps with consecutive frames, wherein the overall motion vectors are ascertained based on a comparison of image segments corresponding to each other in consecutive frames and the motion vectors of the image features are ascertained based on a comparison of image features corresponding to each other in consecutive frames.
This has the advantage that a respective environment of the motor vehicle can be correspondingly flexibly and currently captured and analyzed by identifying and classifying the respective dynamic and static image segments in the frames as such, respectively.
Therein, it is provided in a further advantageous embodiment that the reference image features are buffered and a coincidence of the respective motion vector with the overall motion vector is monitored for the reference image features over multiple consecutive time steps, for example via a suitable measure, and only a preset portion, for example 80 percent, of the reference image features with the greatest coincidence, thus the reference image features with the greatest consistency with the overall motion over a preset period of time and/or only reference image features with a coincidence exceeding a preset limit value continue to be defined as a reference image feature.
This has the advantage that only respectively the best reference image features are further used as such and a determinable portion of reference image features can be newly preset, that is the available computing capacity is utilized with greatest possible efficiency and effectiveness with a maximum number N of processible image features. In a further advantageous embodiment, it is provided that the reference image feature or features for the ground image segment or the ground image segments are buffered with the associated motion vector or the associated motion vectors over a preset number of time steps.
This has the advantage that consistency of the respective motions can be particularly well assessed and monitored.
In a further advantageous embodiment, it is provided that one or more characteristics of the image segments are buffered over a preset number of time steps, in particular respective positions of a contour for the respective image segments, for example the above mentioned polygons as an approximation for the contour, and/or a size of the image segments. Thus, there can for example be derived a motion of the corresponding object associated with the image segment in the environment of the motor vehicle towards the camera from a uniform increase of the size of the image segment and vice versa a motion away from the camera from a constant reduction of the size of the image segment.
This has the advantage that additional criteria can be taken into account in classification and thus the accuracy of the method is further increased.
In a further advantageous embodiment, it is provided that the overall motion vectors of the image segments are buffered over a preset number of time steps and the image segments are categorized in a respective motion class, in particular in a rotation class and/or a translation class and/or a scaling class, based on the buffered motion vectors. Therein, especially image segments and in particular correspondingly image features, which are associated with a scaling class, can be subjected to further analyzing steps by the computing device to simplify the classification of the image segment as a static or dynamic image segment.
This has the advantage that the method is further increased in accuracy and it cannot only be differentiated between static and dynamic image segment, but the dynamic image segments can be further divided. This is for example advantageous for object formation, in which respective image segments are associated with one or more objects in the environment of the motor vehicle. Here, image segments, which move in similar or identical manner, can for example be associated with one object. In a further advantageous embodiment, it is provided that a free space recognition is performed for the frame after classification, thus a respective free space in the
environment of the motor vehicle is recognized in the frame, in which a first safety distance to image segments classified as dynamic is ascertained and a further, second safety distance different from the first one to image segments classified as static is ascertained. Therein, the environment of the motor vehicle can be divided into sectors in the frame, which are then respectively associated with dynamic or static image segments or dynamic or static objects, which each are composed of one or more image segments. Therein, the respective safety distances can be made according to different specifications depending on the class of the image segments in the respective sector or corresponding motion classes of the image segments in the respective sector. The respective safety distances can also be set depending on further factors such as for example an activated parking mode, a vehicle speed or further motor vehicle parameters.
In a further advantageous embodiment, it is provided that the image characteristic includes a luminance and/or a color and/or a texture characteristic, for example a contrast structure.
These characteristics have proven particularly advantageous for segmenting an image into image segments in the area of the driver assistance devices for motor vehicles.
In a further advantageous embodiment, it is provided that the frame is provided by a camera device of the driver assistance device and represents an environment of the motor vehicle.
The invention also relates to a computing device for a driver assistance device of a motor vehicle, which is formed to segment a provided frame into a plurality of image segments depending on at least one image characteristic and respectively to detect at least one image feature in the image segments of the segmented frame as well as to ascertain a respective motion vector for the detected image features.
Therein, it is important that the computing device is formed to ascertain a respective overall motion vector for the image segments and to define an image feature of an image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance, as well as to define image segments, which represent a subfloor or ground in the frame, as a ground image segment and finally to classify at least one image segment as a static or dynamic image segment. Therein, the respective image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of one or more ground image segments by more than a presettable value.
Here, advantages and advantageous embodiments of the driver assistance device correspond to advantages and advantageous embodiments of the described method.
The invention also relates to a motor vehicle with such a computing device or a driver assistance device with such a computing device.
The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations Implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim.
Moreover, implementations and feature combinations are to be considered as disclosed, in particular by the implementations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.
Below, embodiments of the invention are explained in more detail based on schematic drawings. There show:
Fig. 1 a motor vehicle with an exemplary embodiment of a computing device, which is part of a driver assistance device;
Fig. 2 a schematic representation of an exemplary embodiment of a method for operating a computing device of a driver assistance device of a motor vehicle; and
Fig. 3 an exemplary frame, for which a free space recognition was performed. Therein, identical or functionally identical elements are provided with the same reference characters in the figures.
In Fig. 1 , an exemplary embodiment of a computing device 1 of a driver assistance device 2 in a motor vehicle 3 is illustrated. Therein, the computing device 1 is presently coupled to a camera device 4, which is formed to provide a frame, presently a series of frames, to the computing device 1 . Therein, the frame or frames each represent an environment 5 of the motor vehicle 3. Therein, the driver assistance device 2 is presently also coupled to a brake device 6 such that a brake intervention can for example be performed by the driver assistance device 2.
Therein, the computing device 1 is formed to segment the provided frame into a plurality of image segments depending on at least one image characteristic and to respectively detect at least one image feature in the image segments of the segmented frame as well as to ascertain a respective motion vector for the detected image features.
Here, it is important that the computing device 1 is also formed to ascertain a respective overall motion vector for the image segments and to define an image feature of an image segment as a reference image feature if the respective motion change of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance. Further, the computing device 1 is formed to define image segments representing a subfloor in the frame as a ground image segment and to classify an image segment as a static or dynamic image segment.
Therein, the image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of one or more ground image segments by more than a preset value.
Thus, the computing device 1 is in particular formed to perform a method as it is explained based on Fig. 2.
In Fig. 2, an exemplary embodiment of a method for operating the computing device 1 is illustrated. Presently, providing 7 a frame to the computing device 1 is first effected. Here, it can for example be a frame of the camera device 4 (Fig. 1 ). Therein, a method step is segmenting 8 the provided frame into a plurality of image segments. This occurs depending on at least one image characteristic, for example a color or a brightness, such that image areas with the same color and/or the same brightness, preferably contiguous image areas respectively with the same characteristic, are segmented to an image segment. A further method step is detecting 9 respectively at least one image feature in the image segments of the segmented frame. Thus, an image feature, for example an edge or other shape, is detected as an image feature in each image segment.
In a further method step, correspondingly ascertaining 10 a respective motion vector for the detected image features is effected. Thus, a motion vector magnitude and thereby a flow speed of an optical flow and a motion vector orientation and thereby a flow orientation are presently determined for each image feature, which is quantified by the motion vector. Thus, an optical flow can be determined from the motion vectors in the frame for the respective image features. A further step is ascertaining 1 1 a respective overall motion vector for the individual image segments. Based on the respective motion vectors for the detected image features and the ascertained overall motion vectors, defining 12 the image features of the respective image segments as the reference image features, for which the respective motion vectors coincide with the overall motion vector of the image segment associated with the image feature except for a presettable tolerance, is then effected. Therein, the tolerance can in particular be individually preset for each image segment. However, even if a single tolerance is preset for all of the image segments, thus, since the overall motion vector is specific to each image segment, an individual selection criterion for the reference image features, thus the image features, the motion of which is consistent with the motion of the associated image segment, is yet achieved by defining 12. Thereby, erroneously recognized image features or other image features, which are not suitable for classifying 14 the image segment as a static or dynamic image segment described below, remain unconsidered.
A further step is defining 13 image segments, which represent a subfloor in the frame, as a ground image segment based on the segmenting 8.
Finally, classifying 14 the respective image segments as either static or dynamic image segment is effected. In classifying 14, the image segments are classified as dynamic image segments, for which the motion vector of at least one reference image feature associated with the image segment to be classified differs from the motion vector of at least one reference image feature of at least one ground image segment by more than a preset value. Therein, a feedback loop can be realized from defining 12 to detecting 9 in that a coincidence of the respective motion vector with the overall motion vector is verified for the reference image features over multiple consecutive time steps and only a preset portion of the reference image features with the greatest coincidence and/or only reference image features with a coincidence exceeding a preset limit value continue to be defined as a reference image feature.
Thus, the method is performed for a series of time steps with consecutive frames and reference image features once recognized and defined as such are further used as an image feature, however, they are therein checked for their persistent suitability.
In the present example, recognizing 15 a free space is performed for the frame based on classifying 14, thus the environment 5 (Fig. 1 ) is analyzed with respect to present free spaces and there is associated a safety distance with the environment for objects, which correspond to an image segment classified as dynamic, and a further safety distance with objects, which correspond to an image segment classified as static. Finally, outputting 16 a brake and/or acceleration signal or a direction signal for safe drive is presently effected based on the ascertained safety distances.
In Fig. 3, an exemplary frame 17 of the environment 5 is illustrated. Therein, a ground area 18 of the frame is presently divided into a plurality of sectors 19a to 19v. Based on the image segments recognized as dynamic, which presently represent two respective objects 20a, 20b, here, respective first safety distances df to dk and dn to dq, respectively, were ascertained for the sectors 19f to 19k as well as 19n to 19q. Based on the image segments classified as static, therein, second safety distances da to de and dl and dm were presently ascertained. Here, the distances dq to dv, which are correspondingly associated with the vectors 19q to 19v, were ascertained based on the free space recognition and correspondingly also based on image segments classified as static.
The described procedure for classifying image segments as static or dynamic thus results in extremely useful free space recognition, in which outputting 16 corresponding brake and/or steering and/or acceleration signals can be effected based on the ascertained distances da to dv and for example further information such as a steering angle of the motor vehicle.

Claims

Claims
1 . Method for operating a computing device (1 ) of a driver assistance device (2) of a motor vehicle (3) including the method steps of:
a) segmenting (8) a provided frame (17) into a plurality of image segments depending on at least one image characteristic;
b) detecting (9) at least one image feature in the respective image segments of the segmented frame (17);
c) ascertaining (10) a respective motion vector for the detected image features; d) ascertaining (1 1 ) a respective overall motion vector for the image segments; e) defining (12) an image feature of an image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance;
f) defining (13) image segments representing a ground in the frame (17) as a ground image segment;
g) classifying (14) an image segment as a static or dynamic image segment, wherein the image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of at least one ground image segment by more than a preset value.
2. Method according to claim 1 ,
characterized in that
if the motion vector of the at least one reference image feature of the image segment to be classified does not differ from the motion vector of the reference image feature of the ground image segment by more than the preset value, in classifying (14), a reference image feature of a ground image segment closest to the image segment to be classified is selected and the image segment to be classified is classified as a dynamic image segment if the motion vector does not coincide with the motion vector of the selected reference image feature of the ground image segment for more than a preset portion of the reference image features of the image segment to be classified.
3. Method according to claim 2,
characterized in that
a length interval and/or an angle interval is preset as a tolerance for the ground image segment based on the motion vectors of multiple reference image features of the ground image segment depending on a minimum value and a maximum value of the lengths and/or angles of the motion vectors.
4. Method according to any one of the preceding claims,
characterized in that
the method is performed for a series of time steps with consecutive frames, wherein the overall motion vectors are ascertained in consecutive frames based on a comparison of image segments corresponding to each other and the motion vectors of the image features are ascertained based on a comparison of image features corresponding to each other in consecutive frames.
5. Method according to claim 4,
characterized in that
a coincidence of the respective motion vector with the overall motion vector is monitored for the reference image features over multiple consecutive time steps and only a preset portion of the reference image features with the greatest coincidence and/or only reference image features with a coincidence exceeding a preset limit value continue to be defined as a reference image feature.
6. Method according to claim 4 or 5,
characterized in that
the reference image feature or features for the ground image segment or segments are buffered with the associated motion vector or vectors over a preset number of time steps
7. Method according to any one of claims 4 to 6,
characterized in that
one or more characteristics of the image segments are buffered over a preset number of time steps, in particular respective positions of a boundary for the image segments and/or a size of the image segments.
8. Method according to any one of claims 4 to 7,
characterized in that
the overall motion vectors of the image segments are buffered over a preset number of time steps and the image segments are categorized in a respective motion class, in particular in a rotation class or a translation class or a scaling class, based on the buffered motion vectors.
9. Method according to any one of the preceding claims,
characterized in that
after classifying (14), a free space recognition is performed for the frame (17), in which there are ascertained a safety distance (df-dk, dn-dq) to image segments classified as dynamic and a further safety distance (da-de, dl, dm, dr-dv) to image segments classified as static.
10. Method according to any one of the preceding claims,
characterized in that
the image characteristic includes a luminance and/or a color and/or a texture characteristic.
1 1 . Method according to any one of the preceding claims,
characterized in that
the frame (17) is provided by a camera device (4) of the driver assistance device (2) and represents an environment (5) of the motor vehicle (3).
12. Computing device (1 ) for a driver assistance device (2) for a motor vehicle (3), which is formed to segment (8) a provided frame (17) into a plurality of image segments depending on at least one image characteristic and to respectively detect (9) at least one image feature in the image segments of the segmented frame (17), as well as to ascertain a respective motion vector for the detected image features,
characterized in that
the computing device (1 ) is formed to ascertain a respective overall motion vector for the image segments and to define an image feature of an image segment as a reference image feature if the respective motion vector of the image feature and the overall motion vector of the image segment associated with the image feature coincide except for a presettable tolerance, as well as to define image segments, which represent a ground in the frame (17), as a ground image segment; and finally to classify an image segment as a static or dynamic image segment, wherein the image segment is classified as a dynamic image segment if the motion vector of at least one reference image feature of the image segment to be classified differs from the motion vector of at least one reference image feature of at least one ground image segment by more than a preset value.
13. Motor vehicle (3) with a computing device (1 ) according to claim 12.
PCT/EP2018/065584 2017-06-22 2018-06-13 CLASSIFICATION OF STATIC AND DYNAMIC IMAGE SEGMENTS IN A DEVICE FOR DRIVING A MOTOR VEHICLE Ceased WO2018234109A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102017113794.7 2017-06-22
DE102017113794.7A DE102017113794A1 (en) 2017-06-22 2017-06-22 Classification of static and dynamic image segments in a driver assistance device of a motor vehicle

Publications (1)

Publication Number Publication Date
WO2018234109A1 true WO2018234109A1 (en) 2018-12-27

Family

ID=62684770

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2018/065584 Ceased WO2018234109A1 (en) 2017-06-22 2018-06-13 CLASSIFICATION OF STATIC AND DYNAMIC IMAGE SEGMENTS IN A DEVICE FOR DRIVING A MOTOR VEHICLE

Country Status (2)

Country Link
DE (1) DE102017113794A1 (en)
WO (1) WO2018234109A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118552880A (en) * 2024-07-26 2024-08-27 江苏汇智智能数字科技有限公司 Picture identification processing method based on image detection and model analysis

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050152580A1 (en) * 2003-12-16 2005-07-14 Kabushiki Kaisha Toshiba Obstacle detecting apparatus and method
US20080291285A1 (en) * 2007-05-24 2008-11-27 Casio Computer Co., Ltd. Image processing apparatus for calculating global motion vector of plurality of image frames
WO2016151977A1 (en) * 2015-03-26 2016-09-29 パナソニックIpマネジメント株式会社 Moving body detection device, image processing device, moving body detection method, and integrated circuit
WO2017017140A1 (en) * 2015-07-29 2017-02-02 Connaught Electronics Ltd. Method for capturing an object on a road in the environment of a motor vehicle, camera system and motor vehicle using the same

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10375287B2 (en) * 2009-10-21 2019-08-06 Disney Enterprises, Inc. Object trail-based analysis and control of video
GB2484133B (en) * 2010-09-30 2013-08-14 Toshiba Res Europ Ltd A video analysis method and system
US9147260B2 (en) * 2010-12-20 2015-09-29 International Business Machines Corporation Detection and tracking of moving objects
US9852341B2 (en) * 2013-02-04 2017-12-26 Harman International Industries, Incorporation Method and system for detecting moving objects

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050152580A1 (en) * 2003-12-16 2005-07-14 Kabushiki Kaisha Toshiba Obstacle detecting apparatus and method
US20080291285A1 (en) * 2007-05-24 2008-11-27 Casio Computer Co., Ltd. Image processing apparatus for calculating global motion vector of plurality of image frames
WO2016151977A1 (en) * 2015-03-26 2016-09-29 パナソニックIpマネジメント株式会社 Moving body detection device, image processing device, moving body detection method, and integrated circuit
US20180012368A1 (en) * 2015-03-26 2018-01-11 Panasonic Intellectual Property Management Co., Ltd. Moving object detection device, image processing device, moving object detection method, and integrated circuit
WO2017017140A1 (en) * 2015-07-29 2017-02-02 Connaught Electronics Ltd. Method for capturing an object on a road in the environment of a motor vehicle, camera system and motor vehicle using the same

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118552880A (en) * 2024-07-26 2024-08-27 江苏汇智智能数字科技有限公司 Picture identification processing method based on image detection and model analysis

Also Published As

Publication number Publication date
DE102017113794A1 (en) 2018-12-27

Similar Documents

Publication Publication Date Title
Marzougui et al. A lane tracking method based on progressive probabilistic Hough transform
US10943131B2 (en) Image based lane marking classification
KR102099265B1 (en) System and method for pedestrian-vehicle collision warning based on pedestrian state
US8447139B2 (en) Object recognition using Haar features and histograms of oriented gradients
US7139411B2 (en) Pedestrian detection and tracking with night vision
US9213901B2 (en) Robust and computationally efficient video-based object tracking in regularized motion environments
US8670592B2 (en) Clear path detection using segmentation-based method
CN101950350B (en) Clear path detection using a hierachical approach
EP1837803A2 (en) Headlight, taillight and streetlight detection
CN107667378B (en) Method and apparatus for identifying and evaluating road surface reflections
CN102314599A (en) Identification and deviation-detection method for lane
CN111783573B (en) High beam detection method, device and equipment
JP2012109945A (en) Vulnerable road user protection system
US12243321B2 (en) Method for determining a semantic free space
Maldonado-Bascon et al. Traffic sign recognition system for inventory purposes
US20230280466A1 (en) Vehicle lidar system and object detection method thereof
Shang et al. A novel method for vehicle headlights detection using salient region segmentation and PHOG feature
Ashik et al. Recognizing bangladeshi traffic signs in the wild
WO2018234109A1 (en) CLASSIFICATION OF STATIC AND DYNAMIC IMAGE SEGMENTS IN A DEVICE FOR DRIVING A MOTOR VEHICLE
KR102885172B1 (en) A road guard detection method and apparatus for 3D point cloud with cascade classifier
CN115115978A (en) Object recognition method, device, storage medium and processor
Sadik et al. Vehicles detection and tracking in advanced & automated driving systems: Limitations and challenges
KR101437228B1 (en) Obstacle detection device and method using boundary weighting
JP7379268B2 (en) Image processing device
Freeman et al. Enhancing surveillance camera FOV quality via semantic line detection and classification with deep hough transform

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18732704

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18732704

Country of ref document: EP

Kind code of ref document: A1