EP3872693A1 - Verfahren und systeme zur objektdetektion - Google Patents

Verfahren und systeme zur objektdetektion Download PDF

Info

Publication number
EP3872693A1
EP3872693A1 EP20160077.2A EP20160077A EP3872693A1 EP 3872693 A1 EP3872693 A1 EP 3872693A1 EP 20160077 A EP20160077 A EP 20160077A EP 3872693 A1 EP3872693 A1 EP 3872693A1
Authority
EP
European Patent Office
Prior art keywords
pixels
image
determined
pixel
implemented method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP20160077.2A
Other languages
English (en)
French (fr)
Inventor
Piotr BOGACKI
Rafal Dlugosz
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aptiv Technologies AG
Original Assignee
Aptiv Technologies Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aptiv Technologies Ltd filed Critical Aptiv Technologies Ltd
Priority to EP20160077.2A priority Critical patent/EP3872693A1/de
Priority to US17/178,098 priority patent/US11562575B2/en
Priority to CN202110216813.4A priority patent/CN113327252B/zh
Publication of EP3872693A1 publication Critical patent/EP3872693A1/de
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]

Definitions

  • the present disclosure relates to methods and systems for object detection, in particular for headlight recognition or for traffic sign recognition.
  • Digital imaging devices such as digital cameras, are used in various automotive applications.
  • One such application is adaptive headlight control to switch between low beam and high beam.
  • Headlight control is relevant to the safety of the vehicles, since high beam light may dazzle a driver from an oncoming or a preceding vehicle by an excessive glare.
  • Adaptive headlight control may be carried out based on headlight detection.
  • the present disclosure provides a computer implemented method, a computer system, a vehicle, and a non-transitory computer readable medium according to the independent claims. Embodiments are given in the subclaims, the description and the drawings.
  • the present disclosure is directed at a computer implemented method for object detection, the method including the following steps carried out by computer hardware components: acquiring an image; determining a pixel of the image as a base pixel; determining coordinates of a plurality of sets of target pixels, each set of target pixels including a plurality of pixels in a respective pre-determined relationship to the base pixel; for each of the sets of target pixels, determining information representing values of the pixels in the respective set of target pixels; and determining whether a pre-determined object is shown in the image based on the determined information.
  • pixels along pre-determined sets of pixels (for example lines) in the image may be evaluated, and based on this evaluation it may be determined whether a pre-determined object (for example a headlight of a vehicle or a pre-determined traffic sign) is shown on the image.
  • a pre-determined object for example a headlight of a vehicle or a pre-determined traffic sign
  • the pre-determined relationship may be a spatial relationship.
  • each set of target pixels may be a line starting from the base pixel.
  • the respective information representing values of the pixels in the respective set of target pixels for each of the sets of target pixels may be referred to as a descriptor, which may describe the information provided in the image so that a machine learning method, for example an artificial neural network, may evaluate the image, for example to determine whether a pre-determined object is shown on the image, without having to consider each and every pixel of the image.
  • a machine learning method for example an artificial neural network
  • the pixels of the image may be evaluated to determine information that is more descriptive or more useful for the machine learning method than the pixel values as such, and this information may be the descriptor.
  • the descriptor may describe certain image regions.
  • the descriptor and the method which determines and uses the descriptor may be used in a system for providing an image descriptor for distinguishing vehicle lights from other light sources in proximity of the road and may be used in various ADAS (advanced driver-assistance systems) functions, for example adaptive headlight control (AHC).
  • ADAS advanced driver-assistance systems
  • AHC adaptive headlight control
  • the pre-determined object may include or may be a headlight of a vehicle.
  • Information indicating whether a headlight of a vehicle is present in the image may be used for adaptive headlight control, for example to avoid glaring of other drivers, while providing as much light as possible in the surrounding of the vehicle on which the method is carried out.
  • the pre-determined object may include or may be a traffic sign.
  • Information indicating whether a pre-determined traffic sign is present in the image may be used for cruise control (for example to set the maximum allowable speed) or for general vehicle behavior (for example to avoid overtaking in zones where overtaking is not allowed).
  • each set of target pixels may include pixels along a line through the base pixel in the image.
  • a line in an image consists of several (discrete) pixels, so that the line actually is an approximation of a line in the image.
  • the pixels which approximate the line through the starting point and with the desired direction or through the desired end point may, for example, be determined according to Bresenham's method.
  • the respective lines of the plurality of sets of target pixels may have pairwise different directions. According to another aspect, the respective lines of the plurality of sets of target pixels may include pairs of lines having opposite directions.
  • the image may be acquired based on determining an area of high intensity (or high brightness) in an input image (which may for example be provided by a camera mounted on a vehicle) and determining a crop of the input image around the area of high intensity (or high brightness) as the image. Pixels in the input image may be considered as having high intensity if the intensity is higher than a pre-determined threshold.
  • the pre-determined threshold may be determined based on the input image (for example based on an average intensity or a median intensity). It will be understood that intensity and brightness may be used interchangeably.
  • the information representing values of the pixels in the respective set of target pixels may include or may be the values of the pixels in the respective set of target pixels. Using the values of the pixels may keep all information of the pixels.
  • the information representing values of the pixels in the respective set of target pixels may include or may be binarized values of the pixels in the respective set of target pixels. Binarized values of the pixels may be represented by "0" and "1" or any other binary representation having two different values.
  • Using a binary representation may reduce memory requirements, since instead of the entire information of the pixels (which is more than one bit per pixel in case of grayscale images or color images), only one bit per pixel needs to be stored.
  • One of the two binary values may be used for representing bright pixels (which may have an intensity or brightness higher than a pre-determined threshold), and the other one of the two binary values may be used for representing dark pixels (which may have an intensity or brightness lower than a pre-determined threshold).
  • the pre-determined threshold may be determined based on the input image (for example based on an average intensity of the input image or a median intensity of the input image).
  • a fuzzy representation with more than two possible values (for example with four values representing "very dark”, “dark”, “bright” and “very bright”) may be used.
  • the information representing values of the pixels in the respective set of target pixels may include or may be information of pixels along a pre-determined order of the pixels until a first pixel with a pre-determined property is present.
  • the pre-determined order may for example be an order of subsequent pixels along a line.
  • the pre-determined property may for example be having an intensity (or brightness) lower than a pre-determined threshold (i.e. a dark pixel).
  • the information of pixels may include a count of pixels.
  • the information may provide a number of bright pixels before the first dark pixel is present in the pre-determined order. Even if after the first dark pixel one or more further bright pixels would be present, these one or more further bright pixels would not be included in the count. It has been found that this may make the method robust against noise or reflections.
  • the information representing values of the pixels in the respective set of target pixels may include or may be information of pixels having a pre-determined property (for example of being a bright pixel or for example of being a dark pixel).
  • the information representing values of the pixels in the respective set of target pixels includes the number of pixels with the pre-determined property.
  • the number of pixels may be the count of pixels.
  • all bright pixels may be counted in the respective sets of pixels, irrespective of an order of the pixels in the set and irrespective of where dark pixels may be present (for example in between bright pixels).
  • the information representing values of the pixels in the respective set of target pixels may include or may be an indicator for each of the pixels indicating whether the respective pixel has the pre-determined property (for example of being a bright pixel or for example of being a dark pixel). This may result in binarized representations for the respective pixels.
  • the present disclosure is directed at a computer system, said computer system including a plurality of computer hardware components configured to carry out several or all steps of the computer implemented method described herein.
  • the computer system may include a plurality of computer hardware components (for example a processor, for example processing unit or processing device or processing network, at least one memory, for example memory unit or memory device or memory network, and at least one non-transitory data storage). It will be understood that further computer hardware components may be provided and used for carrying out steps of the computer implemented method in the computer system.
  • the non-transitory data storage and/or the memory may include a computer program for instructing the computer to perform several or all steps or aspects of the computer implemented method described herein, for example using the processor and the at least one memory.
  • the present disclosure is directed at a vehicle including a camera configured to acquire the image (or an input image which may be cropped to obtain the image) and the computer system as described above.
  • the vehicle further includes: at least one headlight; and a control system configured to control the at least one headlight based on whether a headlight of another vehicle is shown in the image acquired by the camera.
  • the control system may control the headlight to switch between low beam and high beam, thereby providing an Adaptive Headlight Control (AHC) functionality.
  • AHC Adaptive Headlight Control
  • the present disclosure is directed at a non-transitory computer readable medium including instructions for carrying out several or all steps or aspects of the computer implemented method described herein.
  • the computer readable medium may be configured as: an optical medium, such as a compact disc (CD) or a digital versatile disk (DVD); a magnetic medium, such as a hard disk drive (HDD); a solid state drive (SSD); a read only memory (ROM), such as a flash memory; or the like.
  • the computer readable medium may be configured as a data storage that is accessible via a data connection, such as an internet connection.
  • the computer readable medium may, for example, be an online data repository or a cloud storage.
  • the present disclosure is also directed at a computer program for instructing a computer to perform several or all steps or aspects of the computer implemented method described herein.
  • Headlights of vehicles may operate in two modes by generating low beams and high beams.
  • Low beams in some countries must be used at daytime but mainly low beams are designed to illuminate the road at night when other vehicles are present.
  • high beams may be used since high beams provide significantly more light.
  • Adaptive Headlight Control may be provided in vehicles as a functionality of automatically switching between low beam and high beam.
  • a descriptor may be provided which may be used for AHC or for any other applications where vehicle lights need to be distinguished from other sources of lights located in the proximity of the road.
  • the AHC system may be responsible for automatic switching between high and low beams in the situation when a driver from an oncoming or preceding car could be dazzled by an excessive glare.
  • AHC is intended to work mainly at night or under insufficient light conditions. This may result in the necessity of finding special characteristics of vehicles moving at night time.
  • a descriptor may be provided which may describe light blobs that are detected in the image from the camera mounted on (for example inside) the vehicle.
  • a blob may be an area in an image of high intensity or high brightness (for example higher than in the surrounding pixels).
  • various processing stages may be carried out based on an image received from a camera mounted on the vehicle.
  • the processing stages may include image preprocessing, image segmentation and spots/blobs recognition in the camera image, determination of at least one descriptor, and classification of the light spots, like will be described in more detail below.
  • An original camera image ( I 1 ) may be the input for this stage and an enhanced image ( I 2 ) may be produced as its output.
  • the I 2 images may be received as an input, output from the previous stage of image preprocessing.
  • the aim of this stage may be the localization of light sources in the (camera) images, and cropping the part of the images where the light sources are found.
  • the result of the signal processing carried out at this stage is a set of images, I 3, n , with predefined size, e.g. 15x15 (odd version) or 16x16 (even version) pixels, which are resized parts (i.e. crops which may have been resized) of the I 2 image.
  • the center of the detected light blob may more or less overlap with the center of a given image I 3, n .
  • the index n may be the number of light blobs found in a given image frame.
  • particular I 3,n images are further processed.
  • Various image descriptors may be determined in order to extract specific features of the particular light spots.
  • Various descriptors applicable in AHC functionality may include Histogram of Oriented Gradients (HoG), Local Binary Pattern (LBP), SIFT (Scale-Invariant Features), SURF (Speeded Up Robust Features), intensity features, color information, and/ or blob location.
  • HoG Histogram of Oriented Gradients
  • LBP Local Binary Pattern
  • SIFT Scale-Invariant Features
  • SURF Speeded Up Robust Features
  • the feature values i.e. the values of the descriptors
  • a classifier for example for a convolutional neural network (CNN) classifier.
  • the light spots can be assigned to one of the three categories (or classes): "headlights”, “taillights”, and "other" classes of light sources. It will be understood that various kinds of classifiers may be used.
  • the descriptor according to various embodiments may be used with different classifiers and classification methods.
  • the AHC system may operate in real-time with limited hardware resources.
  • a high accuracy of light spot classification in the camera images for AHC may require using many (for example more than 30) different descriptors which may be provided in parallel as inputs to the CNN.
  • a higher number of used descriptors does not necessarily correspond to a better accuracy of the AHC classification.
  • the more descriptors are calculated the more computational power is needed.
  • the optimal sets of descriptors needs to be selected experimentally which is an exhaustive task. Comprehensive investigations performed with descriptors of different type show that when the descriptors are used separately in the classification task, an efficiency of the AHC function does not exceed 40-50 %, which is not sufficient according to safety requirements.
  • a descriptor is provided which, after being applied, leads to significant improvement of light spot classification. This fact was verified during experimental tests regarding the AHC function.
  • Using the descriptor according to various embodiments may provide low computational power demand, which may be useful for example in situations when the hardware resources are limited.
  • Using the descriptor according to various embodiments allows replacing or eliminating some number of other commonly used descriptors, while keeping or even improving the classification accuracy. Since the eliminated other descriptors may be more computationally complex, this may translate into a reduction of the computational effort.
  • Patent application WO2017093037 which is incorporated herein by reference in its entirety, may be used at the second stage of the overall procedure described above.
  • This patent application describes a method that allows to localize light spots in images with dark background. A similar situation appears at night, when the AHC functionality is active. Under such conditions, the general background is also dark and at the same time the light sources are visible as bright spots.
  • the descriptor according to various embodiments may be used in the procedure of the classification of light spots that is applied in the AHC functions, and may be applied, for example, at the stage of determination of at least one descriptor as described above.
  • the light spots in the image may represent objects of different type located in the vicinity of the moving vehicle.
  • the procedure may be divided into several steps, as described in more detail below.
  • the light spots in the I 3,n images may be centered.
  • the descriptor may receive centered spots, which may have been centered using a commonly used method.
  • the processed I 3,n images with the light spots may first be binarized with an appropriate threshold, wherein the threshold may be adaptively adjusted according to brightness distribution. This is illustrated in Fig. 1 for selected typical situations.
  • Fig. 1 shows an illustration 100 of various light spots 102, 106, 110, 114, 118, 122, 126, 130 and their binary representations 104, 108, 112, 116, 120, 124, 128, and 132 according to various embodiments.
  • binarization allows that subsequent steps of the method according to various embodiments may focus only on the contours of the spot.
  • the binarization may reduce the amount of data necessary to unambiguously classify the spot to one of the categories. After the binarization, only one bit is required to store a single pixel.
  • Fig. 1 presents several typical cases that may be encountered during the binarization.
  • Sub-diagrams (a) and (b) present more regular spots usually associated with vehicles' lights.
  • Sub-diagram (c) shows a situation in which a square spot is rotated to some degree.
  • the (d) - (h) sub-diagrams illustrate less regular spots. Such spots are usually associated with other light sources, however sometimes due to a fuzzy effect may also be provided by car lights.
  • the overall image may binarized and then stored in the memory.
  • binarization may be performed "on the fly” (OTF) over only selected pixels, which may simplify the overall operation.
  • OTF on the fly
  • only those pixels that are involved in the determination (for example counting operation) of the descriptor as described in the fourth step below may be binarized.
  • the binarization of an overall image requires j ⁇ k operations, each operation including a thresholding operation, where j and k are the sizes (width and height) of the I 3 images.
  • j and k are the sizes (width and height) of the I 3 images.
  • the binarization is performed only on those pixels in the I 3,n images, which are overlapped by particular directions as described in the third step below. Furthermore, in the OTF approach, the memory requirements may be reduced.
  • the plurality of directions along which the light pixels are counted for the descriptor may be selected.
  • the pixels in the respective lines may be referred to as target pixels.
  • the number of used directions may vary, for example depending on how exactly the spots need to be described and/or depending on how much computational power and/or memory space is available.
  • Fig. 2A to Fig. 2G show illustrations of various directions used in a descriptor according to various embodiments.
  • Fig. 2A to Fig. 2G illustrate in detail which pixels are taken into account in which case. While Fig. 2B to Fig. 2G only show one quadrant, the same indexing approach is used in the remaining three quadrants (not shown in Fig. 2B to Fig. 2G ).
  • Fig. 2A shows an illustration 200 of all directions up to BD16 descriptors (four quadrant view) according to various embodiments.
  • Fig. 2B shows an illustration 202 of selected directions for all up to BD32 descriptor (one quadrant view) according to various embodiments.
  • Fig. 2C shows an illustration 204 of selected directions for BD4 descriptor (one quadrant view) according to various embodiments.
  • the BD4 descriptor uses only four directions, two vertical directions (N (for north), S (for south)) and two horizontal directions (W (for west), E (for east)).
  • N and E directions which are located in or adjacent to the first quadrant are illustrated.
  • Fig. 2D shows an illustration 206 of additional directions used in BD8 descriptor (one quadrant view) according to various embodiments.
  • the BD8 descriptor uses the same directions as BD4, with additional four diagonal directions: NW, NE, SE, SW, which are illustrated in Fig. 2A and Fig. 2D .
  • Fig. 2E shows an illustration 208 of additional directions used in BD16 descriptor (one quadrant view) according to various embodiments.
  • the BD16 descriptor in the comparison with the BD8 one, additionally uses intermediate directions denoted as: NNE, ENE, ESE, SSE, SSW, WSW, WNW and NNW, as illustrated in Fig. 2A and Fig. 2E .
  • Fig. 2F and Fig. 2G show illustrations 210 and 212 of additional directions used in BD32 descriptor (one quadrant view) according to various embodiments.
  • an efficient way of indexing over the two-dimensional input matrix is provided.
  • the pixels to be considered in particular directions are selected based on Bresenham's algorithm, as illustrated in Fig. 2B to Fig. 2G . Indexing over the matrix of pixels in particular cases may be performed as presented below.
  • the light pixels along particular directions may be counted, starting from the center of the spots ( I 3 images), i.e. starting from the base pixel.
  • the counting may be performed using the Binary Coded Decimal (BCD).
  • BCD Binary Coded Decimal
  • BIN direct binary representation
  • the resulting code may have smaller memory usage, however, the information about the distribution of light pixels in particular directions may be lost.
  • the BIN representation may be more suitable for describing non convex objects in which discontinuities may appear in bright areas when looking in particular directions.
  • a dark pixel may be a pixel with brightness (or intensity) below a pre-determined threshold.
  • a dark pixel may be represented by a value of 0 in a binarized image.
  • a bright pixel may be a pixel with brightness (or intensity) above a pre-determined threshold.
  • a bright pixel may be represented by a value of 1 in a binarized image.
  • the counting operation may be performed so that it stops after reaching a first dark pixel in a given direction. This may allow roughly filtering out noise, which may be due to, for example, reflections of irregular surfaces, for example reflections on water. Such problems with reflections may result in a non-regular shape of the spot.
  • Fig. 3A shows an illustration 300 of an example of a non-convex spot.
  • a contour 302 of a blob (in other words: spot) is shown, and it is assumed that the area 304 inside the contour 302 is bright, and the area 306 outside the contour 302 is dark.
  • Dark pixels are illustrated by filled (in other words: black) circles, and bright pixels are illustrated by unfilled (in other words: white) circles in Fig. 3A .
  • the center pixel which may also be referred to as a base pixel
  • the sequence of bright and dark pixels along the pre-determined directions may be determined.
  • Fig. 3B to Fig. 3E show illustrations of different options of counting pixels of particular directions for the non-convex spot of Fig. 3A according to various embodiments.
  • Fig. 3B shows an illustration 308 of the BCD approach according to various embodiments, in which all bright pixels in the particular directions are counted.
  • the result for the NE direction is 3 (with a binary representation of 0011), as pixel 314 inside an object visible in the upper right corner, stuck to the regular part of the spot (i.e. inside the area 304 which includes bright pixels), is also taken into account.
  • the E direction four bright pixels (with a binary representation of 0100) are present; in the SE direction, four bright pixels (with a binary representation of 0100) are present; in the S direction, six bright pixels (with a binary representation of 0110) are present; in the SW direction, five bright pixels (with a binary representation of 0101) are present; in the W direction, five bright pixels (with a binary representation of 0101) are present; in the NW direction, six bright pixels (with a binary representation of 0110) are present; and in the N direction, six bright pixels (with a binary representation of 0110) are present.
  • the binary representations of the pixel count are indicated next to the respective direction in Fig. 3B .
  • Fig. 3C shows an illustration 310 of counting of the pixels with detection of a discontinuity (which may be referred to as BCD_CUT approach) according to various embodiments, in which the pixels are counted in a given direction (for example from the base pixel outwards, without counting the base pixel) until a dark pixel is encountered for the first time. The remaining pixels are then cut-off (in other words: not counted), independently on their values.
  • the first dark pixel in the NE direction is pixel 312, so that the bright pixels more outside than pixel 312 are not taken into account when counting.
  • the bright pixel 314 is not counted, and the count of bright pixels until the first dark pixel occurs in the NE direction is 2 (represented by binary value 0010).
  • the pixel counts in the other directions are identical to the pixel counts as described with reference to Fig. 3B , since in the other directions, no bright pixel occurs after the first dark pixel.
  • the binary representations of the pixel count are indicated next to the respective direction in Fig. 3C .
  • Fig. 3D shows an illustration 316 of a direct representation of pixels (which may be referred to as the BIN approach) according to various embodiments, in which the values of the binarized pixels (in other words: information indicating whether a pixel is a bright pixel or a dark pixel) is used as the information representing the values of the pixels in the respective set of target pixels (in other words: is used as the descriptor).
  • the innermost pixel closest to the base pixel is bright, so that the last digit of a binary number representing the binarized pixel values is 1.
  • the next pixel in the NE direction is also bright, so that the second last digit of the binary number is 1.
  • the binary representation for the NE direction is 0010011.
  • the binary representations of the binarized pixel values are indicated next to the respective direction in Fig. 3D .
  • Fig. 3E shows an illustration 318 of a direct representation of pixels with detection of a discontinuity (which may be referred to as BIN_CUT approach) according to various embodiments, in which a discontinuity (a first dark pixel) in particular directions (BIN_CUT case) is detected, and the bright pixels after the first dark pixel are treated as dark pixels irrespective of their actual brightness or intensity, similar to the approach illustrated in Fig. 3C , so that the actual bright pixel 314 is treated as a dark pixel in the binary representation of the NE direction, so that the binary representation is 00000011.
  • the binary representations in the other directions are identical to the binary representations as described with reference to Fig. 3D , since in the other directions, no bright pixel occurs after the first dark pixel.
  • the binary representations of the binarized pixel values are indicated next to the respective direction in Fig. 3E .
  • the determined information (for example the counted number of pixels), the descriptor, is stored in a memory.
  • the sizes of the I 3,n images may be small, for example not exceeding 32x32. This results in a maximum of 16 pixels in each direction. In this case, only several typical 32-bit integer variables may be sufficient to describe the spot.
  • the number of integer values equals the number of directions divided by 2, for example 2 for BD4, 4 for BD8, etc.
  • the number of directions that are used for the descriptor may have an impact on the accuracy of the method, like will be described below.
  • the number of directions may differ, as mentioned above and shown in Fig. 2A to Fig. 2G . Every increasing of the number of directions may allow to describe the light spot in more detail.
  • Each following descriptor i.e. each descriptor with additional directions that are considered) may allow distinguishing other details of the spot.
  • the binary descriptor with four directions may not allow distinguishing a circle from a square, and a rectangle from an ellipse.
  • the BD4 descriptor may allow distinguishing a general ellipse (i.e. an ellipse which is not a circle) from a circle, and a general rectangle (i.e. a rectangle which is not a square) from a square.
  • the BD descriptor also may not allow determining if the mentioned geometric figures are rotated. While rotating a square object, in each of the four directions there are equal numbers of pixels. However, while rotating the object, the number of pixels in each direction equally increases, reaching their maximum values for rotations by 45, 135, 225, 315 degrees. In case of rotating an ellipse or a rectangle objects, the BD4 descriptor may allow determining if the object is oriented vertically or horizontally. However, it may not be possible to determine the rotation angle.
  • the binary descriptor with eight directions uses the vertical, horizontal, as well as diagonal directions.
  • the BD8 descriptor may allow distinguish shapes such as circles, squares, rectangles, ellipses and other more irregular objects. In this case, it may also be possible to determine if a regular object is rotated.
  • the BD8 descriptor may be sufficient.
  • head vehicle lights may also be desired to distinguish head vehicle lights from tail vehicle lights.
  • the method after an additional adaptation may also be used in traffic sign recognition (TSR) functions to distinguish digits on traffic signs.
  • TSR traffic sign recognition
  • a number on a traffic sign (for example indicating a speed limit) may be localized and particular digits of this number may be treated as spots to be recognized.
  • the contrast between the digits and their background may be low.
  • the digits may look like spots with shapes that differ for the '0' and '5' cases.
  • the descriptor according to various embodiment may be used to distinguish between '0' and '5', and the output of the descriptor in case of '0' may be different from the output of the descriptor in case of '5'.
  • speed limit traffic signs e.g. 20 and 25 mph.
  • BD16 or BD32 descriptors may be required.
  • Fig. 4 shows a flow diagram 400 illustrating a method for object detection according to various embodiments.
  • an image may be acquired.
  • a pixel of the image may be determined as a base pixel.
  • coordinates of a plurality of sets of target pixels may be determined.
  • Each set of target pixels may include a plurality of pixels in a respective pre-determined relationship to the base pixel.
  • information representing values of the pixels in the respective set of target pixels may be determined for each of the sets of target pixels.
  • it may be determined whether a pre-determined object is shown in the image based on the determined information.
  • the pre-determined object may include or may be a headlight of a vehicle. According to various embodiments, the pre-determined object may include or may be a traffic sign.
  • each set of target pixels may include pixels along a line through the base pixel in the image.
  • the respective lines of the plurality of sets of target pixels may have pairwise different directions.
  • the image may be acquired based on determining an area of high intensity in an input image and determining the image as a crop of the input image around the area of high intensity.
  • the information representing values of the pixels in the respective set of target pixels may include or may be the values of the pixels in the respective set of target pixels.
  • the information representing values of the pixels in the respective set of target pixels may include or may be binarized values of the pixels in the respective set of target pixels.
  • the information representing values of the pixels in the respective set of target pixels may include or may be information of pixels along a pre-determined order of the pixels until a first pixel with a pre-determined property is present.
  • the information representing values of the pixels in the respective set of target pixels may include or may be information of pixels having a pre-determined property.
  • the information representing values of the pixels in the respective set of target pixels may include or may be an indicator for each of the pixels indicating whether the respective pixel has the pre-determined property.
  • Each of the steps 402, 404, 406, 408, 410 and the further steps described above may be performed by computer hardware components.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
EP20160077.2A 2020-02-28 2020-02-28 Verfahren und systeme zur objektdetektion Pending EP3872693A1 (de)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP20160077.2A EP3872693A1 (de) 2020-02-28 2020-02-28 Verfahren und systeme zur objektdetektion
US17/178,098 US11562575B2 (en) 2020-02-28 2021-02-17 Methods and systems for object detection
CN202110216813.4A CN113327252B (zh) 2020-02-28 2021-02-26 用于对象检测的方法和系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP20160077.2A EP3872693A1 (de) 2020-02-28 2020-02-28 Verfahren und systeme zur objektdetektion

Publications (1)

Publication Number Publication Date
EP3872693A1 true EP3872693A1 (de) 2021-09-01

Family

ID=69742939

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20160077.2A Pending EP3872693A1 (de) 2020-02-28 2020-02-28 Verfahren und systeme zur objektdetektion

Country Status (3)

Country Link
US (1) US11562575B2 (de)
EP (1) EP3872693A1 (de)
CN (1) CN113327252B (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4202860A1 (de) 2021-12-22 2023-06-28 Aptiv Technologies Limited Bestimmung der position des zentralen punktes von punktwolkendaten
EP4202837A1 (de) 2021-12-22 2023-06-28 Aptiv Technologies Limited Ein merkmal, das die form eines räumlich verteilten datensatzes beschreibt
EP4202853A1 (de) 2021-12-22 2023-06-28 Aptiv Technologies Limited Quasi-rotationsinvarianter formdeskriptor

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017093037A1 (en) 2015-11-30 2017-06-08 Delphi Technologies, Inc. Method for identification of candidate points as possible characteristic points of a calibration pattern within an image of the calibration pattern

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1837803A3 (de) * 2006-03-24 2008-05-14 MobilEye Technologies, Ltd. Vorderlicht-, Schlusslicht- und Straßenbeleuchtungsdetektion
US8456327B2 (en) * 2010-02-26 2013-06-04 Gentex Corporation Automatic vehicle equipment monitoring, warning, and control system
KR101848451B1 (ko) * 2013-08-19 2018-04-12 젠텍스 코포레이션 차량 후미등과 점멸 적색 정지등을 구별하기 위한 차량 촬상 시스템 및 방법
US9483706B2 (en) * 2015-01-08 2016-11-01 Linear Algebra Technologies Limited Hardware accelerator for histogram of gradients
US10255511B2 (en) * 2016-01-04 2019-04-09 Texas Instruments Incorporated Real time traffic sign recognition
JP6602743B2 (ja) * 2016-12-08 2019-11-06 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置および情報処理方法
WO2018167879A1 (ja) * 2017-03-15 2018-09-20 三菱電機株式会社 光量調整装置、光量調整方法及び光量調整プログラム
EP3547208A1 (de) * 2018-03-27 2019-10-02 Aptiv Technologies Limited Vorrichtung und verfahren zum clustern von lichtpunkten
EP3553694A1 (de) * 2018-04-12 2019-10-16 Aptiv Technologies Limited Abstandsschätzung von fahrzeugscheinwerfern
CN108764235B (zh) * 2018-05-23 2021-06-29 中国民用航空总局第二研究所 目标检测方法、设备及介质
EP3605384A1 (de) * 2018-08-03 2020-02-05 Aptiv Technologies Limited Vorrichtung und verfahren zur erkennung von fahrzeuglichtern auf einem bild
CN110450706B (zh) * 2019-08-22 2022-03-08 哈尔滨工业大学 一种自适应远光灯控制系统及图像处理算法

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017093037A1 (en) 2015-11-30 2017-06-08 Delphi Technologies, Inc. Method for identification of candidate points as possible characteristic points of a calibration pattern within an image of the calibration pattern

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LU YU ET AL: "Finger vein identification using polydirectional local line binary pattern", 2013 INTERNATIONAL CONFERENCE ON ICT CONVERGENCE (ICTC), IEEE, 14 October 2013 (2013-10-14), pages 61 - 65, XP032527231, DOI: 10.1109/ICTC.2013.6675307 *
SHOUYI YIN ET AL: "Fast Traffic Sign Recognition with a Rotation Invariant Binary Pattern Based Feature", SENSORS, vol. 15, no. 1, 19 January 2015 (2015-01-19), pages 2161 - 2180, XP055692241, DOI: 10.3390/s150102161 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4202860A1 (de) 2021-12-22 2023-06-28 Aptiv Technologies Limited Bestimmung der position des zentralen punktes von punktwolkendaten
EP4202837A1 (de) 2021-12-22 2023-06-28 Aptiv Technologies Limited Ein merkmal, das die form eines räumlich verteilten datensatzes beschreibt
EP4202853A1 (de) 2021-12-22 2023-06-28 Aptiv Technologies Limited Quasi-rotationsinvarianter formdeskriptor
US11699245B1 (en) 2021-12-22 2023-07-11 Aptiv Technologies Limited Feature describing the shape of spatially distributed data set
US11715222B1 (en) 2021-12-22 2023-08-01 Aptiv Technologies Limited Quasi-rotation-invariant shape descriptor

Also Published As

Publication number Publication date
US11562575B2 (en) 2023-01-24
US20210271907A1 (en) 2021-09-02
CN113327252A (zh) 2021-08-31
CN113327252B (zh) 2024-03-22

Similar Documents

Publication Publication Date Title
US11562575B2 (en) Methods and systems for object detection
US9773176B2 (en) Image processing apparatus and image processing method
CN111612781B (zh) 一种屏幕缺陷检测方法、装置及头戴显示设备
Ashtari et al. An Iranian license plate recognition system based on color features
Mu et al. Lane detection based on object segmentation and piecewise fitting
US7566851B2 (en) Headlight, taillight and streetlight detection
JP6817104B2 (ja) 付着物検出装置、付着物検出方法
JP4942510B2 (ja) 車両画像認識装置およびその方法
Kosaka et al. Vision-based nighttime vehicle detection using CenSurE and SVM
EP1449168A2 (de) Methode und system zur verbesserung der fahrsicherheit im auto durch verwendung von bildverbesserung
JP2011103070A (ja) 夜間車両検知装置
CN105303157A (zh) 延伸用于avm停止线检测的检测范围的算法
CN111046741A (zh) 车道线识别的方法和装置
CN103093229A (zh) 车标定位方法及装置
CN114841874B (zh) 一种图像处理方法、装置、设备及存储介质
CN111695374B (zh) 监控视角中斑马线区域的分割方法、系统、介质及设备
Sakagawa et al. Vision based nighttime vehicle detection using adaptive threshold and multi-class classification
JP7261006B2 (ja) 車外環境認識装置
Chen et al. A novel method for license plate localization
Patel et al. A novel approach for detecting number plate based on overlapping window and region clustering for Indian conditions
CN114298987A (zh) 一种反光条检测方法及装置
CN112102353B (zh) 运动目标的分类方法、装置、设备及存储介质
CN112001344A (zh) 机动车目标检测装置及方法
JP7155870B2 (ja) 車両検出装置と方法及び電子機器
CN110837772A (zh) 一种用于夜间交通监控的汽车前照灯识别方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220301

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230223

RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: APTIV TECHNOLOGIES LIMITED

TPAC Observations filed by third parties

Free format text: ORIGINAL CODE: EPIDOSNTIPA

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: APTIV TECHNOLOGIES AG