JP2018206373A - Method and device for classifying targets for vehicle - Google Patents

Abstract

To propose a method for classifying targets (110) for a vehicle (100).SOLUTION: The method includes at least one classification step, an extraction step, and a creation step. The classification step classifies a target (110) from a camera image (115), using an attribute sorter (120). The extraction step extracts at least one unique target feature allocated to the target (110) from the camera image (115). The creation step creates a unique sorter (130), using the unique target feature.SELECTED DRAWING: Figure 1

Description

  The proposal relates to a method or device as described in the preamble of the independent claims. The subject of this proposal is also a computer program.

  Object identification using camera systems is becoming increasingly important in many fields. Especially in an automobile environment, a camera system having a function for identifying an object such as a pedestrian is often required for various assist functions up to automatic driving. In this case, a corresponding object identification device assigns the identified object to the object type using the trained attribute classifier.

  Against this background, the proposal introduced here includes a method for classifying objects for vehicles, a device using this method, and finally a corresponding computer program as described in the main claim. Introduced. By means of the dependent claims, preferred embodiments and improvements of the device described in the independent claims are possible.

  The advantages that can be obtained with the proposed proposals are the addition of known attribute classifiers, and the creation of the unique classifiers introduced here, so that the object identification reaction time as well as the object tracking of the objects. The robustness of time is improved.

  A method for classifying objects for vehicles is proposed. The method includes at least one classifying step, extracting step and creating step. In the classifying step, one object is classified from the camera image using an attribute classifier. In the extracting step, at least one unique object feature assigned to the object is extracted from the camera image. In the creating step, a unique classifier is created using the unique object features.

  This method may be implemented, for example, in the control unit, for example in software or hardware or in a mixed format consisting of software and hardware. For example, the method may be implemented in an object identification device for a vehicle camera.

  With the method introduced here, the objects detected in the camera image can no longer be classified not only by the attribute classifier but also by the newly created unique classifier. This allows for a faster classification of objects and also classification of objects that have not been known to date or are difficult to identify.

  In the classifying step for using and / or updating the unique classifier, another object is classified and extracted from another camera image using an attribute classifier and a unique classifier, At least one other unique object feature assigned to another object may be extracted from another camera image and created in which the unique classifier may be adapted using the other unique object feature .

  In the extracting step, for example, one object feature representing at least one color and / or structure and / or shape of the attribute-classified object is extracted from the camera image. This type of object feature is not considered by the attribute classifier. This is because an object feature of this type generally does not apply to all object examples of one object type, but in a specific use case, for example, a reliable object of an identified object. This is because it can be used for tracking.

  In the creating step, the unique classifier may be stored in a circular storage device.

  In order to provide a camera image, the method includes a calculating step, in which the camera image is calculated using a peripheral detection device. The surrounding detection device may be a camera, for example a vehicle camera and / or a radar device, or may at least have these.

  In order to be able to adapt or update the attribute classifier, the method may, according to a preferred embodiment, comprise a step of memorizing, in which the step of classifying the object in the classifying step. When the classification accuracy for classification falls below the threshold, the camera image is stored. The step of memorizing may be performed before the step of extracting, in which case at least one object feature of an attribute-detected object, eg evaluated as inaccurate, is placed on the camera image. If done, the classification accuracy is below the threshold.

  In order to allow new training of the attribute classifier of the object identification device, the camera image may be transmitted to a training device located outside the object identification device in the memory step.

  The proposals introduced here further provide a device configured to carry out, control or execute the steps of the method variant introduced here with corresponding devices. According to this proposed example of a change in the shape of the apparatus, the problem based on the proposal can be solved quickly and effectively. This device may be part of an object identification device.

  For this purpose, the device comprises at least one computing device for processing signals or data, at least one storage device for storing signals or data, and for reading sensor signals of sensors or data or control At least one interface to the sensor or actuator to output signals to the actuator and / or at least one communication interface to read or output data embedded within the communication protocol. The arithmetic device may be, for example, a signal processor, a microcontroller, etc. In this case, the storage device may be a flash memory, an EPROM or a magnetic storage device. The communication interface may be configured to read or output data wirelessly and / or with wired guidance, in which case the communication interface capable of reading or outputting wired-directed data is The data can be read, for example, electrically or optically from a corresponding data transmission line or output to a corresponding data transmission line.

  A device may here be interpreted as an electrical device that processes sensor signals and outputs control signals and / or data signals based thereon. The device may have an interface that can be configured in hardware and / or software. In a hardware-based configuration, the interface may be part of a so-called system ASIC that contains the various functions of the device, for example. However, the interface may be a dedicated integrated circuit or at least partially made up of discrete elements. In a software-based configuration, the interface may be a software module provided next to another software module on the microcontroller, for example.

  According to a preferred embodiment, the device controls the classification of the object. To this end, the device can access sensor signals such as classification signals, extraction signals and creation signals. The control includes, for example, an attribute classifier configured to perform an attribute classification of the object identified in the camera image using an attribute classifier, at least one unique object feature assigned to the object Through an actuator, such as an extractor configured to extract the image from the camera image, at least a unique classifier configured to create a unique classifier using unique object features.

  Any one of the preceding embodiments can be stored on a machine-readable carrier or storage medium, such as a semiconductor storage device, a hard disk, or an optical storage device, particularly when the program product or program is executed on a computer or device. Also advantageous is a computer program or computer program product having program code used to implement, execute and / or control the steps of the method.

1 is a schematic view of a vehicle equipped with a device according to one embodiment for classifying objects. FIG. 1 is a schematic diagram of an apparatus according to one embodiment for classifying objects for a vehicle. FIG. 2 is a flowchart of a method according to one embodiment for classifying objects for a vehicle. 2 is a flowchart of a method according to one embodiment for classifying objects for a vehicle.

  Several embodiments of the proposal introduced here are shown in the drawings and are described in detail below.

  In the following description of the preferred embodiment proposed here, the same or similar reference numerals are used for components having similar action, which are shown in the various drawings, in which case these A repeated description of the components is omitted.

  FIG. 1 shows a schematic diagram of a vehicle 100 with an apparatus 105 according to one embodiment for classifying objects 110.

  The device 105 is configured to classify an object 110 that is optionally treated as a separate vehicle in this embodiment. For this purpose, the device 105 is configured to classify the object 110 obtained from the camera image 115 using the attribute classifier 120. According to this embodiment, the camera image 115 is read and provided by the periphery detection device 125 of the vehicle 100. Further, the device 105 is configured to extract from the camera image 115 at least one unique object feature assigned to the object 110. Further, the apparatus 105 is configured to create a unique classifier 130 using unique object features and to classify the object 110.

  The features of the device 105 described below are optional.

  According to one embodiment, the device 105 is configured to extract from the camera image 115 an object feature that represents at least one color and / or structure and / or shape of an attribute-classified object. Yes. In addition, the device 105 stores the unique classifier 130 created according to this embodiment in a circular storage device.

  Furthermore, the device 105 is configured to store the camera image 115 when the classification accuracy associated with the classification of the object 110 falls below a threshold during classification by the attribute classifier 120. For this purpose, the device 105 is configured to transmit the camera image 115 to a training device 132 arranged outside the device 105 and / or outside the object identification device having the device 105.

  Further, the device 105 is configured to provide or transmit the object 135 classified by the device 105 to the drive assist system 140 of the vehicle 100.

  According to this embodiment, the device 105 uses the attribute classifier 120 and the unique classifier 130 to classify another object 145 from another camera image 150 and to assign another object 145 assigned to this other object 145. Specific object features are extracted from another camera image 150 and are configured to adapt the unique classifier 130 using the other unique object features.

  FIG. 2 shows a schematic diagram of an apparatus 105 according to one embodiment for classifying objects for a vehicle. In this case, this may be the device 105 described in FIG. FIG. 2 shows a usage example of the attribute classifier 120 and the unique classifier 130.

  The attribute classifier 120 represents or contains at least one learned vehicle model 200 that has been previously learned in offline training using a number of example objects 205. Object 110, 145, which is an object or vehicle that is atypical or not yet known during the training time of offline training in camera image 115 and / or another camera image when used in road traffic Is displayed, the objects 110 and 145 are identified incorrectly and / or slowly by the attribute classifier 120 alone. The apparatus 105 introduced here extracts the main features of the objects 110, 145 in the form of unique object features 210, thereby calculating the unique classifier 130 or adapting the unique classifier 130. . The unique classifier 130, according to this embodiment, is stored in a storage device, in which case an additional application 220 is executed for each memory write 215 in the storage device.

  In other words, the apparatus 105 introduced here enables on-line learning of the classifiers 120, 130 for object detection of the camera system or for the camera system. For this purpose, the device 105 is, according to an alternative embodiment, supported on the inside or outside of the periphery detection device 125 having at least one camera system.

  In this case, the problem with this device 105 is, for example, an uncertain and / or unclear object for object detection, for example in a camera system incorporated in the vehicle to identify the object 110,145. The online update of the classifiers 120 and 130 for identification is performed in accordance with the object classification performed “reliably” by tracking, validation over time, etc. See also FIG. 4 for this.

  Unlike known systems, the apparatus 105 introduced here can also better identify example objects 205 below numerical values in a suitable format. In addition, the trained attribute classifier 120 in today's systems is attributed for identification of a given object type to ensure proper object identification that is globally accepted in all situations. ing. For such attribute identification, corresponding features that are generally applicable to all example objects 205 are used, i.e. unique features such as color and distinct structure, such as license plates, in this case, Rejected as an identifying feature. As a result, on the other hand, object identification is generally performed for a relatively long time. This is because classifier detection needs to be validated through validation to obtain a low error rate. On the other hand, for example, the target object 205 that is below a numerical value such as a small-volume production vehicle or a new generation vehicle is particularly difficult to detect. Inaccurate object detection, however, the classification of objects that are delayed in time by tracking / validation does not give rise to evaluations or fits in today's systems, but inadequate detection of this object. Remains persistently. However, the online training of the classifiers 120, 130 introduced here improves the quality level or the identification rate.

  For this purpose, another classifier, i.e. a unique classifier 130, is preferably implemented in parallel in the already existing attribute classifier 120 in the device 105. The attribute classifier 120 identifies essentially all forms of the desired object type under the conditions associated with the identification of new / atypical object forms and reaction times. For objects detected by this attribute classifier 120 and classified via temporal validation (tracking) and verification, unique features such as color and structure for this object are displayed in the video image. It can be extracted from a possible camera image 115 and used to create / adapt another unique classifier 130. Thereby, on the other hand, stable object tracking is possible even in the case of image obstruction by a window wiper during rain or partial occlusion of an object. On the other hand, objects can be detected and classified more quickly, for example after being completely occluded by another vehicle or running at the head after a curve. Thus, features that cannot be used for the attribute classifier 120 because they prevent general identification of all possible object forms can be used for the unique classifier 130.

  However, even if the attribute classifier 120 is provided with, for example, a color as a training feature, if there is no blue vehicle in the example data, here the example object 205, the blue vehicle is no longer identified.

  By adding such a unique classifier 130 in parallel with the attribute classifier 120 in the local vicinity of the vehicle, the advantages associated with, for example, the reaction time of identification and the robustness of object tracking are provided.

  In this case, the basic object detection performed via the attribute classifier 120 is further necessary to ensure that the requirements imposed on the quality of object identification at the time of product shipment are met.

  In the online training phase, the input of the unique classifier 130 is connected to the input of the attribute classifier 120, and the output of the unique classifier 130 is connected to the output of validation, validation and integration. See also FIG. 4 for this. As initial data for the unique classifier 130, a unique feature is extracted from the image range of the camera images 115, 145 in which the object is identified by video. This unique object feature 210 is designed to accurately describe the object form in the form of height, width, structural information, color, etc., and exclusively identify this object accurately.

  According to this embodiment, the unique classifier 130 is designed to classify a plurality of object forms. Alternatively, there may be a predetermined number of a plurality of cases of the unique classifier 130 that are satisfied according to the driving situation. In this case, the management of the classifier 130 is realized via an annular storage device, in which the oldest writing is rejected when a new object form is to be stored.

  As already described in FIG. 1, the device 105 further allows an optional correction of the attribute classifier 120. In this case, in addition to the adaptation to the provided object form, an evaluation of the attribute classifier 120 relating to the identification time and robustness of the respective object form may be performed. Each object 110, which has been slowly or uncertainly identified by the attribute classifier 120 and has been previously classified by a selective identification method, such as temporal verification or eigenmotion identification as detailed in FIG. For 145, the corresponding image data of the camera image 115 is filed in a separate storage device. This separate storage device is shown in FIG. 1 as a training device or part of a training device, which may be part of the device 105 in alternative embodiments. Example object form data that has been poorly identified or identified by the latest version of the attribute classifier 120 is available for online connection and / or use for new training of the attribute classifier 120. Or it may be filed centrally for each factory.

  FIG. 3 shows a flowchart of a method 300 for classifying objects for a vehicle according to one embodiment. In this case, this may be a method 300 that can be performed by one of the devices described in the preceding figures.

  The method 300 includes a classification step 305, an extraction step 310, and a creation step 315. In a classifying step 305, the object is classified from the camera image using an attribute classifier. In an extracting step 310, at least one unique object feature assigned to the object is extracted from the camera image. In a creating step 315, a unique classifier is created using the unique object features.

  The following embodiments of method 300 are optional.

  In step 305 of classifying, additionally or selectively according to alternative embodiments, another object is classified from another camera image using an attribute classifier and a unique classifier, in this case extracting. In step 310, at least one other unique object feature assigned to another object is extracted and created from another camera image, and in step 315, the unique classifier uses another unique object feature. To be adapted.

  In an extracting step 310, object features representing at least one color and / or structure and / or shape of the attribute-classified object are extracted from the camera image according to this embodiment.

  In step 315 of creating, the unique classifier is stored in the annular storage.

  Further, according to this embodiment, the method 300 optionally includes a step 320 for calculating and a step 325 for storing.

  In the calculating step 320, the camera image is calculated using the periphery detection device.

  In the memory step 325, when the classification accuracy related to the classification of the object falls below the threshold value in the classification step 305, the camera image is stored. According to this embodiment, in step 325 of storing, the camera image is transmitted to a training device arranged outside the object identification device.

  The method steps introduced herein may be repeated and may be performed in a sequence different from the sequence.

  FIG. 4 shows a flowchart of a method 300 for classifying objects according to one embodiment. This may be the method 300 described in FIG. 3 with additional steps.

  The method 300, according to this embodiment, allows a wide range of object detection and for this purpose has a block 400 to detect and a block 405 to verify. The detecting block 400 has a so-called appearance-based method 410 which comprises a generating step 415, a combining step 420 and a classification step 305 as already described in FIG. The detecting block 400 additionally comprises a so-called discrepancy-based method 425 and / or a so-called motion-based method 430 for hypothesis generation according to alternative embodiments.

  The verifying block 405 has a validation step 435 and a time integration step 440.

  The creating step 315 is located between the detecting step 400 and the verifying block 405. Between the verifying block 405 and the creating step 315, the method 300 further includes another verifying step 445.

  At block 400 to detect, a stable group of features describing the identifiable object is searched. In an appearance-based method 410, a characteristic of an object type, here a vehicle, is detected based on a training data set. According to this embodiment, vehicle features are filtered by a Gabor filter “Gabor-Filter” in an appearance-based method 410 and an attribute classifier is classified by a support vector machine “Support Vector Machine”. In this case, in a generating step 415, a characteristic feature, here a Haar feature “Haar-Merkmale”, is generated from the training image. In the combining step 420, the feature collection is combined with the full description of the object type in the data bank. In order to allow a feature vector to be clearly assigned to one object type, in step 305 the attribute classifier, which is the AdaBoost algorithm “AdaBoost-Algorithmus” according to this embodiment, takes time. Trained and / or a probability distribution of features is modeled.

  At verify block 405, the wrong and / or incorrect object hypothesis obtained from detect block 400 is validated and improved by time integration. For this purpose, in a validation step 435, the parameterized model and / or template is compared with the image data, and according to this embodiment a 3D grid model for the vehicle is created, which 3D grid model. Is projected onto the image plane to produce a contour pattern. This pattern is searched for in a grayscale image divided into scene segments. In addition or alternatively, at least one license plate, light and / or window glass is used. At time integration step 440, time integration is performed. In this case, physical objects and subjects that are not alive are identified. Non-living physical objects are in this case identified using a narrative by extrapolation based on physical laws, and the subject is identified by confirmation of intention, behavior and / or alternative behavior. Thus, for example, pedestrians, bicycle riders, passenger cars, trucks, etc. are identified. Further, in a time integration step 440, the motion model of the object hypothesis is adapted, an information collection of behavior models is implemented, and / or adjustments with surrounding models are performed, for example, inferred travel lane lengthening. The validation of the existing shape and the vehicle motion is performed, for example, the validation of the vertical foot of the object and the road geometry.

  Between the creating step 315 and another verifying step 445 and / or verifying block 405, a training step 450 is arranged, in which the training is performed. In addition, a validity query 455 is performed between the creating step 315 and the verifying block 405.

  If an example has an “and / or” connection between a first feature and a second feature, this means that this example is a second feature according to one embodiment. And should be read as having only the first feature or only the second feature according to another embodiment.

DESCRIPTION OF SYMBOLS 100 Vehicle 105 Device 110 Object 115 Camera image 120 Attribute classifier 125 Perimeter detection device 130 Eigen classifier 132 Training device 135 Object 140 Drive assist system 145 Another object 150 Another camera image 200 Vehicle model 205 Object example 210 Object Feature 215 Write Memory 220 Application 300 Method 305 Classify Step 310 Extract Step 315 Create Step 320 Calculate Step 325 Memory Step 400 Detect Block 405 Verify Block 410 Appearance-Based Method 415 Generate Step 420 Combine Step 425 Disagreement based method 430 Motion based method 435 Validating step 440 Time integrating step 445 Another verification step 450 Training step 455 Validation

Claims (10)

A method (300) for classifying an object (110) for a vehicle (100), the method (300) comprising at least:
Classifying an object (110) from a camera image (115) using an attribute classifier (120) (305);
Extracting from the camera image (115) at least one unique object feature (210) assigned to the object (110);
Creating (315) a unique classifier (130) using the unique object features (210);
A method (300) for classifying an object (110) for a vehicle (100) having:   In the classification step (305), another object (145) is classified and extracted from another camera image (150) using the attribute classifier (120) and the unique classifier (130). In step (310), at least one other unique object feature (210) assigned to said another object (145) is extracted from said another camera image (150) and created (315) The method (300) of claim 1, wherein the unique classifier (130) is adapted using the other unique object feature (210).   One object feature representing at least one color and / or structure and / or shape of the object (110) categorized from the camera image (115) in the extracting step (310) The method (300) of claim 1 or 2, wherein (210) is extracted.   The method (300) according to any one of the preceding claims, wherein in the creating step (315), the unique classifier (130) is stored in a circular storage device.   The method (300) according to any of the preceding claims, comprising the step (320) of calculating the camera image (115) using a peripheral detection device (125).   6. The method of claim 1, further comprising a step (325) of storing the camera image (115) when the classification accuracy regarding the classification of the object (110) falls below a threshold value in the classification step (305). The method (300) of any one of the above.   The method (300) of claim 6, wherein the memory step (325) transmits the camera image (115) to a training device (132) located outside the object identification device.   An apparatus (105) designed for performing and / or controlling the steps of the method (300) according to any one of claims 1 to 7 in corresponding units.   Computer program designed to carry out the method (300) according to any one of claims 1 to 7.   A machine-readable storage medium in which the computer program according to claim 9 is stored.

Images