DE102013109604A1 - Camera-based object recognition technology - Google Patents

Abstract

The disclosure relates to a camera-based object recognition technique, which comprises at least one object recognition method and a system (1) for carrying out the method. A recognition process (RP) is supplied with images (Pi, Pi + 1, Pi + 2) from the environment of a vehicle (3) taken by at least one camera (K1-K4) and having a known sequence. A first process section (PS1) recognizes macrostructures (Y) in individual images (Pi), in particular according to the HOG recognition method. A second process section (PS2) recognizes microstructures (X) and in particular carries out a tracking of recognized microstructures (dX) in the sequence of at least two images (Pi, Pi + 1). The first and second process sections (PS1, PS2) are executed in parallel or overlapping. Information about the detected and in particular tracked microstructures (X, dX) is fed to the first process section (PS1) and a relevance evaluation is carried out for image areas (Lok1, Lok2) which are detected by the first process section (PS1) for detection of macroscopic errors. Structures and, if necessary, objects are to be processed prioritized.

Description

The invention relates to a camera-based object recognition technology for detecting objects in the surroundings of a vehicle.

From the JP 2011 253 214 A is a device for detecting pedestrians known.

The previously known object recognition methods and object recognition devices are not optimally designed.

It is an object of the present invention to provide a more efficient camera-based object recognition technology with which the computational effort for the object recognition can be reduced and / or the processing speed for the object recognition can be increased. The technology includes at least one object recognition method and an object recognition system.

The present disclosure relates to a camera-based object recognition method in which images of the surroundings of a vehicle are periodically supplied to a recognition process. The images are taken by one or more cameras arranged on the vehicle and within the images structures are to be recognized, which can be assigned to objects. The recognition process comprises at least a first process section for recognizing macro-structures within individual images and a second process section for recognizing micro-structures within a sequence of at least two images.

The recognition of macro structures usually requires a high computational effort. It is executed based on a comparison of image areas with previously known structure patterns and / or object patterns. An object pattern can be composed of several structure patterns. For the recognition of the macrostructures, various process parameters can be influenced in order to carry out rough or detailed recognition and / or to influence the number of structures / objects to be recognized. The larger the image areas to be processed and the object patterns or structure patterns to be included in the comparison, or the larger their number, the higher the computational effort becomes.

The recognition of microstructures can be carried out with a comparatively low computational effort. It can involve two main stages, namely the (static) recognition of microstructures in a single image and the (dynamic) tracking of recognized microstructures in a sequence of at least two images.

According to the present disclosure, it is provided that result information from the second process section is fed to the first process section via recognized and in particular tracked microstructures from the second process section. On the basis of the recognized and, in particular, tracking microstructures in the sequence of images, a relevance evaluation is made for local image areas in these images. This relevance evaluation scales the process for recognizing marrow structures carried out in the first process section. Scaling can be done by influencing various process parameters. The scaling can in particular relate to the selection of the local image areas in the individual images, which are processed prioritized for the recognition of marko structures. The scaling may also relate to a grid size (coarse / fine) for the recognition of macrostructures. It may alternatively or additionally relate to the number and / or size and / or detailing of the structural patterns and / or object patterns to be used in the first process section. In addition, any other possibilities for scaling the recognition process in the first process section are possible.

In particular, the recognition of microstructures in the second process section comprises the detection of dynamic changes between two subsequent images. These changes are also called optical flow. Such dynamic changes may preferably be shifts in microstructures tracked between two sequential images.

The relevance evaluation can be made on the basis of different criteria, wherein the selection of the criteria can be dependent on the type of structures and / or objects to be recognized. For the recognition of pedestrians or other mobile road users, a relevance score may preferably be based on the local distribution, i. H. in particular the local arrangement density, and / or the local orientation of recognized and in particular tracked microstructures.

In particular, those local image areas with a high relevance can be evaluated, in which a high density of detected, in particular of tracked, microstructures has been established. Alternatively or additionally, an evaluation with a higher relevance can be made for those areas in which the orientation of the dynamic changes of the microstructures corresponds to a particular comparison direction. This comparison direction can be determined as a function of the structures / objects to be recognized. For the detection of pedestrians or other road users can be an area in which the orientation corresponds to the vehicle indicative comparison direction, be rated with higher relevance. The comparison direction can be determined separately depending on the arrangement of the camera on the vehicle, the current proper motion of the vehicle and for each image area to be checked.

The macro-structures and / or objects recognized in the recognition process as well as the recognized and in particular traced micro-structures can be made available to further applications. In these applications, other detection methods can be carried out in parallel or downstream, for example to increase driving safety. The assignment of the recognized structures to objects can also be carried out in the downstream applications or supplemented there. The downstream applications or processing may also access recognition data from other signal processing and perform a so-called sensor data fusion. Such other signal processing may, for example, process data from radar sensors, lidar sensors or from navigation databases.

In the subclaims further advantageous embodiments of the disclosure are given.

The invention is illustrated by way of example and schematically in the drawings. Show it:

1 FIG. 4 is a plan view of a vehicle with four cameras and a controller in which an object recognition method according to the present disclosure is performed; FIG.

2 3 is a schematic block diagram illustrating the flow of information in an object recognition method according to the present disclosure;

3 FIG. 3 is a flowchart illustrating, by way of example, processing in a second process section; FIG.

4 and 5 : Exemplary representations of a sequence of camera images and associated processing results from the first and second process sections;

6 FIG. 3 is a flowchart illustrating, by way of example, processing in a first process section.

The present disclosure relates to an object recognition technology. The technology comprises at least one object recognition system ( 1 ) for detecting objects (O1 to O4) in the vicinity of a vehicle ( 3 ) as well as an object recognition method.

An object recognition system ( 1 ) is in 1 exemplified. The system ( 1 ) has at least one camera (K) and an electronic control unit (hereinafter referred to as ECU) ( 2 ) on. At the in 1 represented vehicle ( 3 ) four cameras (K1 to K4) are arranged, which surround the vehicle ( 3 ) in different directions. To the vehicle ( 3 ) are four examples of foreign objects (O1 to O4). The foreign object (O1) is a pedestrian with a stroller, the object (O2) is another pedestrian, the object (O3) is a street lamp and the object (O4) is a building.

The ECU ( 2 ) can be configured arbitrarily. It preferably includes at least a CPU, a volatile memory (RAM) and a nonvolatile memory (ROM). In the ECU ( 2 ), in particular in the ROM, a program is stored, which is used for the execution of the object recognition method. The processing steps and process sections described later can preferably be implemented by corresponding portions of the program. Alternatively or additionally, individual or all processing steps and / or process sections may be executed by separate hardware. This also includes the possibility of performing individual processing steps and / or process sections on other ECUs and reporting their results to the ECU ( 2 ) to be led back.

• • Blind Spot Detection (Überwachung des toten Winkels)
• • Parking Assist (Einparkhilfe)
• • Cross Traffic Alert (Warnung über kreuzende Verkehrsteilnehmer)
• • Forward Collision Warning, Rear Collision Warning (Warnsysteme zur Vermeidung von Auffahrunfällen)
• • Pedestrian Collision Warning, Pedestrian Collision Detection (Fußgängerkollisionswarnung und Fußgängerkollisionserkennung)
• • Lane Departure Warning, Lane Change Assist (Spurverlassenswarnung, Spurwechselassistenz)

The object recognition technique disclosed herein may be used for various purposes, particularly for methods and systems for increasing traffic safety. In other words, the results and output information of the object recognition method disclosed herein or the output information of the object recognition system disclosed herein may be supplied to other applications or processing. Such applications or processing include in particular:

• • Blind Spot Detection
• • Parking Assist
• • Cross Traffic Alert
• • Forward Collision Warning, Rear Collision Warning (warning systems to avoid rear-end collisions)
• • Pedestrian Collision Warning, Pedestrian Collision Detection (Pedestrian Collision Warning and Pedestrian Collision Detection)
• • Lane Departure Warning, Lane Change Assist (lane departure warning, lane change assistance)

In 2 For example, a camera-based object recognition method according to the present disclosure is exemplified. When Input information is shown on the left side three camera images (Pi, Pi + 1, Pi + 2), which are taken from a camera (K, K1 to K4) of the vehicle and the detection process (RP) are supplied. Within the recognition process (RP), at least a first process section (PS1) and a second process section (PS2) are executed. The first process section (PS1) serves to detect macro-structures within individual images. The second process section (PS2) serves to detect and track microstructures in a sequence of at least two images. In other words, the first process section (PS1) serves to detect static image features that can be assigned to objects (O1-O4), and the second process section (PS2) serves to detect the optical flow in a sequence of at least two images.

There is a known relationship between the images (Pi, Pi + 1, Pi + 2) processed in the first and second process sections (PS1, PS2). In particular, for each of the images (Pi, Pi + 1, Pi + 2) its origin from a particular camera (K1-K4) and the time of recording known. Through the process sections (PS1, PS2), successive images can preferably be processed by a single camera. Alternatively, however, it is also possible to process images and sequences of images that originate from different cameras. This is useful, for example, when two or more cameras record the same or overlapping surrounding areas of the vehicle.

The recognition results from the recognition process (RP), in particular the recognized macrostructures (Y) from the first process section (PS1) and the recognized and in particular traced microstructures from the second process section (PS2) can be used by the downstream applications or processes (App1 to App9) are supplied.

The object recognition technique disclosed here will be explained below by way of example for the recognition of pedestrians in the surroundings of a vehicle. The processing steps, process sections and components, which execute the processing for this purpose, can be modified and adapted for the recognition of other objects as well as for the further processing of the recognized structures in other applications.

First, an example of captured images (Pi, Pi + 1, Pi + 2) from the front environment of a vehicle ( 3 ) explained. Next, two examples of processes performed in the first process section (PS1) and the second process section (PS2) will be explained, to then clarify the results of such processings on the images.

In 4 In the top line are three original pictures (Pi, Pi + 1, Pi + 2) taken by a camera (K1). The camera (K1) is at the front of a vehicle ( 3 ) (cf. 1 ) and substantially in the axial direction of the vehicle ( 3 ) so that they form a partial area or front area of the surroundings in front of the vehicle ( 3 ).

In the mentioned frames (Pi, Pi + 1, Pi + 2), which are in the top line of 4 are shown already identified and in particular tracked micro-structures are located. The microstructures (X) detected in the second process section (PS2) are shown as bright dots. The microstructures tracked in the sequence of two images, ie the displacements (dX) of the traced microstructures from a first image (Pi) to a subsequent image (Pi + 1) are shown as red lines. For better visualization, the recognized microstructures (X) in the second and third rows of 4 for the image (Pi) additionally marked as points and the displacements (dX) as lines.

The microstructures can be recognized and tracked in any way. In practice, various methods for detecting microstructures and for tracking microstructures have been proposed, which may be used alternatively or in combination. In the recognition of micro-structures, preferably small image areas (micro-image areas) of, for example, 3 × 3 pixels to 6 × 6 pixels or 10 × 10 pixels are checked for particularly characteristic image patterns. Such image patterns can be present, for example, as crossing points of contrast transitions, ie, for example, as intersections of two or more contrast lines. The angles of the respective contrast transitions or contrast lines within a micro-image area can be calculated quickly. Once a microstructure (X) has been detected in an image (Pi), its type and position can be registered. When a microstructure (X) of the same type is recognized in a sequence image (Pi + 1) within a predetermined range around the position of the previously registered microstructure (X), these two microstructures can be associated with each other. The assignment of two microstructures (X) of the same type in a sequence of images (Pi, Pi + 1) with detection of the change in position is referred to as tracking. The change of the position from the first image (Pi) to the next image (Pi + 1) can be recognized as a shift (dX). A variety of such shifts (dX) for a variety of tracked microstructures forms the optical flow. The optical flow can be considered Vector field in which for each shift (dX) the starting position, direction and length is deposited. Alternatively, another representation of the displacements (dX) may be selected.

In the second process section (PS2), multi-level processing of the image material is preferably carried out. The individual stages of processing may depend on the type and size of images taken (Pi, Pi + 1, Pi + 2) as well as the desired recognition result. 3 shows an exemplary procedure for the detection of microstructures in the second process section (PS2), which is suitable for the detection of pedestrians.

In the 3 The processing shown can be repeated and executed for each frame (Pi, Pi + 1, Pi + 2) in the sequence of pictures from a camera (K). Alternatively, the processing may be performed only for every second or third image or process the images from different cameras, if their spatial relationship is known.

At step (S10), a next image to be processed for recognition is read. The image usually consists of a plurality of pixels that contain brightness information and, if necessary, color information. For each pixel, one or more brightness and possibly color values can be stored separately.

At step (S20), pre-processing of the image is performed. The pre-processing can serve to optimize the image material for the later processing steps, in particular the compensation of exposure ratios, the conversion into a black-and-white image or the adaptation of the image noise.

At the in 3 In the example shown in the preprocessing, a Gaussian conversion is carried out (also called Gaussian filtering). The Gaussian transformation changes the contrast of the image material. In other words, blurring or blurring is performed. The application of Gaussian image processing can be carried out in an adapted manner for specific image areas, specific image colors, specific brightness areas or a combination of the aforementioned areas. The Gaussian conversion can be used to eliminate or smooth contrast changes that are based on local reflections or other unwanted optical effects, so that they do not influence or less affect the recognition of the microstructures to be assigned to the objects.

In the subsequent step (S30), the recognition of micro-structures is carried out in each case in one frame of the sequence. The recognition of microstructures can be done in any way. In the present example, so-called Harris-Corner processing is performed. Alternatively, another known processing may be used. At the end of step (S30), a listing of the microstructures recognized in the processed image with the respective type and the respective position is preferably present.

In the subsequent step (S40), the optical flow of detected micro-structures in the sequence of the previous image (Pi) to the present image (Pi + 1) is determined. The microstructures (X) registered for the previous image (Pi) and the microstructures (X) recognized for the present image (Pi + 1) are compared with each other, whereby matching microstructures (X) are traced. Preferably, for each tracked microstructure (X), a shift (dX) from the previous image (Pi) to the present one (Pi + 1) is registered. The registration takes place, for example, in a vector field.

In the processing step (S50), the local distribution and / or the respective displacement direction is analyzed for the traced microstructures. This can be done for example with known mathematical analysis methods on the basis of the aforementioned vector field. Alternatively, another method of analysis may be used. In the analysis, local image areas (Lok1, Lok2) can be determined with an increased density of recognized and in particular traced microstructures. Such image areas (Lok1, Lok2) are in the pictures in the bottom line of 4 exemplified.

Alternatively or additionally, local image regions (Lok1, Lok2) can be detected in which there is a specific orientation of the displacements (dX) and / or in which there are several different orientations of the displacements (dX). For the detection of pedestrians, for example, areas may be important in which the orientation of a shift (dX) points in a comparison direction in which a collision with the vehicle movement could threaten. Alternatively or additionally, those regions may be important in which there are many different orientations of several shifts (dX). This may indicate a majority of pedestrians or other road users.

In step (S60), information about the detected and, in particular, tracked microstructures, in particular the analysis result, is transmitted to the first process section (PS1). Based on this information, a relevance score is performed to determine the scaling of the one performed in the first process section (PS1) Make recognition process. The relevance assessment can be done in any convenient way. It can be executed as a processing step in the first process section (PS1) or in the second process section (PS2) or in a separate process.

In the final step (S70), information about recognized and / or tracked microstructures, i. in particular information about the optical flow, to subsequent processing or applications (App1 to App9) transmitted. Then the processing ends once. It can be executed again for the next picture (Pi + 2).

For the object recognition method according to the present disclosure, another configuration of the second process section (PS2) may be selected other than that in FIG 3 shown.

In the object recognition technique according to the present disclosure, the first process section (PS1) and the second process section (PS2) are preferably performed in parallel or temporally overlapping. The two process sections (PS1, PS2) can process the same individual images (Pi, Pi + 1, Pi + 2). Alternatively, the process sections (PS1, PS2) can partially process different images. For example, one of the process sections (PS1, PS2) can be supplied with only every second or third image, while the other process section (PS2, PS1) processes all the images.

Hereinafter, an exemplary processing for recognizing macrostructures in the first process section (PS1) will be explained. In the process section (PS1), macrostructures (Y) can basically be recognized in any way. In practice, various methods have been proposed for recognizing macrostructures in frames that are suitable for the detection of different types of objects (O1-O4). These methods can be used individually or in combination, and the selection of the methods may depend on the type of objects to be recognized.

In 6 For example, a preferred processing for recognizing macrostructures is exemplified in the first process section (PS1) suitable for the recognition of pedestrians. In a first step (S110), the next image to be processed (Pi + 1) is read. It may have the same shape and size as that in step (S10) according to FIG 3 read picture.

In the subsequent step (S120), pre-processing of the image material may be performed. A pre-processing can also be chosen here in any way and serve the optimization of the image material for further detection steps. In particular, a size change (size scaling), a contrast adjustment, an edge detection, an influence of the image noise or similar processing can be carried out.

In step (S130), a relevance score for local image areas of the currently processed frame (Pi + 1) is executed. The relevance assessment can be done in different ways. In particular, it can depend on the respective macro-structures and / or objects to be recognized. The relevance evaluation is carried out on the basis of the information supplied by the second process section (PS2) via the microstructures recognized there and in particular tracked there.

For the detection of pedestrians, for example, it makes sense to evaluate those image areas (Lok1, Lok2) with a higher relevance, in which there is a high density of recognized and in particular of tracked microstructures (X, dX). Alternatively or additionally, for the pedestrian recognition, those local image areas with a higher relevance are to be evaluated, in which the orientation of the displacements (dX) of tracked microstructures (X) in one direction to the vehicle (FIG. 3 ), that is, for example, is aligned for the images of the front camera (K1) in a direction towards the center of the image. Furthermore, such local image areas can be evaluated with a higher relevance, in which several different orientations for shifts (dX) of traced microstructures (X) occur.

The recognition of macro-structures may preferably be performed by a comparison process in which a comparison of image information of a selected image area with one or more structural patterns is performed. This comparison process can preferably be carried out with a so-called sliding search window. The search window has a predetermined and optionally adjustable size and is moved iteratively over the image areas that are evaluated as being relevant. Each time the comparison process is performed, the current content of the search window is taken into account for comparison, that is, the image information (pixels) contained therein. The size and / or resolution of the structural patterns for which a comparison is made is preferably selectable and may be determined depending on the relevance score. In other words, a different detailing for the comparison process can be selected for different image areas (Lok1, Lok2) depending on the respective relevance score. Thus, for areas with the highest relevance, a high number and a high degree of detail of structure patterns can be used. Alternatively or In addition, an offset range of the sliding search window can be selected to be very fine from one processing to the next. As a result, a fine and accurate detection is achieved.

If the image areas evaluated with high relevance have been processed, optionally in step (S150) a further recognition of macrostructures in local image areas having a lower relevance can be carried out. The execution of step (S150) may also be made dependent on the relevance score. For these other areas with a lower relevance, the recognition can be carried out, for example, roughly and with a smaller number of structure patterns. Depending on the availability of free computing time, the execution of step (S150) may be repeated with higher levels of detail, if necessary.

In a final step (S160), the information about detected macrostructures generated in the preceding steps (S140, S150) can be transmitted to subsequent processes or applications (App1 to App9). Then, the processing is ended to be executed again for the next picture, if necessary.

The recognition of objects, in particular the assignment of recognized macro structures to objects, can take place in one of the subsequent applications (App1-App9) or in a further processing step within the recognition process (RP). Alternatively, the detection can be done as part of the first process section (PS1) or the second process section (PS2).

The comparison of image contents, in particular the image contents of the sliding search window, with structural patterns can be carried out in any desired manner. Various comparative methods are known which can be used individually or in combination. For the detection of pedestrians, it has proven advantageous to use structure patterns in the form of histograms of oriented gradients (HOG). The associated recognition or comparison method is referred to below as the HOG recognition method. The HOG method is for example from the publication Dalal, N., Triggs, B .: "Histograms of Oriented Gradients for Human Detection", IEEE Computer Vision and Pattern Recognition, Vol. 1, pp. 886-893, 2005 or the publication Harris, C., Stephens, M .: "A combined corner and edge detector", proceedings often he 4th Alvey Vision Conference, pp. 147-151, 1988 known.

In the HOG recognition method, a local image area to be compared is divided into cells. Each of the cells has a predetermined size of, for example, 8x8 pixels. Within each cell, a gradient of the brightness distribution is calculated. This gradient can be given in predetermined angular ranges, for example in 8 or 18 ranges for covering the angles from 0 ° to 360 °. The gradient indicates in which direction the greatest change in brightness is calculated. The gradient may also reflect the intensity of the brightness change.

For each of the cells defined in the image area, a histogram of the gradients is then created and stored. In other words, the detected gradients are ordered according to their direction and counted in groups. Each group corresponds to one of the given angular ranges.

A texture pattern may comprise a macroblock of a plurality of such cells for which characteristic distributions of gradients are specified. An object pattern may include a plurality of texture patterns. The number of structure patterns to be included in the comparison in the first process section and / or the resolution or size of the search windows and / or structure patterns to be used in the first process section may be determined as a function of the relevance evaluation.

In the 4 and 5 For example, recognition results of an object recognition method according to the present disclosure are exemplified. As already stated above, shows 4 in the upper line a sequence of three single images (Pi, Pi + 1, Pi + 2) as original images. In these images, microstructures (X) detected by the second process section (PS2) and displacements (dX) by traced microstructures (X) are shown.

In the pictures in the second line of 4 Contrast lines are highlighted from the respective original images above. The contrast lines can be generated, for example, in the pre-processing by edge detection. Such edge detection is also known as gradient image processing. The edge detection can be performed in different directions, for example in a horizontal and a vertical direction, with each horizontal and vertical contrast transitions are highlighted separately. The edge detection may be performed alternatively or in addition to the Gaussian image processing in step (S20) of the processing in the second process section (PS2).

In the pictures in the bottom line of 4 Image areas (Lok1, Lok2) are marked with an increased density of recognized and in particular traced microstructures. These local image areas (Lok1 and Lok2) can also be called Relevance areas are called. Each relevance area can be assigned a certain degree of relevance. In the lower left picture of 4 In addition, the local arrangement and distribution of the detected microstructures is shown.

In the top line of 5 are again highlighted the high or increased relevance evaluated image areas (Lok1, Lok2) in which a recognition of macro-structures according to step (S140) is performed. In the pictures in the bottom line of 5 Then the respectively recognized macro-structures (Y) as well as recognized macroscopic structures recognized objects (O), here recognized pedestrians, are drawn.

By transmitting results from the second process section (PS2) via recognized and in particular tracked microstructures (X, dX), the process for recognizing macrostructures (Y) executed in the first process section (PS1) can be significantly optimized. In particular, those image regions can be prioritized in which, according to the evaluation of the optical flow, objects with an increased risk of collision or an increased object density could exist. As a result, safety-critical foreign objects (O1, O2) in the vicinity of the vehicle ( 3 ) and in particular pedestrians are detected faster. It also increases the likelihood that relevant alien objects will be detected without increasing the overall computational cost of the recognition process (RP). In other words, it is achieved that elaborate processing is excluded from image areas in which there are presumably no or noncritical foreign objects. On the other hand, image areas in which presumably many or security-critical foreign objects are present are preferably processed. As a result, the overall efficiency of the object recognition technology can be increased.

Modifications of the invention are possible in various ways. In particular, the features shown or described with respect to the individual embodiments can be combined with one another in any desired manner, replaced with one another, supplemented or omitted.

The cameras (K1 to K4) can be configured as desired. You can take a black and white picture or a color picture. In particular, CMOS cameras with an image size of 1280 × 800 or more pixels and an image acquisition frequency of 30 Hz can be used. Instead of the aforementioned structure patterns and object patterns for the recognition of pedestrians, other structure patterns or object patterns may be used. These patterns can serve, for example, the recognition of other road users such as cars, trucks or cyclists. Alternatively or additionally, patterns may be used to detect commonly-encountered environmental objects such as houses, street lights, street signs, lane boundaries, curbs, or the like. The patterns may have different sizes and use comparison methods other than the HOG recognition method mentioned above. A mixture of different recognition methods and pattern types can also be used.

Furthermore, the object patterns and pattern may be in different resolutions or detailing.

The information acquired in the second process section (PS2) about the identified and, in particular, tracked microstructures can be transmitted in blocks or continuously to the first process section (PS1). This makes it possible to achieve high parallelization of the processings in the process sections (PS1, PS2). For example, the images to be processed can be divided into several processing zones, which are processed in a predetermined order. The recognition results from the second process section (PS2) can be transmitted separately to the first process section for each of these zones, so that the relevant areas can be established with a very short time delay. When processing an image from the front environment of the vehicle in the second process section, it is thus possible first to process microstructures in the center of the image and in the middle right and middle left regions. Collision-capable objects are most likely to be expected in these areas. In the first process section (PS1), the detection of macrostructures can accordingly commence first in the relevant areas of these zones. In the further course of time, for the same processed image (Pi + 1), recognition information about microstructures in the further zones can then be supplied to the first process section (PS1), so that the recognition of macrostructures in the relevant areas located there as well can be continued.

The relevance assessment can be done in any way. For example, different relevance classes can be provided, wherein the image areas to which the highest relevance class is assigned are considered first for the recognition of macrostructures in the first process section (PS1). Subsequently, the other image areas can be processed with decreasing relevance. The recognition of macro-structures can be limited in time overall and for example, be aborted when reaching a maximum processing cycle length. Alternatively, a complete check for the presence of recognizable macro-structures can take place for individual images from the image sequence. As a result, even those image areas, which are assigned a low relevance, occasionally subjected to a fine detection.

LIST OF REFERENCE NUMBERS

1

 Object recognition system - Object recognition system

2

Electronic control unit (ECU) - Electronic control unit (ECU)

3

 Vehicle - Vehicle

X

 Micro structure - Micro structure

dX

 Displacement of a tracked microstructure - Displacement of a tracked microstructure

Y

 Macro structure - Macro structure

RP

 Recognition process - Recognition process

PS1

 First process section - First process section

PS2

 Second process section - Second process section

pi

 Picture - Picture

K

 Camera / - Camera / picture

Ki

 Image recorder - recording device

O,

 Object / Foreign Object - Object / Foreign

Oi

object

Appi

Application / Processing - Application / Processing

HOG

 Histogram of directed gradients - Histrogram of oriented gradients

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• JP 2011253214 A [0002]

Cited non-patent literature

• Dalal, N., Triggs, B .: "Histograms of Oriented Gradients for Human Detection", IEEE Computer Vision and Pattern Recognition, Vol. 1, pp. 886-893, 2005 [0054]
• Harris, C., Stephens, M .: "A combined corner and edge detector", proceedings often he 4th Alvey Vision Conference, pp. 147-151, 1988 [0054]

Claims (16)

Camera-based object recognition method for detecting objects (O1-O4) in the surroundings of a vehicle ( 3 ), wherein a recognition process (RP) recurring images (Pi, Pi + 1, Pi + 2) from the environment of the vehicle ( 3 ) are supplied with a known chronological sequence, which is determined by one or more on the vehicle ( 3 ), wherein the recognition process (RP) comprises at least a first process section (PS1) for recognizing macro-structures (Y) within individual images (Pi) and a second process section (PS2) for recognizing Having micro-structures (X), characterized in that result information from the second process section (PS2) via detected micro-structures (Y) to the first process section (PS1) are supplied to make a relevance assessment for local image areas (Lok1, Lok2) and to scale the process performed in the first process section (PS1) to recognize the macro-structures (Y). Object recognition method according to claim 1, characterized in that the macro-structures (Y) are static macro-structures, in particular specific object shapes. Object recognition method according to claim 1 or 2, characterized in that the microstructures (X) are tracked in a sequence of at least two images (Pi, Pi + 1). Object recognition method according to at least one of claims 1, 2 or 3, characterized in that in the second process section (PS2) detected micro-structures (X) are tracked in a sequence of at least two images (Pi, Pi + 1, Pi + 2) , Object recognition method according to one of the preceding claims, characterized in that in the second process section (PS2) shifts (dX) of tracked micro-structures (X) are detected. Object recognition method according to one of the preceding claims, characterized in that the first process section (PS1) and the second process section (PS2) are executed in parallel or overlapping in time. Object recognition method according to one of the preceding claims, characterized in that in the first process section (PS1) recognized macro-structures (Y) for the detection of certain objects (O1-O4), in particular for the detection of pedestrians, further processed. Object recognition method according to one of the preceding claims, characterized in that in the first process section (PS1) a comparison of a selected image area of a single image (Pi) is carried out with one or more structural patterns. Object recognition method according to claim 6, characterized in that the comparison is carried out with a sliding search window, wherein the content of the search window is compared with a structural pattern. Object recognition method according to one of the preceding claims, characterized in that the structural patterns processed in the first process section (PS1) are filed as a histogram of directed gradients (HOG). Object recognition method according to one of the preceding claims, characterized in that the number of structural patterns to be included in the comparison in the first process section (PS1) is determined as a function of the relevance score. Object recognition method according to one of the preceding claims, characterized in that the resolution and / or size of the search window used in the first process section (PS1) is determined as a function of the relevance score. Object recognition method according to one of the preceding claims, characterized in that the resolution and / or size of the structure pattern used in the first process section (PS1) is determined depending on the relevance score. Object recognition method according to one of the preceding claims, characterized in that the image areas (Lok1, Lok2) of a single image (Pi, Pi + 1, Pi + 2), which are to be processed prioritized in the first process section (PS1), depending on the relevance score to be selected. Object recognition method according to one of the preceding claims, characterized in that in the second process section (PS2) a recognition and in particular tracking of microstructures (X) is carried out according to the Harris-Corner process. Object recognition system ( 1 ) for detecting objects (O1-O4) in the vicinity of a vehicle ( 3 ), whereby the system ( 1 ) at least one camera (K1-K4) and an ECU ( 2 ), characterized in that the ECU ( 2 ) trained is to carry out a method according to any one of the preceding claims.

Images