DE102018123393A1 - Detection of parking areas - Google Patents

Abstract

The disclosure relates to a computer-implemented object recognition method for recognizing at least one parking area in the vicinity of a vehicle. Ambient image recordings are obtained which comprise an image representation of a surrounding area of the vehicle. Visually perceptible boundary structures of a parking area marking are recognized in the surrounding image recording. Parking areas can be identified on the basis of the boundary structures recorded.

Description

The invention relates to a technology for recognizing parking areas in the vicinity of a vehicle.

State of the art

Object recognition methods are known from practice, with which objects in the surroundings of a vehicle can be recognized mechanically. Object detection is based on the processing of data from at least one environmental sensor. For this purpose, camera sensors are preferably used, with which an area of the vehicle surroundings is captured in image recordings.

Various computer-aided methods are known with which certain structures in an image recording, e.g. Road markings, traffic lights or other vehicles can be recognized. Such methods often place high demands on the computing power of the computer technology used and / or the sensors used. In order for object recognition methods to be used during the operation of a vehicle, the methods must also meet real-time conditions.

task

The object of the invention is to demonstrate an improved object recognition method for recognizing parking areas. The invention solves the problem with the features of the independent claims.

Object recognition process

The main claim relates to a computer-implemented object recognition method for recognizing at least one parking area in the vicinity of a vehicle.

The method is particularly suitable for the detection of parking spaces for cars which are oriented transversely to the direction of travel of the vehicle. Such parking areas are often arranged in groups, especially in large parking lots e.g. in front of shopping centers. The method can also be used for the detection of parking spaces that are aligned lengthways or diagonally to the direction of travel of the vehicle.

The process consists of several steps. The steps can be carried out individually or several times. In addition, the steps can be carried out in one or more phases, which are carried out in particular in a predetermined sequence and / or timing. The steps of the object recognition method are preferably carried out in succession. Individual steps can be carried out at least partially in parallel. A step can contain one or more sub-steps. The method steps can be implemented in one or more (software) modules.

One step of the method includes acquiring at least one surrounding image recording, which comprises an image representation of a surrounding area of the vehicle. The environmental image recording is preferably obtained from an environmental sensor which is arranged on the vehicle. An environmental image recording can also be obtained, which is fused from a plurality of image recordings, in particular camera images and / or other data recorded in relation to the image raster.

The surrounding image recording comprises an image representation of a surrounding area of the vehicle. The image display preferably has a two-dimensional pixel grid, each pixel comprising one or more image information. An image recording has an image coordinate system. A pixel or other image information has a definable image position in the image coordinate system. The image position is preferably determined by a row index and a column index of a pixel. Other image coordinates can also be used.

Image information can in particular comprise a gray value, a color value, a brightness value and / or a contrast value. Alternatively or additionally, image information can also include depth information, a disparity or some other image signal value. Ambient image recordings of a mono camera, in particular with a fish-eye lens, are preferably used. Alternatively or additionally, image recordings of one or more stereo cameras or of a camera network can be used. The image recordings can also be composed of several image recordings or otherwise preprocessed.

The surrounding area of the vehicle captured in the image recording is preferably located to the side of the vehicle (in relation to the direction of travel). Alternatively or additionally, the detected surrounding area can also include areas in the direction of travel in front of and / or behind the vehicle.

A further step of the object detection method includes the detection of at least one optically perceptible linear boundary structure of a parking area marking in the surrounding image recording.

A parking area in the sense of this disclosure is suitable and provided for one Parking the vehicle, especially a car, on it. Such a parking area is essentially in a driving level on which the vehicle can move. A parking area is usually limited by at least two parking area markings.

Parking area markings are often applied to the driving level with a colorant, so that the parking area marking stands out visually, in particular due to a different color and / or brightness, from the background of the driving level. The edge areas of the parking area markings have an optically perceptible linear boundary structure in the surrounding image recording.

Visually perceptible linear boundary structures can be recorded using various methods. Optically perceptible boundary structures in the form of edges are preferably recorded in a surrounding image recording. Edges can be captured by evaluating gradients of one or more image information. A gradient can be calculated in particular by comparing neighboring image information in an image recording.

Another step of the object recognition method includes identifying at least one parking area based on the at least one detected boundary structure. A parking area object is generated for an identified parking area. A parking space object can be generated by an electronically storable data structure.

An identified parking area is preferably made available by a parking area object in a structured database. A parking area object is a virtual model of a recognized parking area in the vicinity of the vehicle. The data structure includes recorded properties of an identified parking area, in particular the position and / or orientation and / or width of the parking area. Another property of a parking area object can be the location within a parking area group. In addition, information about the reliability of the detection of a parking space object can be stored.

The object recognition process comprises several independent aspects. Each of the aspects is based on an independent inventive idea. The aspects can be used individually or in combination as part of the object recognition process. A step of a sub-process can be subordinate or subordinate to a step of the object recognition process. In a preferred embodiment, the partial methods build on one another. In particular, the sub-processes are carried out in a predetermined order.

Image preprocessing process

An independent aspect of this disclosure is an image preprocessing process. The image preprocessing method comprises the characterizing features of claim 2. In this partial method, a processing section is determined in an environmental image recording.

A processing section comprises one or more scan segments. The at least one optically perceptible boundary structure is detected within the processing section. The processing section can be redetermined in particular for several image recordings. The processing section is dynamic, i.e. the processing section of a first image recording can differ from the processing section of a second image recording taken later.

A processing section defines an area of an environmental image. This area can consist of one or more parts. A processing section preferably comprises a substantially smaller amount of image information than the entire surrounding image recording.

Determining the processing section has the advantage that the amount of data to be processed is reduced for further processing steps. The determination of the processing section in particular requires less computing effort than the further processing of the image information excluded from the processing section.

Processing of image information can be limited to the processing section. A processing section is preferably formed around areas of a surrounding image recording in which there are optically perceptible linear boundary structures. Parking space markings are more likely to be shown in these areas. If possible, other areas should not be part of the processing section. The amount of data to be processed should be as small as possible. The at least one optically perceptible boundary structure is detected preferably or exclusively within the processing section.

Determining the processing section comprises the following steps, which can be carried out one or more times.

One step involves defining scan segments. A scanning segment comprises a partial area of an environmental image recording. A scan segment can be determined in particular by shape and / or size. To be favoured rectangular scan segments with a predetermined edge length defined in pixels.

The processing section can be formed from a selection of certain scanning segments. The scan segments are used for the local processing of image information to determine its probable relevance for the detection of parking areas. A scanning segment can be moved in the image like a window in the image acquisition. Scan segments can be generated individually and / or moved to a defined position. A scan segment can also be defined by a certain selection of image information.

For example, a scan segment can comprise a partial area of 20 × 20 or 100 × 100 pixels. The individual scanning segments are preferably regularly distributed over the surrounding image recording or a predetermined section of the surrounding image recording.

In a preferred embodiment, adjacent scan segments overlap. In particular, scanning segments can be distributed at regular intervals in the horizontal and vertical directions of the surrounding image recording. The scanning segments can directly adjoin one another in the vertical direction. The scan segments can overlap in the horizontal direction. Alternatively, the scanning segments can also adjoin one another in both directions or overlap. Adjacent or overlapping in the other direction is also possible.

Another step involves applying an edge filter to the surrounding image acquisition or a scanning segment to determine edge features.

With an edge filter, optically perceptible boundary structures are highlighted in an image. For example, a Sobel filter or another edge filter known to the person skilled in the art can be used as the edge filter.

An edge filter can be applied in particular to a group of pixels within one or more scan segments. The edge information generated when the filter is used can be stored in an ordered data structure, in particular in a pixel grid. Edge information generated in this way can be assigned to a specific pixel or a specific group of pixels of the surrounding image recording. The edge information is preferably stored as additional image information in the same or an additional pixel grid. The edge information can be present in particular in the form of one or more gradient images for an environmental image recording.

To determine edge characteristics, the edge information generated using an edge filter is evaluated. An edge feature can be, for example, a gradient value above a certain threshold. Various techniques are known to those skilled in the art for determining edge features. Edge features can in particular be assigned to a specific pixel in a surrounding image recording. A pixel in turn has an image position known from the pixel grid.

Another step includes the directional counting of edge features with respect to at least one predetermined direction in a scan segment.

This step is preferably carried out for each scan segment. In directional counting, edge features are counted along a predetermined direction. One direction has a predetermined angle in the image coordinate system of the surrounding image recording.

The counting can be carried out within a counting corridor with a predetermined width in pixels. In addition, the count can be carried out in a number of such counting corridors which are offset parallel to one another.

For example, 5 parallel counting corridors for several predetermined directions or angles can be determined in the image coordinate system in a scanning segment.

Directional counting of edge features is a particularly efficient way of recognizing linear boundary structures.

The result of a count is preferably stored in a direction-related sum. If parking area markings are imaged in a certain partial area in a surrounding image recording, it is very likely that many edge features will lie one behind the other in this area in this area. In direction-related counting, only edge features on an essentially linear structure are recorded in a direction-related sum. Edge features on a curved structure, however, are distributed over several direction-related sums. Individual high direction-related sums therefore speak for an essentially linear boundary structure in this scanning segment. This method can be carried out particularly computationally and is therefore particularly suitable for real-time applications.

Another step includes including a scan segment in the processing section when a directional sum of the edge features in that scan segment exceeds a processing relevance threshold.

The processing section is preferably formed from scanning segments in which a direction-related count yields a direction-related sum above the processing relevance threshold. Alternatively or additionally, other recording criteria can also decide on the inclusion of a scanning segment in the processing section. For example, adjacent scan segments in addition to previously recorded scan segments can be included in the processing section.

Feature extraction process

Another standalone aspect of this disclosure is a feature extraction process. This feature extraction method can be used as a sub-method of the object recognition method. The feature extraction method comprises the characterizing features of claim 3.

The feature extraction method is used to determine characteristic features of a surrounding image that are more likely to lie on or on an image of a marking strip. A feature is characteristic image information or a combination of characteristic image information. A feature can be assigned to one or a group of pixels. A feature has a position in the image coordinate system directly or indirectly via an assignable pixel.

A feature generally represents an image element that is interesting for further image processing, in particular object recognition. Objects to be recognized generally have characteristic features in an image display that are (machine) recognizable by machine. Such characteristic features are to be determined using a feature extraction method. In order to automatically determine a feature, image information can be evaluated automatically. The determination of edge features is particularly advantageous for the detection of marking strips.

The object of the feature extraction method is to find one or preferably a plurality of edge-feature pairs. An edge feature for which an edge feature pair can be found is more likely to lie on a marking strip of a parking area. It is therefore advantageous to search specifically for edge-feature pairs.

One step of the method involves determining or relating edge features in an environmental image recording. Edge characteristics can be determined using one or more edge filters. Alternatively or additionally, edge features can be obtained from another module or another source.

The transitions between a generally dark or gray background and a white (or other colored) marking strip are particularly characteristic of a marking strip. Image recordings of such transitions usually have an edge between two surfaces in the image. A surface adjacent to an edge generally has an essentially uniform brightness or color. An edge in a surrounding image recording is characterized by edge features.

In a further step, an edge feature is determined as the reference edge feature. For a reference edge feature, you can then try to find an edge feature pair.

Known properties of a marking strip are used a priori when determining edge features. A common marking strip has two opposite edges. Edge features on one of the edges differ from edge features on the opposite edge of the marking strip. It is expected that there will be a transition from light to dark on one side of the strip and an opposite transition on the opposite side. If there are country-specific differences in road markings, there may also be other characteristic transitions.

In addition, marking strips have a customary width within an expected interval. Marking strips with a width of 5 to 20 cm are common. Depending on the recording technique and the distance of the marking strip to the environmental sensor used, the representation of a marking strip in an environmental image recording also has a width within an expected interval in image coordinates. It is therefore to be expected that for an edge feature on an edge of a marking strip there is an associated counter-edge feature on the opposite side of the marking strip. An edge feature pair can be formed from these two edge features.

In order to determine pairs of edge features, it is advantageous to proceed from one reference edge feature from a plurality of known edge features, with an associated counterpart. Edge feature to look for. The reference edge characteristic forms the starting point for the search.

Starting from a reference edge feature, a second edge feature is sought as the associated counter-edge feature. The search takes place along a predetermined search direction. A search is preferably carried out along the horizontal direction or a pixel line of the surrounding image recording if marking strips of a parking area lying transverse to the direction of travel are to be recognized. Alternatively or additionally, other search directions e.g. along the vertical or a diagonal in the surrounding image acquisition for a counter-edge feature. A surrounding image recording has a first and a second image axis. The first image axis is preferably a lying axis. The second image axis is preferably an upright image axis.

It is particularly advantageous to search for a counter-edge feature in the surrounding image recording within a predetermined distance. The specified distance is based on the expected width of a marking strip. In addition, imaging parameters of the recording technique of the surrounding image recording can be used. For example, a reference edge feature can be searched for within a distance of 15 or 50 pixels for a counter edge feature.

The search is advantageously carried out in only one search direction, in particular running to the left or right, in the surrounding image recording.

If a counter-edge feature can be found for a reference-edge feature, this counter-edge feature is assigned to the reference-edge feature to form a pair of edge features. Pairing can be done by providing a reference from one feature to another feature. Alternatively or additionally, an additional data structure can be created for an edge feature pair. It is particularly advantageous to preferentially or exclusively consider edge features for which a counter-edge feature can be found during further processing. Edge characteristics or edge-characteristic pairs can be released for further processing. Alternatively or additionally, edge features for which no pair of edge features can be found can be deleted, excluded from further processing, or treated subordinately.

Filtering method

Another essential aspect of this disclosure is a filtering method. The filtering method comprises the characterizing features of claim 4.

The filtering method involves finding edge points that lie in the driving plane of the vehicle. Using this method, edge features in an environmental image recording that belong to an image of a marking strip at floor level can be distinguished from other edge features that are not at floor level. For example, edge features on a vehicle or a road sign in height above the ground can be distinguished from edge features on a parking area.

This filtering method can advantageously be used in combination with the aforementioned feature extraction method. For example, it can happen that edge features on a white stripe on the body of a parked vehicle are detected during a feature extraction that are difficult to distinguish mechanically from edge features on a marking stripe of a parking area. A suitable distinguishing criterion is the height of a point in the vicinity of the vehicle in relation to the driving level.

In a two-dimensional environment image recording, only reduced information of the three-dimensional environment is available. It requires special technical measures to determine the position of a surrounding point in the surroundings from a two-dimensional picture of the surroundings. Various methods are known for this, which often place high demands on the sensors used (e.g. stereo cameras). The particular advantage of the method disclosed here lies in the very computationally efficient design and high compatibility with various environmental sensors.

One aim of the method is to determine edge points that are highly likely to be at ground level in the driving plane of the vehicle. Edge points that lie in the driving plane of the vehicle are preferably released for further processing.

One step of the method includes the determination of edge features with an associated image position in a surrounding image recording. Alternatively or additionally, edge features can also be obtained from another module or another source.

For at least one edge feature, an associated edge point is determined in a movement space coordinate system.

A movement space coordinate system is a at least two-dimensional coordinate system in the movement space of the vehicle. A three-dimensional coordinate system is preferably used, which is fixed in relation to the vehicle and is carried along with the vehicle when the vehicle moves in its surroundings. A vehicle coordinate system that lies on the rear axle of the vehicle is preferably used.

Alternatively, the entire object detection method can also be carried out with a movement space coordinate system that is defined in relation to the surroundings of the vehicle (“world coordinate system”). In this case, the vehicle must be localized within this world coordinate system. Various methods for transferring the disclosed method to an environmentally stable world coordinate system are known to the person skilled in the art. All steps disclosed here can be carried out analogously by a person skilled in the art with an environment-fixed movement space coordinate system.

The determination of an edge point for an edge feature in an environmental image recording takes place on the (mathematical) assumption that the edge point lies in a predetermined driving plane. This is particularly advantageous for the use of the method with image recordings from a mono camera. Pixels of a two-dimensional environment image recording can be transformed into a three-dimensional movement space coordinate system using known camera parameters and the additional assumption of a spatial plane on which the point imaged in the environment image recording lies.

If the assumption of the driving plane applies to a point of the surroundings of the vehicle which is imaged in the surroundings image recording, the transformation of the position of the depicted point in the movement space coordinate system can be determined or estimated by this transformation. If the assumption is incorrect, the particular position is faulty. During a movement of the vehicle, the error in determining the position is very likely to change due to the change in the camera perspective. When looking at several temporally offset surrounding image recordings during a movement of the vehicle, this effect can be used to differentiate between edge features at the level of the driving level and edge features above or below the driving level.

A further step involves the accumulation of edge points in the movement space coordinate system during a movement of the vehicle over at least two successively recorded surroundings image recordings.

When edge points are accumulated, information from several surrounding image recordings, which were recorded at different points in time, are collected in a common movement space coordinate system.

During the movement of the vehicle, a marking strip that is spatially defined in the surroundings shifts with respect to the vehicle. Correspondingly, the image of this marking strip shifts in surrounding image recordings taken at different times. The movement space coordinate system is also shifted with respect to the environment during the movement of the vehicle.

Edge points from an older surrounding image recording are shifted between two time-taken surroundings image recordings in accordance with the vehicle's own movement between the recording times in the moving space coordinate system moved along. The influence of the vehicle's own movement on the position of an edge point in the movement space coordinate system is thus compensated. Information about the vehicle's own movement can be obtained from an external source, e.g. in a bus network of the vehicle. Alternatively or additionally, motion information can also be generated from surrounding image recordings.

In order to be able to use image information, in particular edge points, from a plurality of image recordings taken at different times, older edge points are transformed into a younger movement space coordinate system. Such a transformation corresponds to a displacement of the edge points in a vehicle coordinate system carried along.

If a proximity criterion between an older edge point that has been shifted according to the vehicle's own movement and a younger edge point is met, these two points are accumulated in the movement space coordinate system. When accumulating, it is determined how often edge points have already been recorded in the vicinity of a specific point in the environment. Measurement information from several image recordings is thus collected.

The determination of an edge point for an edge feature in a surrounding image recording corresponds to a measurement or estimation of a surrounding position which is associated with this edge feature. Moving one Edge point in the movement space coordinate system corresponds to a model-based prediction of the measured or estimated position of the edge point at a later point in time.

It is to be expected that a correctly measured point will also be measured again in a subsequent image acquisition - if necessary at a point shifted according to the vehicle's own movement. The shifted position of an edge point in the movement space coordinate system is thus the expected position of a new measurement or estimation of the same environment point in a new environment image recording.

If the assumption of a driving plane is correct for the determination of an edge point, it is to be expected that edge points at several positions in the movement space coordinate system to be expected will coincide with one another with new measurements or are close to one another over several image recordings. A frequent accumulation of edge points at a certain point in the movement space coordinate system therefore suggests that there is a linear boundary structure at the level of the driving plane at this point.

In the event of an incorrect assumption of a driving plane when determining an edge point, the measurement or estimation of the position of a surrounding point is erroneous. In most cases, a new erroneous measurement or estimation of the position in a new surrounding image recording at a later point in time after a movement of the vehicle will not match the expected or model-based predicted position from previous measurements. Edge points to edge features on the same marking strip in several successive surrounding image recordings are therefore not expected to accumulate.

A further step includes the identification of edge points which, on the basis of the correctly made assumption regarding their position on the driving level, fulfill the proximity criterion to older edge points and are accumulated with them. These identified edge points are released for further processing. Alternatively or additionally, representatives of these accumulated edge points can be released. Released edge points are preferred during further processing or only taken into account.

Line recognition process

Another standalone aspect of this disclosure is a line detection method. The line recognition method is used to recognize line objects in a set of edge points. The line recognition process can be carried out as a sub-process of the object recognition process. The line recognition method comprises the characterizing features of claim 8.

The method includes the creation of line objects in a movement space coordinate system based on previously determined edge points. The method is preferably applied to a set of released edge points. The creation includes the following steps.

At least two separate sets of points are obtained, the first set of points comprising edge points of a first type and the second set of points comprising edge points of a second type. Typically, edge features differ from one another on two opposite sides of a marking strip. It is advantageous to separate edge points of a first type, which are typically on a certain side of a marking strip, from edge points of a second type, which are typically on another side of a marking strip. The edge points can preferably be stored in separate groups. As an alternative or in addition, the data structures of the edge points can have additional information which suggests a certain type of the associated edge feature.

The aim of this method is to recognize geometric lines or to generate associated data structures that lie with high probability on the contour of a marking strip. It is therefore expedient to only consider edge points for the determination of the line objects that are typically on a specific side of a marking strip.

A generated line object preferably also carries information relating to the type of the underlying edge points. It is advantageous to also store the line objects in separate groups, a group comprising line objects of a certain type. For easier understanding, edge points or line objects of the first type can be regarded as “left” elements of a marking strip and edge points or line objects of the second type as “right” elements of a marking strip.

A line in a plane can be mathematically determined using two parameters. For this purpose it is advantageous to use an angle and a transverse distance to a reference point in a reference coordinate system. Alternatively, other parameter combinations can also be used analogously become. Various parameterizations of lines are known to the person skilled in the art here.

Predefined and discretely distributed parameter combinations can be compared with recognized edge points in order to identify lines in a surrounding image recording. A particularly efficient method for this is the counting of edge points within predefined counting strips disclosed here.

One step of the method involves defining counting strips in the movement space coordinate system. A counting strip has a predetermined direction angle and a predetermined transverse distance in the movement space coordinate system. The directional angle can be related to one of the axes of the coordinate system, for example. The transverse distance can be determined from a reference point, e.g. define the zero point of the coordinate system on a line with the aforementioned direction angle. The transverse distance or the direction angle can also be defined in relation to another counting strip.

The counting strips are defined in discrete parameter steps. Depending on the available computing power and / or the required recognition accuracy, different parameter steps and value ranges can be advantageous. Counting strips are preferably defined in steps of 0.5 degrees in a range of values from -30 to +30 degrees. Steps of 10 cm are preferred for the transverse distance. The counting strips are preferably determined in a range of approximately 2 m to 20 m around the vehicle.

The width of a metering strip can be given implicitly by the staggering of the transverse distances. The width of a counting strip can also be explicitly specified. Parallel counting strips can directly adjoin one another or overlap. Overlapping counting strips are preferably used.

In a further step, the edge points are counted within a set of points that lie within a counting strip in the movement space coordinate system. The counting is carried out for a set of points in combination with a plurality of predefined counting strips. When counting, a counting strip sum of the edge points located therein is formed for a counting strip. The counting strip totals form a distribution over different angles and transverse distances of the previously defined counting strips.

The distribution of the meter strip totals for the various meter strips, each with a direction angle and a transverse distance, can be described as a mathematical function with the two discretely distributed parameters direction angle and transverse distance. This function forms a parameter combination of the direction angle and transverse distance on an associated counting strip sum.

In a further step, at least one counting strip is determined which has a local counting maximum within the distribution of the counting strip sums. A counting strip stands for a certain combination of parameters, in particular a transverse distance and a direction angle. Various methods are known to the person skilled in the art for determining local maxima within a two-dimensional distribution with a discrete value base.

In a further step, a line object in the motion coordinate system is defined for a metering strip with a local counting maximum with the direction angle and the transverse distance of this metering strip. By analyzing local maxima in a discrete distribution of predetermined parameter combinations, lines in the vicinity of the vehicle can be determined in an efficient manner, on which as many detectable edge features or edge points as possible lie.

Based on the line objects generated, contour line objects are preferably determined which have at least one end. In a particularly advantageous embodiment, clustering of edge points is used for the determination of contour line objects.

Strip detection method

Another independent aspect of this disclosure is a stripe detection method. This method can also be carried out as a sub-method of the object recognition method or independently. The stripe detection method includes the characterizing features of claim 10.

The method includes the determination of stripe objects based on previously recorded contour line objects. Corresponding contour lines of a marking strip can be determined with the method. A stripe object can be created from two contour line objects. The process consists of several steps.

A step involves acquiring at least two sets of lines, the first set of lines comprising contour-line objects of a first type. The second set of lines includes contour line objects of a second type. The contour-line objects of the first type are preferably the left side of a marking strip and the second type the right side of a marking strip.

In a further step, a contour-line object of the first set of lines is compared with a counter-contour-line object of the second set of lines. The comparison takes place in pairs. This step is carried out repeatedly.

Within a comparison pair, a measure of agreement between the contour line object and the counter-contour line object is determined. The measure of conformity depends on parallelism, distance and / or mutual coverage between the contour line objects of a comparison pair. The calculation of a measure of conformity on the basis of these criteria is particularly advantageous for an efficient assignment of a large number of recognized contour line objects to one another. The measure of conformity can also be determined from a selection or a combination of individual criteria (parallelism, distance, mutual coverage).

The typical widths and shapes are known for a marking strip in the vicinity of a vehicle. This a priori information can be used for the determination of stripe objects.

If the match size of a comparison pair is above a match threshold, a stripe object is created based on the contour line object and the counter contour line object of the comparison pair.

The measure of conformity is preferably calculated by comparing two coverage areas. A cover area denotes the area transverse to a contour line object up to a (possibly mentally extended) second contour line object.

Parking space detection method

Another independent aspect of this disclosure is a parking area detection method. With this method, parking areas between a plurality of recognized strip objects in the surroundings of the vehicle can be recognized. The method includes the characterizing features of claim 12.

The method comprises the generation of a parking area object on the basis of previously detected stripe objects. A group of connected parking spaces in the vicinity of the vehicle is preferably identified. The method is particularly suitable for the detection of parking areas which are oriented transversely to the direction of travel or the lane and are parallel to one another in a row or a group.

One aim of the method is to identify parking areas in a number of previously recognized stripe objects and to differentiate them as robustly and efficiently as possible from other open spaces between the stripe objects. For example, a passage between two strip objects can be distinguished from a parking area.

The process consists of several steps. One step involves acquiring stripe objects, a stripe object having a position and an orientation in a movement space coordinate system.

The stripe objects may have been created or recognized by one of the methods described in this disclosure. As an alternative or in addition, stripe objects can also be obtained from other sources or another software module or a sensor.

The recognition of the parking areas is based on a pattern recognition. Parking spaces in larger parking lots are usually arranged in groups. The regular intervals and / or forms of the arrangement of the parking areas form recognizable patterns. These patterns can advantageously be used to identify a parking area.

In one step, a reference stripe object is determined from a set of related stripe objects. This step can be repeated for different stripe objects. A reference stripe object serves as a starting point when looking for patterns in the set of known stripe objects. This search can be carried out repeatedly for several or all detected strip objects. Several passes are preferably carried out in succession for different reference strip objects.

In a further step, a plurality of predetermined spacing patterns are applied to the related stripe objects. A spacing pattern is formed by parallel staggered pattern lines, which are determined with a repetitive pattern offset starting from a reference strip object. These pattern lines are used to calculate deviations between the predetermined pattern and the detected stripe objects.

Sample lines can be viewed as virtual calculation aids. In the implementation of the method, such pattern lines can be implemented using various parameterizations. For example, it is possible to combine a certain pattern offset with an index. Each pattern line is through one Combination of a reference strip object, a pattern offset and an index specified.

In addition, it can be determined whether the pattern lines are determined in one or both directions across the reference strip object.

In a further step, an optimal spacing pattern with the best fit between the stripe objects and the pattern lines of the spacing pattern is determined. A spacing pattern is optimal if the spacing pattern has a minimal deviation or best fit between the related stripe objects and the pattern lines of this pattern.

The fit is preferably calculated between a stripe object and a pattern line near the stripe object. An error function is particularly suitable for determining the fit.

An optimal spacing pattern has an optimal pattern offset. The various predetermined spacing patterns each have a predetermined pattern offset. The predefined pattern offsets are preferably based on the parking area widths usually found in parking lots. Several pattern offset values in a range from 2 m to 4 m are preferred. In Germany in particular, pattern offset values of 2.3 m to 2.5 m are advantageous.

In a further step, a set of fits is determined from a reference strip object and at least one further strip object. A fit set contains stripe objects that lie within a tolerance range around a pattern line of the optimal spacing pattern. A set of fits preferably comprises at least three stripe objects.

Pattern recognition takes place in two stages. In the first stage, an optimal distance pattern is determined from a selection of predetermined distance patterns. In the second stage those stripe objects are identified that coincide with the optimal pattern. It is more likely that they belong to a coherent group of parking spaces. Since the typical arrangements of parking areas are known, this application of predetermined patterns and testing against the detected strip objects is particularly advantageous. The processing of recognized strip objects can thus be carried out very efficiently.

In a further step, a coherent group of parking area objects between the strip objects of a fit set is determined.

In particular, a parking area object is created between two stripe objects of a set of fits. The parking areas are preferably created as a coherent group. A parking area has a certain size and location. There is also a relationship between a parking area and the associated strip objects.

It is possible that there are several separate groups of parking spaces in the vicinity of the vehicle. In this case, several sets of fits can be determined for different reference strip objects.

The drawings of this disclosure are explained in more detail below and further advantageous features and embodiments of the invention are described.

Figure list

• 1: eine Draufsicht auf ein Fahrzeug, in dessen Umgebung sich zwei Parkflächen befinden;
• 2: eine Umgebungs-Bildaufnahme mit mehreren Abtastsegmenten und einem Vergrößerungsausschnitt (Z);
• 3: zwei zeitlich versetzt aufgenommene Umgebungs-Bildaufnahmen während einer Bewegung des Fahrzeugs sowie einen Vergrößerungsausschnitt (Y);
• 4: schematische Darstellung eines Fahrzeugs auf einer Fahrebene und Kanten-Punkten in unterschiedlichen Höhen gegenüber der Fahrebene;
• 5: ein Bewegungsraum-Koordinatensystem mit einem Raster und darin verortete Kanten-Punkten vor und nach einer inkrementellen Bewegung des Fahrzeugs zwischen zwei Umgebungs-Bildaufnahmen;
• 6: mehrere vorbestimmte Zählstreifen mit verschiedenen Querabständen und Richtungswinkeln in einem Bewegungsraum-Koordinatensystem;
• 7: eine Verteilung von Zählstreifen-Summen mit zwei lokalen Maxima;
• 8: ein erzeugtes Linien-Objekt sowie ein durch Clustering von Kanten-Punkten bestimmtes Kontur-Linien-Objekt in einem Bewegungsraum-Koordinatensystem;
• 9: erfasste Kontur-Linien-Objekte in einem Bewegungsraum-Koordinatensystem;
• 10: Kontur-Linien-Objekte mit unterschiedlichen Übereinstimmungsmaßen sowie ein Streifen-Objekt aus zwei Kontur-Linien-Objekten;
• 11: Bestimmung eines Übereinstimmungsmaßes zwischen einem Kontur-Linien-Objekt und einem Gegen-Kontur-Linien-Objekt;
• 12: Anwendung zweier vorbestimmter Abstands-Muster mit zwei verschiedenen Muster-Versätzen auf ein Bezugs-Streifen-Objekt;
• 13: Bestimmung eines Passungs-Satzes für ein optimales Abstands-Muster zwischen Parkflächenbegrenzungen;

The invention is illustrated by way of example and schematically in the drawings. Show it:

• 1 : a plan view of a vehicle in the vicinity of which there are two parking spaces;
• 2nd : an environmental image recording with several scan segments and a magnification section ( Z. );
• 3rd : two temporally offset ambient images during a movement of the vehicle as well as an enlarged section ( Y );
• 4th : schematic representation of a vehicle on a driving level and edge points at different heights compared to the driving level;
• 5 : a movement space coordinate system with a grid and edge points located therein before and after an incremental movement of the vehicle between two surrounding images;
• 6 : a plurality of predetermined counting strips with different transverse distances and direction angles in a movement space coordinate system;
• 7 : a distribution of counting strip sums with two local maxima;
• 8th : a generated line object and a contour line object determined by clustering edge points in a movement space coordinate system;
• 9 : Detected contour line objects in a movement space coordinate system;
• 10th : Contour line objects with different dimensions of conformity as well as a stripe object from two contour line objects;
• 11 : Determination of a measure of conformity between a contour line object and a counter-contour line object;
• 12th : Application of two predetermined distance patterns with two different pattern offsets to a reference strip object;
• 13 : Determination of a fit rate for an optimal distance pattern between parking area boundaries;

The elements shown in the drawings can be part of the environment of the vehicle, i.e. real objects or shapes, or images of these real objects in an environmental image or virtual elements that serve to represent a data structure or a process step.

The various data structures, in particular objects of the disclosed methods, can also be visualized in other forms of representation. The invention is not restricted to the particularly simple to understand form of representation chosen here. In particular, the claimed data structures or process steps or process results can be selected differently in a graphic output of a device.

Particularly advantageous embodiments have been chosen for the representation in the drawings. Occasionally, elements for easier understanding are deliberately not shown in the drawings, since otherwise they would no longer be understandable to the human viewer.

1 shows a vehicle ( 1 ) with an environmental sensor ( 11 ). The environmental sensor ( 11 ) is preferably a camera. It is particularly advantageous in practice for fish eye cameras ( 11 ) that have a particularly wide coverage.

There are two parking spaces in the vicinity of the vehicle ( 2nd ). The parking areas ( 2nd ) are marked by parking area markings ( 201 ) marked. At the edges of the parking area markings ( 201 ) optically perceptible linear boundary structures ( 202 ).

A visually perceptible boundary structure is in particular the transition from a gray concrete or asphalt floor to a white or yellow color stripe. As an alternative or in addition, interrupted markings can also be detected instead of solid marking strips.

The vehicle ( 1 ) has a camera in this embodiment ( 11 ) that takes up the area next to the vehicle transversely to the direction of travel. As an alternative or in addition, image recordings of further cameras, in particular in the front or rear direction of the vehicle, can be obtained. In a particularly advantageous embodiment, image information from the combination of a plurality of image recordings taken simultaneously is used.

The parking area markings ( 201 ) are usually formed by marking strips, which are painted on the background of the environment. The parking area markings ( 201 ) are generally at the level of the driving level. In the vicinity of the vehicle ( 1 ) can look like in 1 shown are also other objects such as a traffic sign or a directional arrow also painted on the ground. Such objects are also recorded in a surrounding image ( 300 ) usually shown.

2nd shows an exemplary environment image recording ( 300 ). An environmental image capture ( 300 ) can represent a snapshot, so-called frame, of a video recording. The surrounding image acquisition ( 300 ) includes image information that can usually be displayed in a pixel grid. Image information can be assigned to a specific point in an image. The individual pixels can be determined via one or two indices, in particular a row and a column index. Alternatively or additionally, a different image coordinate system can be selected.

Depending on the arrangement and orientation of an environmental sensor on the vehicle, an environmental image recording ( 300 ) a certain perspective. Areas above and below the horizon can also be imaged in the surrounding image recording. As a rule, only a certain area of an ambient image recording ( 300 ) Interesting. Especially for real-time applications, the amount of data to be processed, especially image information, is decisive for the processing speed. It is therefore advantageous to process only image information from an image section that is as limited as possible.

In an advantageous embodiment, only image information within a certain, in particular static, section is considered. Additional, in particular dynamic, restrictions of the image information to be processed are preferably made within the static image section. A (dynamic) processing section ( 301 ) within a static region of interest (ROI). The determination of the processing section ( 301 ) is preferred for every related surrounding image recording ( 300 , 300 ' ) carried out.

The surrounding image acquisition ( 300 ) can be used in a large number of scan segments ( 302 ) can be divided. Within a scan segment ( 302 ) a particularly efficient preprocessing of the image information can be carried out. If preprocessing within a scan segment ( 302 ) If the result is that the scanning segment has features for an optically perceptible linear boundary structure, this scanning segment ( 302 ) to a processing section ( 301 ) added.

A processing cutout ( 301 ) consists of at least one scan segment ( 302 ) together. The scan segments ( 302 ) in a processing section ( 301 ) can adjoin, overlap and / or be at a certain distance from each other. The processing section ( 301 ) can be formed in one or more parts. In 2nd are just the scan segments ( 302 ) which corresponds to the processing section ( 301 ) have been added.

Within a scan segment ( 302 ) is based on edge characteristics ( 310 ) searched. The enlarged section ( Z. ) shows a scan segment ( 302 ) with multiple edge features ( 310 ) along an optically perceptible linear boundary structure ( 202 ).

The edge features ( 310 ) are preferred by applying an edge filter to the surrounding image ( 300 ) or a section, in particular the processing section ( 301 ), determined. In this preferred embodiment, an edge feature is assigned to a specific pixel in the image acquisition. The edge features ( 310 ) form a discrete set.

Within the scan segment ( 302 ) the edge features ( 310 ) counted according to direction. In a preferred embodiment, a plurality of predetermined counting directions ( 315 ) used. The counting directions ( 315 ) can be defined by discrete angular steps. The edge characteristics of several starting points in a respective counting direction are preferred ( 315 ) counted. The directional count has the particular advantage that edge features are based on a linear boundary structure ( 202 ) lead to a particularly high meter result.

In a further particularly advantageous embodiment, a histogram for the number of counted edge features ( 310 ) along a counting direction ( 315 ) created. In particular, several histograms can be generated for different counting directions, parallel counting directions being recorded at transversely offset locations within a histogram.

If a directional count for a counting direction ( 315 ) exceeds a processing relevance threshold, this scan segment ( 302 ) the processing section ( 301 ) added.

Sampling segments for which no count sum lies above a processing relevance threshold value are preferably not added to the processing section. In an advantageous embodiment, however, additional criteria for the inclusion of a scanning segment in the processing section can be taken into account. For example, adjacent scan segments of previously added scan segments can also be included.

The determination of a processing section ( 301 ) is preferred for every related surrounding image recording ( 300 ) performed. A dynamic processing section ( 301 ) has the advantage that the amount of data to be processed changes depending on the ambient conditions, especially during a movement of the vehicle ( 1 ), can adjust.

Directional counting is particularly suitable for distinguishing edge features ( 310 ) at linear boundary structures ( 202 ) of other boundary structures, especially on curved contours.

2nd also shows edge features of a first type ( 310a) and edge features of a second type ( 310b) . Typically, edge features on a certain side of a marking strip, for example the left side, can be assigned to a certain type. In a preferred embodiment, the edge features ( 310a , 310b ) distinguished on the basis of the gradient of the image information. A local gradient in the horizontal direction of the image recording is preferably considered for the detection of parking areas lying transversely to the direction of travel.

3rd shows two temporally staggered environmental images ( 300 , 300` ). The two environmental images each show an image of the surroundings of the vehicle ( 1 ) at a certain point in time while the vehicle is moving. According to the movement of the vehicle ( 1 ) the depicted objects move in the surrounding images.

In a preferred embodiment, edge features ( 310 ) for each related surrounding image recording ( 300 ) certainly. The surrounding image recordings are preferably obtained in a specific timing.

The enlarged section ( Y ) in 3rd also shows a typical arrangement of two edge features on the contours of a parking area marker ( 201 ) for which an edge feature pair ( 320 ) can be determined. The edge feature pair ( 320 ) consists of a reference edge characteristic ( 321 ) and a counter-edge feature ( 322 ). The determination of edge-feature pairs ( 320 ) is particularly suitable for the extraction of edge features on the contours of marking strips from a larger amount of edge features on other surrounding objects.

In a preferred embodiment, the image information of a surrounding image recording ( 300 ) line by line, especially along a horizontal search direction ( 325 ), searched. Especially the search for a counter-edge feature ( 322 ) can be searched very efficiently along an image line.

4th shows a vehicle ( 1 ) on a driving level ( 5 ) stands. The vehicle ( 1 ) has an environmental sensor ( 11 ). In the vicinity of the vehicle ( 1 ) located objects form optically perceptible boundary structures ( 202 ), especially on the edges of marking strips or other angular transitions, e.g. on a vehicle or a road sign. The border structures ( 202 ) in the driving level ( 5 ) or at a certain height ( H ) above or below the driving level ( 5 ).

The schematically illustrated surrounding image recording ( 300 ) includes a two-dimensional image of the three-dimensional environment of the vehicle ( 1 ). The mapping takes place physically in an environmental sensor ( 11 ) instead of.

The edge points ( 410 , 410x ) in the (two- or three-dimensional) movement space coordinate system ( 400 ) are preferred from a two-dimensional environmental image recording ( 300 ) certainly. In particular, parameters of a camera calibration can be used for the transformation. In general, not all independent coordinates of an edge point ( 410 ) can be determined directly. The edge point could theoretically be in a variety of possible positions along a projection line ( 411 ) lie.

By additionally assuming a level, especially the driving level ( 5 ), for the position determination of an edge point ( 410 ) an additional independent coordinate can be determined. The position determined in this way is based on a hypothesis that, as in the case of the edge point ( 410 ) can essentially correspond to reality or as in the case of the edge point ( 410x ) can contradict reality.

In the case of an edge feature ( 310 ), which in the surrounding image recording ( 300 ) at a border structure ( 202 ) at the driving level ( 5 ), an associated edge point ( 410 ) can be determined with sufficient accuracy. For edge features ( 310x) connected to a border structure ( 202 ) outside the driving level ( 5 ), an edge point ( 410x ) with an incorrect position in the movement space coordinate system ( 400 ) certainly. The acceptance of the driving level ( 5 ) is not applicable in this case. There is a deviation between the determined position ( 410x ) and the actual position ( 410r) the corresponding feature on the border structure ( 202 ).

For a computer, it is not easy to determine with just two-dimensional image acquisition whether the determination of an edge point ( 410 , 410x ) in a movement space coordinate system ( 400 ) has been correctly determined. However, if several time-delayed environmental images ( 300 , 300 ' ), then by considering the error in determining the edge points ( 410x , 410x` ) determine which edge points ( 410 ) with increased probability actually at the driving level ( 5 ) lie.

From different perspectives of an environmental sensor ( 11 ) the repeated determination of edge points ( 410x ) the same illustrated boundary structure for different errors when determining the position of an edge point ( 410x ) to lead.

In fact at the level of the driving level ( 5 ) lying edge points ( 410 , 410 ' ) can with high probability in several successive images ( 300 , 300 ' ), especially from different perspectives, can be determined correctly. It is expected that edge points ( 410 ) from an older surrounding image recording ( 300 ), which according to the own movement in the movement space coordinate system ( 400 ) with edge points ( 410 ' ) same boundary structure ( 202 ) in a recent surrounding image recording ( 300 ' ) coincide or at least lie close to each other.

Edge points ( 410 , 410 ' ) same boundary structure ( 202 ) can take several time-delayed environmental images ( 300 , 300 ' ) can be accumulated. The accumulation has the effect that the assumed hypothesis of a given driving level ( 5 ) can be tested.

5 shows a particularly advantageous embodiment for the accumulation of edge points ( 410 ) over several surrounding images ( 300 , 300 ' ) away. The vehicle ( 1 ) moves between the recording times of the recordings ( 300 , 300 ' ). The movement space coordinate system ( 400 ) while moving with the vehicle ( 1 ) carried along.

This in 5 displayed grid ( 405 ) is related to the movement space coordinate system ( 400 ) Are defined. The grid ( 405 ) has regularly distributed raster cells. The grid cells in the two dimensions of the driving plane ( 5 ) distributed. The grid cells can be distributed, for example, at a regular distance of 10 cm. The grid preferably divides a specific section of the coordinate system defined by the movement space ( 400 ) defined space into uniform sections. The division can be two-dimensional or three-dimensional. The grid ( 405 ) is particularly advantageous for efficient location of edge points ( 410 ), especially for an assignment to an accumulation point ( 420 ).

From a first picture of the surroundings ( 300 ) become edge points ( 410 , 410x ) certainly. At a later point, edge points ( 410 ' , 410x ' ) based on a recent second image of the surroundings ( 300 ' ) certainly. Edge points ( 410 , 410x ) from a recent surrounding image recording ( 300 ) are acc. the movement of the vehicle ( 1 ) in the movement space coordinate system ( 400 ) postponed. The position of the edge points ( 410 , 410x ) is compensated for movement. Techniques for motion compensation are known to those skilled in the art. In a particularly advantageous embodiment, alternatively or in addition to the edge points, accumulation points ( 420 ) postponed.

For environmental imaging ( 300 ) a variety of edge features or edge points ( 410 , 410x ) have been determined. It is therefore particularly advantageous to use edge points ( 410 ) that are close together. Accumulation points ( 420 ) used to edge points ( 410 , 410x ) summarize in a certain close range. In a particularly advantageous embodiment, the grid cells ( 405 ) used to determine which edge points can be accumulated or combined.

Each grid cell ( 405 ) an accumulation point ( 420 ) assigned. Edge points ( 410 , 410x ) are located in the grid. It is determined in which raster cell an edge point ( 410 ) lies. A located edge point ( 410 ) becomes the accumulation point ( 420 ) assigned to the corresponding raster cell. When summarizing or accumulating edge points ( 410 , 410x ) over several surrounding image recordings, it is determined how many edge points are already in a certain area, in particular in an accumulation point ( 420 ) have been summarized.

In a particularly advantageous embodiment, the accumulation points ( 420 ) the combination of edge points that are close together ( 410 ) of a certain surrounding image recording ( 300 ) as well as the accumulation of edge points ( 410 , 410 ' ) from temporally staggered environmental images ( 300 , 300 ' ).

In one embodiment, the accumulation points ( 420 ) continued and shifted according to the vehicle's own movement. With a new environment image acquisition ( 300 ' ) new edge points ( 410 ' , 410x ' ) certainly. The newly determined edge points ( 410 ' , 410x ' ) are summarized again in certain close ranges. Again accumulation points are preferably used for this. It is possible to add new accumulation points for a new environment image acquisition ( 300 ' ) to create. Alternatively or additionally, accumulation points ( 420 ) over several temporally staggered environmental images ( 300 , 300 ' ) continue to be used.

It is particularly advantageous to have a certain number of accumulation points ( 420 ) for a certain number of raster cells for each recording time. Accumulation points from past environmental images can be treated like edge points. An accumulation point ( 420 ) includes the same or similar information as an edge point ( 410 ). In addition, an accumulation point contains information about the number of edge points already accumulated or combined ( 410 ) or accumulation points ( 420 ), especially from older surrounding images ( 300 ). This information is preferably recorded in an accumulation counter. An accumulation point ( 420 ) a position and an accumulation counter. When accumulating or adding an edge point to an accumulation point, the accumulation counter can be increased by a certain amount.

In 5 the difference between correctly determined edge points ( 410 ) and incorrectly determined edge points ( 410x ) whose actual position ( 410r) in the real environment outside the driving level ( 5 ) lies. Due to the incorrect determination of the edge points ( 410x , 410x ' ) these edge points do not accumulate over several temporally offset environmental images ( 300 , 300 ' ) while the vehicle is moving ( 1 ).

In a first environmental image acquisition ( 300 ) the edge points ( 410 , 410x ) certainly. Edge points ( 410 ), which are located in a common grid cell, become the same accumulation point ( 420 ) assigned. The accumulation counter of the accumulation points ( 420 ) is increased according to the number of edge points combined. The accumulation counter per assigned edge point ( 410 ) increased by one step. In an advantageous embodiment, alternatively or additionally, accumulation points ( 420 ) from an older surrounding image recording ( 300 ) with accumulation points ( 420 ' ) from a recent surrounding image recording ( 300 ' ) accumulated. When an accumulation point is accumulated, the accumulation counters of the accumulated accumulation points ( 420 , 420 ' ) preferably added. Of a plurality of accumulation points accumulated with one another ( 420 , 420 ' ) or edge points ( 410 , 410 ' ) only one accumulation point ( 420 ' ) continued. This is particularly advantageous for efficient memory usage.

With a new surrounding image recording taken later in time ( 300 ' ) the accumulation points ( 420 ) from the past surrounding image recording ( 300 ) acc. the vehicle's own movement. Accumulation points ( 420 ) the edge points recorded in the past ( 410 ) represent. The shifted position of the accumulation points ( 420 ' , 420x ') corresponds to the expected position of newly determined edge points ( 410 ' , 410x ' ) if the boundary structures recorded with the edge points ( 202 ) are actually at the driving level in reality.

If an object in the vicinity of a vehicle is recorded in several time-delayed environmental images ( 300 , 300 ' ) with edge features, it can be expected that the corresponding edge points ( 410 , 410 ' ) accumulate from the temporally offset surrounding image recordings. The accumulation counter of the accumulation points ( 420 ) contains information on how often edge points have been determined in a certain area of the environment. With increasing number of accumulated edge points ( 410 , 410 ' ) the likelihood increases that in a certain area a boundary structure at the driving level ( 5 ) was recorded. The accumulation counter is a confidence estimate of the position determination of edge points ( 410 ).

If an accumulation counter of an accumulation point ( 420 , 420 ' ) reaches or exceeds a confidence threshold, this accumulation point or edge points accumulated therein are released for further processing. For the purposes of this disclosure, in particular for further processing of data, accumulation points ( 420 ) and edge points ( 410 ) can be viewed analogously. An accumulation point ( 420 ) a representative of one or more edge points ( 410 ) be.

6 shows a schematic representation of a selection of predetermined counting strips ( 435 , 435i ) each with a direction angle (Ai) and a transverse distance (Di) of a preferred embodiment. The counting strips ( 435 ) are related to a movement space coordinate system ( 400 ) defined in which edge points ( 410 ) can be determined. Analogous to the edge points shown here ( 410 ) released edge points or accumulation points can also be viewed and / or counted with an accumulation counter above the confidence threshold.

For each counting strip ( 435 ) a line object ( 430 ) determinable. The line object linked to a counting strip preferably has the same directional angle (Ai) and the same transverse distance (Di) as the associated counting strip ( 435i) on. A line object ( 430 ) is preferred by a mathematical straight line, especially in the driving plane ( 5 ), certainly. Alternatively or additionally, a parameter for a third dimension of a line object ( 430 ) be used.

A counting strip ( 435i) encompasses a specific area along and across the linked line object in which edge points ( 410 ) this counting strip ( 435i) be attributed. The counting strips with different directional angles (Ai) are likely to overlap in certain areas. Parallel counting strips ( 435 ) with the same directional angle (Ai) can directly adjoin or overlap or even be at a distance from one another.

For at least one counting strip ( 435i) the edge points within the counting strip ( 410 ) counted. Edge points ( 410 ), particularly in overlapping counting strips, can be counted several times to one counting strip. The number of within a counting strip ( 435i) lying edge points ( 410 ) forms a counting strip Sum or a counting result (c [Ai, Di]). A counting result is based on a direction angle (Ai) and a transverse distance (Di) on a counting strip ( 435i) and in particular a possible line object ( 430 ) assignable.

7 shows a possible distribution of counting results (c [Ai, Di]) for different predetermined transverse distances (Di) and direction angle (Ai). It is preferably a discrete distribution. Local counting maxima ( cMj ) determine. For each local maximum count ( cMj ) is an associated directional angle ( Amj ) and an associated transverse distance ( Dmj ) determinable. In this way it can be mathematically determined for which parameter combinations of directional angle (Ai) and transverse distance (Di) most edge points along a line can be found.

For at least one count maximum ( cMj ) becomes a line object ( 430 ) with a directional angle ( Amj ) and a transverse distance ( Dmj ) of the maximum count generated and / or released for further processing.

A set of line objects ( 430 ) for which a local maximum count ( cMj ) above a minimum count threshold.

The determination of the local count maxima ( cMj ) in the three-dimensional function for the counting result (c [Ai, Di]) is preferably simplified by intermediate steps. The simplification enables a computationally efficient solution to the problem using the expected environmental conditions. The simplification can be done by repeatedly looking at the counting results in two-dimensional distributions, in particular vectors.

In a first step, the counting results are preferably determined for a plurality of counting strips, each with a direction angle (Ai) and a transverse distance (Di). The counting results can be displayed in a three-dimensional histogram. The discretely distributed count results (c [Ai, Di]) can be represented in particular as a plurality of vectors c_Di [Ai] for a predetermined transverse distance [Di] in each case.

In a second step, a maximum of the count results is obtained in at least one, preferably each, vector c_Di [Ai] cmax_Di [Amj] determined at least one transverse distance Di. The direction angle [ Amj ] of the counter maximum cmax_Di [Amj] determined and saved. The counter maxima cmax_Di of the vectors for each transverse distance (Di) can be stored as an intermediate result.

In a third step, the intermediate results from counter maxima are considered as a vector for a large number of transverse distances (Di). Within this vector, local maxima of the counter results and the associated cross distances ( Dmj ) determined. For every local maximum cmax_Dmj [ Amj ] is the associated directional angle ( Amj ) is already known from a previous step and in particular can be called up.

This multi-stage procedure enables the determination of the transverse distances ( Dmj ) and the direction angle ( Amj ) to local maxima of the counting results ( cMj ) are carried out within two-dimensional distributions.

Due to the expected alignment of lines in a parking lot, the theoretically occurring ambiguities can be accepted in a step-by-step determination of local maxima.

8th shows a created line object ( 430 ) in a movement space coordinate system ( 400 ). To the line object ( 430 ) there are a number of edge points ( 410 ) distributed. To determine a start and / or end point of a line object ( 430 ) it is advantageous to cluster edge points ( 410 ) near the line object ( 430 ) to carry out. In the area of a cluster ( 441 ) from edge points ( 410 ) a section of the line object ( 430 ) as a contour line object ( 440 ) can be determined.

In this preferred embodiment, both a start and an end point of a contour line object ( 440 ) Are defined. A segment of a line object ( 430 ) as a contour line object ( 440 ) Are defined. A start or end point of a contour line object ( 440 ) can be near a cluster extreme point ( 410e ) can be determined. A cluster extreme point ( 410e ) lies within a cluster ( 441 ) at an extreme point in the direction of the line object ( 430 ). Which of the edge points ( 410 ) within a cluster ( 441 ) a cluster extreme point ( 410e ), in particular by comparing the distances of the respective edge points ( 410 ) to a reference point on the line object ( 430 ) be calculated.

To determine the start or end point of a contour line object ( 440 ) in particular a plumb line from a cluster extreme point ( 410e ) to the associated line object ( 430 ) be felled.

9 shows two sets of related contour line objects ( 440a , 440b ). It is particularly advantageous to use contour line objects ( 440 ) depending on the underlying type of the recorded Edge features ( 310a , 310b ) or edge points ( 410 ) in a group of a first type or a group of a second type. Contour line objects ( 440 ) of the same type.

Advantageously, only contour line objects ( 440a , 440b ) of different types viewed in pairs. For example, in the first set of contour lines, only "left" contour line objects ( 440a ) from the perspective of the vehicle ( 1 ) saved. The second set of contour lines again contains potential counter-contour line objects ( 440b ), which are expected to be on the right side of a marker strip.

10th shows various contour line objects ( 440 , 440a, 440b ). A contour line object ( 440a ) each with a counter-contour line object ( 440b ) compared. 10th shows in particular different possible positions of a contour line object ( 440a ), especially in comparison to a counter-contour line object ( 440b ). Various possible positions of the contour line objects ( 440a , 440b ) to each other.

Within a comparison pair ( 440a , 440b ) is a measure of conformity ( XS ) certainly. The measure of conformity ( XS ) describes how strong the two contour line objects ( 440a , 440b ) match each other in terms of parallelism, distance and / or mutual coverage. A high degree of conformity ( XS ) stands for a low angular deviation and / or a small offset and / or the expected distance between the contour line object ( 440a ) and the associated counter-contour line object ( 440b ) of a comparison pair. The measure of conformity ( XS ) can in particular comprise a quotient of two areas. Alternatively or additionally, the measure of conformity ( XS ) include a value for approximation or agreement with an ideal area or an ideal distance.

For a correctly recorded marking strip of a parking area marking ( 201 ) it is to be expected that two contour line objects ( 440a , 440b ) are essentially parallel to each other in a certain distance interval. In addition, the start and end points of the two contour line objects are ( 440a , 440b ) as expected on a line perpendicular to the other start or end point. It is therefore advantageous to have a measure of conformity ( XS ) to calculate the parallelism and / or coverage and / or distance between a contour line object ( 440a ) and a counter-contour line object ( 440b ) considered. In an alternative embodiment, a selection or a specific combination of the abovementioned criteria for determining the measure of conformity ( XS If a match measure within a comparison pair is above a match threshold, a streak object ( 450 ) based on the associated contour line object ( 440a ) and the counter-contour line object ( 440b ) Are defined. A stripe object ( 450 ) additional contour lines between the start and / or end points of the contained contour line objects ( 440a , 440b ) include. The streak object ( 450 ) can form its own data structure or consist of a combination of the associated contour line objects. The streak object ( 450 ) may also include other information such as a latitude or an averaged position. A strip object advantageously has a position and an orientation in the movement space coordinate system ( 400 ) on.

The determination of a measure of conformity ( XS ) is a particularly efficient method for determining associated contour line objects from a plurality or a plurality of detected contour line objects.

11 shows a particularly advantageous embodiment for determining a measure of conformity ( XS ). The process consists of several steps. In a first step, a cover area ( Sat ) in the transverse direction of the contour line object ( 440a ) educated. The coverage area ( Sat ) extends to the opposite contour line object ( 440b ). Possibly. one of the contour line objects ( 440a , 440b ) mentally extended by an intersection with a line in the transverse direction of the opposite contour line object ( 440a , 44b).

The coverage area ( Sat ) preferably forms a square. It is based on the start and end points of a contour line object ( 440a ) educated. The coverage area ( Sat ) on two opposite sides by a line at a right angle to the contour line object ( 440a ) limited. The coverage area ( Sat ) of the (possibly extended) contour line object ( 440a ) and the counter-contour line object ( 440b ) limited.

In a further step, a counter-cover area ( Sb ) in the transverse direction of the counter-contour line object ( 440b ) certainly. The determination of the counter-cover area ( Sb ) takes place analogously to the determination of the coverage area ( Sat ) instead, but starting from the counter-contour line object ( 440b ).

In a further step, a cut surface ( IS ) between the cover area ( Sat ) and the counter-cover area ( Sb ) certainly. The Coverage area ( Sat ) and the counter-cover area ( Sb ) usually overlap partially.

In a further step, a union surface ( US ) between the cover area ( Sat ) and the counter-cover area ( Sb ) certainly. The union area ( US ) covers the entire area of the cover area ( Sat ) plus the counter-cover area ( Sb ) without double counting of the overlap areas. The merging surface is formed by merging two surfaces, similar to what is common with sets.

The quotient from the cut surface ( IS ) and the union area ( US ) forms a suitable measure of conformity ( XS ) ("Cutting surface through union surface"). This measure of conformity is advantageously a dimensionless value between 0 and 1, with 0 representing a very poor match and 1 representing a very good match.

In addition, the distance between the contour line objects ( 440a , 440b ) in the measure of conformity ( XS ) flow.

For the measure of conformity ( XS ) a match threshold between 0.5 and 1 is preferably set. A measure of conformity ( XS ) above the match threshold suggests a recognized stripe from the compared contour line objects.

The 12th and 13 show an implementation of a parking area detection method. By using predetermined spacing patterns, parking areas ( 2nd ) between detected parking area markings ( 201 ) be recognized. A stripe object ( 450 ) represents a detected parking area marking ( 201 ). To capture a parking area ( 2nd ) become parking space objects ( 500 ) determined or generated.

12th shows the use of two spacing patterns with pattern lines ( PL1 , PL2 ). Several pattern lines offset in parallel ( PLi ) form a spacing pattern ( pi ). In the method shown, two predetermined spacing patterns are used. A variety of spacing patterns ( pi ) each with a different pattern offset ( POi ) applied.

A first distance pattern ( P1 ) shows a recurring pattern offset (POI) between parallel offset pattern lines ( PL1 ) on. A second spacing pattern has a second predetermined pattern offset ( PO2 ) between parallel staggered pattern lines ( PL2 ) on. The predetermined distance patterns are based on a set of related strip objects ( 450 ) applied. A stripe object ( 450 ) has a position and an orientation in the movement space coordinate system ( 400 ) on. The position of a strip object ( 450 , 450r ) can with the position of a pattern line ( PL1 , P12 ) are compared. To determine a fit rate ( 455 ) from related strip objects ( 450 ) will initially become a reference strip object ( 450r ) from a set of related strip objects ( 450 ) certainly. This step can be carried out several times for different reference strip objects. The application of several predetermined distance patterns is in each case for a reference strip object ( 450r ) carried out.

To a reference strip object ( 450r ) by using several predetermined spacing patterns, other stripe objects ( 450 ) that are determined in combination with the reference strip object ( 450r ) form an expected pattern. The predetermined distance patterns each have a pattern offset ( POi ) that lies within an expected interval of normal parking area widths. Distance patterns with a pattern offset in discrete steps of, for example, 5 cm in an interval of 2.0 to 2.5 m are preferably used.

Among the different spacing patterns used, a spacing pattern ( pi ) with the best fit between the strip objects ( 450 ) and the pattern lines ( PLi ) of the distance pattern ( pi ) certainly. The fit between a strip object ( 450 ) and a pattern line ( PLi ) is determined by comparing the respective position with regard to distance and / or parallelism.

The spacing patterns ( pi ) for several, in particular all detected, strip objects ( 450 ) as a reference strip object ( 450r ) determined. The fit of the streak objects ( 450 ) can differ with regard to several distance patterns ( pi ) with the same pattern offset ( POi ) related different reference strip objects ( 450r ) differentiate. The fit is preferably determined repeatedly for different reference strip objects ( 450r ). A spacing pattern refers to a particular reference stripe object.

In particular, several parking area groups with several optimal distance patterns can be determined in the same surrounding scene.

The (partial) procedure for determining patterns is repeated for several reference strip objects. The reference strip object with the best fit, in particular with a minimal result of an error function, is selected for further processing.

In a preferred embodiment, an error function is used to determine the fit between a spacing pattern and a plurality of stripe objects ( 450r , 450 ) to determine. An optimal spacing pattern ( pi ) can be determined in particular by an optimization process by minimizing the error function. Thus an optimal pattern offset ( POi ) for the well-known stripe objects ( 450 ) can be determined. In a second step, the stripe objects ( 450 ) are determined, which are highly likely to form a group of parking areas with the previously determined optimal pattern offset ( POi ) belong. From stripe objects ( 450 ) that are within a tolerance range ( T ) around a pattern line ( PLi ) of the optimal distance pattern ( pi ), a fit sentence ( 455 ) educated. A fit set ( 455 ) is a selection of stripe objects ( 450 ) from a set of related strip objects ( 450 ). A stripe object ( 450r ) forms the reference strip object of a fit set ( 455 ).

For a fit set ( 455 ) becomes a group of parking lot objects ( 500 ) generated. One parking space object is placed between two neighboring strip objects ( 450 ) of a fit set ( 455 ) generated. The parking lot objects ( 500 ) lie between the strip objects ( 450 , 450r ) of a fit set ( 455 ). The parking lot objects ( 500 ) preferably form a coherent series.

The parking lot objects ( 500 ) of a fit sentence ( 455 ) can have the same or a similar shape to one another. A parking space object ( 500 ) an independent data structure based on the stripe objects that delimit the parking area ( 450 ). A parking area object can in particular have a position and / or a position in the movement space coordinate system ( 400 ) exhibit.

In a preferred embodiment, parking area objects ( 500 ) a coherent group. One or more parking space objects ( 500 ) are readjusted with regard to joint flight and / or a string-free alignment and / or parallelism of the parking areas. As expected, parking spaces in larger parking spaces are aligned parallel to one another and form a common escape, ie the start or end points of the marking strips lie essentially on a common line. As a rule, parking areas in a group of a larger parking lot directly adjoin one another. In particular, two parking spaces ( 2nd ) usually a common parking area marker ( 201 ) between the two parking spaces. This a priori information is advantageously used for the readjustment of the parking area objects (which may not be measured exactly) ( 500 ) utilized.

Embodiments

In a preferred embodiment, the computer-based object recognition method is repeated at least partially for each new surrounding image recording ( 300 ) executed. The individual sub-aspects or sub-processes form different phases of a measurement of optically perceptible linear boundary structures ( 202 ). The object recognition method or at least one of the sub-methods are preferably carried out at a rate of several repetitions per second. In particular, environmental images ( 300 ) with a predetermined timing. Cycle times of 10 to 150 milliseconds are advantageous. Ambient image recordings are preferably obtained at a rate of approximately 15 images per second.

In a preferred embodiment, boundary structures ( 202 ) and related objects in the area surrounding the vehicle are repeatedly recorded. Object detection includes the measurement of one or more properties of an object, in particular a coordinate or a position. For example, by determining an edge point ( 410 ) carried out a measurement.

The virtual objects generated on the basis of the measurement, in particular contour-line objects ( 440 ), Stripe objects ( 450 ) and / or parking lot objects ( 500 ), additionally tracked. With object tracking, new measurements are continuously combined with previously recorded measurements. A measurement is based on one or more surrounding images ( 300 ) at a certain time. Information from measurements at earlier times and model information can be combined with more recent measurements using an estimation filter.

A Kalman filter is preferably used for the tracking. In a prediction step, position measurements of detected objects from older surrounding images ( 300 ) model-based in a younger state at the time of a younger surrounding image recording ( 300 ' ) predicted. For the prediction, a movement model of the vehicle ( 1 ) in combination with a related own movement of the vehicle ( 1 ) and / or odometry data from processing the surrounding image recordings ( 300 , 300 ' ) be used. In an innovation step, information from new measurements in a recent environmental image ( 300 ' ) combined with the older measurements. The assignment of a measurement of a younger object to an older object, in particular an already recorded contour line Object or a stripe object can be carried out using a proximity criterion, in particular a transverse distance or an angle difference between two objects.

Even if no new measurements of an object that has already been captured are available, since the captured object is no longer shown or covered in recent surrounding image recordings, so-called blind tracking can be carried out with a purely model-based prediction of the position of the object. As expected, the inaccuracy of such a purely model-based estimate increases over time. Inaccuracies in the movement model or the related movement information on the vehicle's own movement ( 1 ) can lead to an increasing inaccuracy of blind tracking. The related movement information relating to the vehicle's own movement is preferably compared with the displacement of objects that are still visible in the surrounding image recordings. An adapted own movement of the vehicle can be used for the prediction of a detected object in the surroundings of the vehicle during blind tracking. The adapted movement information can be the result of an optimization of an error function between predicted older object positions and new, younger measurements of the same object.

In a further preferred embodiment, only optically perceptible boundary structures ( 202 ) within the processing section ( 301 ) processed. Alternatively or additionally, the optically perceptible boundary structures ( 202 ) within the processing section ( 301 ) preferred or processed preferentially. The processing section is preferably determined ( 301 ) within a static image section ("region of interest"). The processing section ( 301 ) is preferred for every surrounding image recording ( 300 ) determined dynamically.

The distribution of the scan segments ( 302 ) within a surrounding image recording ( 300 ) is preferably predetermined.

To determine edge features ( 310 ) a Sobel operator is preferred for an ambient image recording ( 300 ), in particular a static section from the surrounding image recording. The Sobel operator can be used in particular in two or more different directions, in particular the horizontal and the vertical image direction. The result is preferably stored in at least one gradient image, preferably in a horizontal and a vertical gradient image. A vertical edge in a surrounding image recording is characterized by a horizontal gradient of at least one piece of image information. The search for significant gradients preferably takes place line by line in a gradient image.

In a preferred embodiment, a surrounding image recording and / or an associated gradient image is searched line by line from the left with respect to the image coordinate system. A parking area marking is generally characterized by a first positive gradient, in particular the gray value, and a negative gradient that follows at a short distance within a line. To determine an edge-feature pair ( 320 ) it is preferred to first search for an edge feature with a positive gradient and then for a counter-edge feature with negative gradient.

To find edge points ( 410 ) in the driving level ( 5 ) of the vehicle ( 1 ), preference is given to accumulation points ( 420 ) used. Alternatively or additionally, edge points ( 410 ) in a certain close range or a grid cell in the movement space coordinate system ( 400 ) can be counted. Between successive recording times, only accumulation points ( 420 ) for further processing. Alternatively or additionally, edge points ( 410 ) to be continued.

In a preferred embodiment, accumulation points ( 420 ) each with an accumulation counter and a room position in a movement space coordinate system ( 400 ) Are defined. The accumulation points ( 420 ) repeats for one image of the surroundings ( 300 ) and an associated recording time. For a first picture of the surroundings ( 300 ) is preferred for each or every released edge feature ( 310 ) an edge point ( 410 ) in the movement space coordinate system ( 400 ) certainly. All edge points ( 410 ), which is a proximity criterion between the edge point ( 410 ) and an accumulation point ( 420 ) meet this accumulation point ( 420 ) assigned. For each assigned edge point ( 410 ) the accumulation counter of the accumulation point ( 420 ) increased by one step.

To process a second, younger surrounding image ( 300 ' ) Movement data are obtained, in particular via a vehicle CAN bus. The movement data describe the vehicle's own movement ( 1 ) between a first recording time of an older surrounding image recording ( 300 ) and a second recording time of the younger surrounding image recording ( 300 ' ).

Updating the position of an accumulation point ( 420 ' ) takes place preferably with each newly acquired surrounding image recording ( 300 ' ) and preferably for each accumulation point to which at least one edge point is assigned. The assignment of edge points ( 410 ' ) in a new, younger environment image recording ( 300 ' ) to an updated accumulation point ( 420 ' ) is also preferred for each related surrounding image recording ( 300 ' ) and preferred for each edge point ( 410 ' ) instead of.

Individual environmental images or edge features or edge points can be skipped during processing. This can be particularly advantageous for fulfilling real-time conditions.

In another embodiment, an expiration rate is applied to an accumulation counter. The decay rate has the effect that the accumulation counter of an accumulation point is reduced over time. This is particularly advantageous when the vehicle is at a standstill. While the vehicle is at a standstill, new surrounding image recordings are continuously taken, the detected optical boundary structures ( 202 ) essentially remain there. Thus, all of the accumulation counters would increase over time. Also faulty edge points ( 410x ) would accumulate during a standstill of the vehicle. It is therefore advantageous to have a measurement, in particular the accumulation counter, only during a movement of the vehicle ( 1 ) to increase.

The accumulation of edge points ( 410 ) over several temporally staggered environmental images ( 300 , 300 ' ) can either be carried out continuously, in particular using a decay rate, or at predetermined accumulation intervals. An accumulation interval can, for example, span a period of a few 100 ms. A measurement of a position of a surrounding object, in particular a contour line object or a stripe object, can consist in particular of processing a plurality of surrounding image recordings during an accumulation interval. In particular, a measurement can only be trusted when a certain confidence is reached at the end of an accumulation interval.

In one embodiment, at least one area of the movement space coordinate system ( 400 ) in a predetermined grid ( 405 ) divided with raster cells. The grid ( 405 ) can in particular be predetermined once or can be parameterized. Each raster cell is preferably an accumulation point ( 420 ) assigned. In a preferred embodiment, with each surrounding image recording ( 300 ' ) a new accumulation point ( 420 ' ) for each raster cell. Accumulation points ( 420 ) or edge points ( 410 ) from older environmental images ( 300 ) are preferred with the newly created accumulation points ( 420 ' ) accumulated. The information in the older accumulation points ( 420 ) when accumulating to the new accumulation points ( 420 ' ) transfer.

The position of an accumulation point ( 420 ' ) when accumulating older accumulation points ( 420 ) or edge points ( 410 ' ) be adjusted. A middle position between a younger accumulation point ( 420 ' ) and an accumulation point ( 420 ) determined, which is accumulated with the former. When determining an average position, the position of an accumulation point in particular can be weighted using its accumulation counter. The use of a grid enables the method to be carried out in a particularly memory-efficient manner. The determination of an average position of the accumulation points in turn increases the accuracy of the position determination or measurement.

In a preferred embodiment, contour line objects ( 440 ) by clustering edge points ( 410 ) along or near a line object ( 430 ) certainly. Clustering is preferably based on a combination of properties of the edge points ( 410 ) and / or their assignable edge features ( 310 ) in a surrounding image recording ( 300 ). In particular, the distance between an edge point ( 410 ) to a reference point in the movement space coordinate system ( 400 ) be used. Alternatively or additionally, a color value of an edge feature ( 310 ) in a surrounding image recording ( 300 ) be used. Alternatively or additionally, the distance of a reference edge feature ( 321 ) in a surrounding image recording ( 300 ) to an associated counter-edge characteristic ( 322 ) within a pair of edge features ( 320 ) be used. Various algorithms are known to the person skilled in the art for carrying out clustering.

In clustering, a set of edge points ( 410 ) individual connected clusters ( 441 ) educated. Edge points ( 410 ) grouped based on one or more properties. It is to be expected that edge features and their associated edge points on a common boundary structure ( 202 ), in particular on a parking area marking, have similar properties to one another.

In a further embodiment, detected strip objects ( 450 ) readjusted. The Readjustment can be carried out by determining an optical transition along a search direction at the end of a strip object ( 450 ) occur. In particular, several search lines parallel to the stripe object at one end of the stripe object ( 450 ) To be defined. A search line defines a series of pixels inside and outside the stripe object. The search line preferably begins within the strip shortly before its end and protrudes beyond the end of the strip. Image information along the search line is preferably evaluated. It is preferred to search for an abrupt jump in the gray value along the search line. Experience has shown that such a jump represents the transition from, for example, a white stripe to a dark background. The end of the streak object ( 450 ) is then set to the point of a characteristic transition along a search line. Alternatively or additionally, several search lines can be defined and their results combined.

The disclosed object recognition method is preferably used for motor vehicles. The process offers particular advantages for this application. In particular, the method can be implemented cost-effectively with large-series components and automotive electronics.

The object detection method and / or a disclosed partial method is carried out in particular by executing instructions on an electronic control device, e.g. a control unit in a vehicle.

The disclosure also includes a software product that contains instructions suitable for execution on an electronic data processing device. The instructions are designed to carry out one of the claimed methods.

The disclosed method is preferably implemented in a computer. The method is preferably carried out fully automatically. In particular, the processing of ambient image recordings in real time operation requires a partially automatic or fully automatic execution of the object recognition process.

Control commands can be generated on the basis of the method or on the basis of the data structures or objects generated. The data structures can in particular be output on a display device. The generated control commands can be used to control a display device or a vehicle guidance device (e.g. parking assistant).

Modifications of the invention are possible in various ways. In particular, the features shown, described or claimed for the respective exemplary embodiments can be combined with one another in any manner, replaced with one another, supplemented or omitted.

Reference list

1

vehicle

11

camera

2nd

Parking area

5

Driving level

6

horizon

201

Parking area marking

202

Border structure

300, 300 '

Ambient image acquisition

301

Processing cutout

302

Scan segment

310

Edge feature

310a

Edge characteristic type 1

310b

Edge characteristic type 2

310x

Edge feature (outside the driving plane)

315

Count direction

320

Edge feature pair

321

Reference edge feature

322

Counter-edge feature

325

Search direction

400

Movement space coordinate system

405

Grid, grid cells

410, 410 '

Edge point

410x, 410x '

Defective position of an edge point (incorrect assumption regarding the driving plane)

410r

Actual position of an edge point in a real environment

410e

Cluster extreme point

411

Projection line

H

Height above driving level

420, 420 ', 420x

Accumulation point

430

Lines object

435, 435i

Counting strips

440, 440a

Contour line object

441

Cluster

440b

Counter-contour line object

450

Stripe object

450r

Reference strip object

455

Fit set

500

Parking space object

Ai, Amj

Directional angle

Tue, Dmj

Transverse distance

c [Ai, Di]

Counting result

cMj

Local count maximum

P1, P2, Pi

Distance pattern

PO1, PO2, POi

Pattern offset

PL1, PL2, PLi

Pattern lines

T

Tolerance range

IS

Cutting surface

US

Union area

XS

Measure of conformity

Sat

Cover area

Sb

Counter-cover area

Y Z

[enlarged image sections]

Claims (17)

Computer-based object recognition method for recognizing at least one parking area (2) in the vicinity of a vehicle (1), the object recognition method comprising the following steps: - Obtaining at least one surrounding image recording (300), which comprises an image representation of a surrounding area of the vehicle (1); - Detecting at least one optically perceptible linear boundary structure (202) of a parking area marking (201) in the surrounding image recording (300); - Identifying at least one parking area (2) based on the at least one detected boundary structure (202). Object recognition process according to Claim 1 , characterized by : - determining a processing section (301) in an environmental image recording (300), which comprises one or more scanning segments (302), the at least one optically perceptible boundary structure (202) being detected within the processing section (301), and wherein determining the processing section (301) comprises the following steps, which are carried out one or more times: ◦ defining scan segments (302) as partial areas of the surrounding image recording (300); ◦ applying an edge filter to the surrounding image recording (300) or a scanning segment (302) to determine edge features (310); ◦ In a scan segment (302): direction-related counting of edge features (310) with respect to at least one predetermined direction (315); ◦ If a directional sum of the edge features (310) in a scan segment exceeds a processing relevance threshold, include this scan segment (302) in the processing section (301). Object recognition process according to Claim 1 or 2nd , characterized by : finding one or preferably a plurality of edge-feature pairs (320), whereby edge features (310) for which an edge-feature pair (320) can be found are released for further processing, and the finding comprising the following steps: ◦ determining or acquiring edge features (310) in a surrounding image recording (300); ◦ determining an edge feature (310) as a reference edge feature (321); ◦ Starting from a reference edge feature (321): searching for a second edge feature (320) as an associated counter-edge feature (322), along a predetermined search direction (325), in particular along a first or second image axis of the surroundings Image recording (300), and / or within a predetermined distance (326) in the surrounding image recording (300); ◦ If a counter edge feature (322) can be found, assign this counter edge feature (322) to the reference edge feature (321) to form an edge feature pair (320). Object detection method according to one of the preceding claims, characterized by : - finding edge points (410) which lie in the driving plane (5) of the vehicle (1) and are released for further processing, and the finding comprises the following steps: ◦ Determining edge features (310) with associated image position in a surrounding image recording (300); ◦ For at least one edge feature (310): Determining an associated edge point (410) in a movement space coordinate system (400) on the assumption that the edge point lies in a predetermined driving plane (5); ◦ accumulation of edge points (410, 410x, 410 ', 410x') in the movement space coordinate system (400) during a movement of the vehicle (1) over at least two successively recorded surrounding image recordings (300, 300 '), whereby one younger edge point (410 ') from a younger surrounding image recording (300') with an older edge point (410) from an older surrounding image recording (300) which is in the movement space coordinate system (400) according to the vehicle's own movement (1) is accumulated when a proximity criterion between these edge points (410, 410 ') is met; ◦ Identification of edge points (410, 410 ') which, on the basis of the correctly made assumption regarding their position on the driving plane (5), meet the proximity criterion for older edge points (410) and are accumulated with them, these edges Points (410 ') or representatives (420') of these accumulated edge points (410 ') are released for further processing. Object recognition process according to Claim 4 The accumulation comprises the following steps: - Defining at least one accumulation point (420) with an accumulation counter and a space position in a movement space coordinate system (400). - For a first surrounding image recording (300) that was recorded at a first point in time, carry out the following steps at least once: ◦ assigning an edge point (410) of the first surrounding image recording (300) to an accumulation point (420 ) in the movement space coordinate system (400), for which a proximity criterion between the edge point (410) and the accumulation point (420) is fulfilled; ◦ incrementing the accumulation counter of an accumulation point (420) when an edge point (410, 410x) is assigned to this accumulation point; - For a second environmental image recording (300`), which was recorded at a later second point in time: ◦ obtaining movement data about the own movement of the vehicle (1) between the first point in time and the second point in time; performing the following steps at least once: ◦ updating the position of an accumulation point (420 ') in the movement space coordinate system (400) in accordance with the movement data; ◦ assigning an edge point (410 ') of the second environment image recording (300') to an updated accumulation point (420 ') for which a proximity criterion is fulfilled; ◦ incrementing the accumulation counter of an accumulation point (420 ') when an edge point (410') is assigned to this accumulation point; - Identification of accumulation points (420`) whose accumulation counter is above a confidence threshold and release of these accumulation points for further processing. Object recognition process according to Claim 4 or 5 , wherein an expiration rate is applied to an accumulation counter so that the accumulation counter is reduced over time. Object recognition process according to Claim 4 , 5 or 6 , characterized by : - determining a grid (405) with grid cells in a movement space coordinate system (400), at least one and in particular each grid cell being assigned an accumulation point (420); - Locating an edge point (410) in the grid (405); - Assign an edge point (410) to the accumulation point (420) of the raster cell in which the edge point (410) is located. Object recognition method according to one of the preceding claims, characterized by : - generating line objects (430) in a movement space coordinate system (400) on the basis of previously determined edge points (410), in particular released edge points (410), wherein the generation comprises the following steps: ◦ obtaining at least two separate sets of points, the first set of points comprising edge points of a first type and the second set of points comprising edge points of a second type; ◦ defining counting strips (435) in the movement space coordinate system (400), each counting strip (435i) having a predetermined directional angle (Ai) and a predetermined transverse distance (Di) to a reference point in the movement space coordinate system (400); ◦ For the at least two sets of points and predefined counting strips: Counting the edge points (410) within a set of points that lie within a counting strip (435) in the movement space coordinate system (400), forming a counting strip sum ( c [Ai, Di]), the counting strip sums (c [Ai, Di]) forming a distribution over different angles of direction (Ai) and transverse distances (Di); ◦ determining at least one counting strip (435) which has a local counting maximum (cMj) within the distribution (c [Ai, Di]); ◦ For a metering strip (435) with a local counting maximum: Define a line object (430) in the movement space coordinate system (400) with the direction angle (Amj) and the transverse distance (Dmj) of this metering strip (435). Object recognition process according to Claim 8 , characterized by : - determining a contour-line object (440) based on a line object (430), the contour-line object (440) having at least one end, and wherein the determination comprises the following steps:: Obtaining or creating line objects (430); ◦ For a line object (430): clustering of edge points (410) along or near the line object (430), the clustering based on a combination of properties of the edge points (410) and / or whose assignable edge features (310) are carried out in a surrounding image recording (300), in particular based on a distance between an edge point (410) and a reference point in the movement space coordinate system (400), a color value of an edge feature (310 ) in a surrounding image recording (300) and / or a distance of a reference edge feature (321) in a surrounding image recording (300) to an associated opposite edge feature (322) of a pair of edge features (320 ); For a cluster (441) of edge points (410): ◦ determining at least one extreme point (410e) within the cluster (441) in the longitudinal direction of the line object (430); ◦ determining at least one end of a contour line object (440) on the basis of a determined extreme point (410e). Object recognition method according to one of the preceding claims, characterized by : - determining stripe objects (450) based on contour-line objects (440), the determination comprising the following steps: ◦ obtaining at least two sets of lines, the first line set includes contour line objects (440a) of a first type and the second line set includes contour line objects (440b) of a second type. ◦ Pairwise comparison of a contour line object (440a) of the first set of lines with a counter-contour line object (440b) of the second set of lines; ◦ Within a comparison pair: determining a measure of conformity (XS) with regard to parallelism, distance and / or mutual coverage between the contour line object (440a) and the counter-contour line object (440b); ◦ If the match size (XS) of a comparison pair is above a match threshold: Define a stripe object (450) based on the contour line object (440a) and the counter-contour line object (440b) of the Comparison pair. Object recognition process according to Claim 10 , the determination of a measure of conformity (XS) between a contour line object (440a) and a counter-contour line object (440b) comprising the following steps: ◦ determination of a coverage area (Sa) in the transverse direction of the Contour line object (440a), the cover area (Sa) extending to the counter-contour line object (440b); ◦ determining a counter-coverage area (Sb) in the transverse direction of the counter-contour line object (440b), the counter-coverage area (Sb) extending to the contour line object (440a); ◦ determining an intersection surface (IS) and a union surface (US) between the cover surface (Sa) and the counter-cover surface (Sb); ◦ Determine the measure of conformity (XS) from the quotient of the cut surface (IS) and the union surface (US). Object recognition method according to one of the preceding claims, characterized by : - generating a parking area object (500) on the basis of previously detected stripe objects (450), the generation comprising the following steps: ◦ obtaining stripe objects (450), wherein a Stripe object (450) has a position and an orientation in a movement space coordinate system (400); ◦ determining a reference stripe object (450r); Mehrerer Applying a plurality of predetermined spacing patterns, whereby for a spacing pattern (P1, P2, Pi) from the reference strip object (450r) staggered pattern lines (PLi) with a recurring pattern offset (POi) are determined relative to one another ; ◦ determining an optimal spacing pattern (Pi) with the best fit between the strip objects (450) and the pattern lines (PLi) of the spacing pattern (Pi); ◦ Determining a fit set (455) from a reference strip object (455r) and at least two further strip objects (450), each within a tolerance range (T) around a pattern line (PLi) of the optimum Distance pattern (Pi) lie; ◦ Determining a coherent group of parking area objects (500) between the strip objects (450, 450r) of a fit set (455). Object recognition process according to Claim 12 , characterized by : - readjusting a parking area object (500) within a coherent group of parking area objects with regard to common alignment and / or spacing-free alignment and / or parallelism of the parking areas. Object detection method according to one of the preceding claims, wherein object tracking is carried out, in particular tracking the position of a stripe object, a contour line object and / or a parking area object. Software product for execution on an electronic data processing device, which comprises instructions which are designed to implement a method according to at least one of the Claims 1 to 14 to execute. Electronic data processing device which is designed to receive signals from one or more environment sensor devices (11) and a method according to at least one of the Claims 1 to 14 to execute, in particular by following the instructions of a software product Claim 15 . Sensor system comprising an electronic data processing device according to Claim 16 and one or more environmental sensor devices (11).

Images