EP3688663A1

EP3688663A1 - Method and device for detecting an obstacle-free region

Info

Publication number: EP3688663A1
Application number: EP18792355.2A
Authority: EP
Inventors: Markus RAFFER; Kevin PAJESTKA; Friedrich Fraundorfer; David SCHINAGL
Original assignee: Tec-Innovation GmbH
Current assignee: Tec-Innovation GmbH
Priority date: 2017-09-26
Filing date: 2018-09-24
Publication date: 2020-08-05
Also published as: AT520579A1; US20200293776A1; US11205074B2; AT520579B1; WO2019060936A1

Abstract

The invention relates to a method for detecting the image of an obstacle-free region surrounding a photograph-capturing unit (1), wherein a) at least one photograph of a portion of the environment (10) of the photograph-capturing unit (1) is created by means of the photograph-capturing unit (1) and a digital picture (D) is created on the basis of said at least one photograph, characterized in b) that the digital picture (D) thus created is examined for the presence of possible boundaries (B) of an obstacle-free region, a boundary indicator value (BI) being determined for each individual pixel (P) of the digital picture (D) in accordance with the picture data of the digital picture (D) within an environment of the pixel (P) in question, said picture data being defined pixel by pixel, the boundary indicator value indicating the probability that the image of the boundary (B) of an obstacle-free region is located within a specified pixel environment, c) that a coordinate parameterization having two parameters is specified for the digital picture (D), which coordinate parameterization corresponds in particular to the pixel coordinates, d) that a boundary curve (BK) of the obstacle-free region, in particular in the form of a polygonal chain, is determined as a curve on which picture elements are located which, in comparison with picture elements in the environment thereof, have a high boundary indicator value (BI), e) that the image of the obstacle-free region is determined by the boundary curve (BK) and optionally further curves or straight lines specified in advance, in particular the boundary edges of the digital picture (D).

Description

Method and device for detecting an obstacle-free area

The invention relates to a method and a device for detecting the image of an obstacle-free region surrounding an image recording unit according to the preamble of patent claim 1.

From the state of the art are guidance such. B. tactile floor guidance systems in urban space, on the basis of which persons with impaired eyesight z. B. with the help of a cane or long stick independently orient and move. Disadvantage of such guidance is, however, that they are scarce in rural areas and that these are difficult or impossible for people with limited motor skills.

Furthermore, AT 513881 A2 discloses a device for detecting an environment which can be used in rural areas or outdoors or for persons with impaired vision and reduced motor skills. The device can be installed, for example, in a shoe or a cane or long stick and determined by means of a distance sensor, a distance to a potential obstacle. Depending on the distance determined, a warning is sent to the user via a feedback unit.

A disadvantage of the known device, however, is that only a single distance value is specified, which determines how far an object is located away from the receiving unit, wherein the distance measurement is measured only along a single direction, starting from the receiving unit.

The object of the invention is therefore to improve the known from the prior art device for distance detection and to provide a method that ensures a safe movement of persons with limited vision and reduced motor skills in any environment. In particular, it is an object of the invention to provide direction-dependent distance information which, for a plurality of directions, in particular for each image column of the digital image, in each case specifies a distance related to this direction.

The invention solves this problem in a method of the type mentioned, wherein a) with the image pickup unit at least one shot of a portion of the environment of the image pickup unit is created and based on this at least a recording is a digital image is created, with the features specified in claim 1. It is provided according to the invention

b) that the digital image thus created is examined for the presence of possible limitations of an obstacle-free region, wherein for each pixel of the digital image, a respective limiting indicator value is determined depending on the pixel-by-pixel predetermined image data of the digital image within an environment of the respective pixel, indicating with what probability within a given pixel environment is the image of the boundary of an obstacle-free area,

c) that for the digital image a coordinate parameterization with two parameters is specified, which corresponds in particular to the pixel coordinates,

d) that a boundary curve of the obstacle-free area, in particular in the form of a train path, is determined as a curve on which there are pixels having a high limiting indicator value compared to pixels in their surroundings, e) that the image of the obstacle-free area by the Boundary curve and possibly further predetermined previously curves or lines, in particular the boundary edges of the digital image is set.

In order to provide a particularly reliable image of the obstacle-free area surrounding the image recording unit, it may be provided that a path is selected as the limiting curve, which runs through a number of pixels of the digital image, wherein for a number of predetermined first parameters, in particular for each image column of the digital image or for a number of image columns each having a predetermined x-coordinate, one pixel each, in particular a pixel, is selected, through which the path runs.

In order to determine the limiting curve particularly reliably, it can be provided that an optimization method is carried out to determine the limiting curve in step d) for individual eligible boundary curves in each case for individual pixels on the respective boundary curve, in particular for a plurality of pixels respectively predetermined first parameters or x-coordinates, in each case a dimension value to be optimized is determined, which is composed of the following partial dimension values,

- the limit indicator value, as well

the value of the second parameter, in particular the y-coordinate, or of a value derived therefrom, and / or

the difference of the second parameter in the respective point from the second parameter in an adjacent point located on the same limit curve, the first parameter of which deviates by a predetermined value, and in particular in one adjacent column of the digital image, or a value derived from this difference.

In order to determine a particularly smooth boundary curve which preferably runs near the bottom, so that, for example, vertical edges extending from the floor are ignored in the determination of the boundary curve, it can be provided that an optimization method is carried out for determining the boundary curve in step d), in which for individual eligible boundary curves in each case those pixels on the respective boundary curve, in particular for a plurality of pixels each having predetermined first parameters or x-coordinates are selected

the highest possible limit indicator values in comparison with other pixels, in particular with the first predetermined parameter or the same x coordinate, and / or

- Have the highest possible values of the second parameter, in particular the y-coordinate, and / or

- Which have a second parameter, in particular a y-coordinate, which corresponds approximately to the second parameter of the respective neighboring point on the boundary curve or to this second parameter has little difference, in particular so that the position of the pixels results in a limiting curve whose change in Direction of the second parameter, in particular the y-coordinate, is as small as possible.

High accuracy and reliability in detection of an obstacle-free boundary can be achieved when the boundary indicator value is calculated pixel-wise by means of a neural network, in particular for all pixels in the same way and independently of the remaining pixels, the neural network being the input value individual brightness or color values within a, in particular for each pixel the same predetermined environment around the respective pixel, in particular with a predetermined relative position to each pixel receives.

To provide effective training for neural networks to provide accurate bound indicator values, it may be provided that the neural network is created using a plurality of predetermined training images, with a training bounding curve being given for each of the training images and the neural network thereon is trained, for the respective training picture, in the In the area of the training boundary curve, the pixels to be supplied have a limit indicator value which differs from the limit indicator values determined in the rest of the image area.

Another advantage of the invention lies in the fact that, in contrast to known from the prior art method in the context of training only manually determines where the boundaries of the walk-in area are or not. As a result, the Annotationsaufwand falls significantly lower than in known methods, which with the pixels each other information such as, for example, that it is a pixel, which z. B. represents a part of a window or a wall is, deposit.

In order to ensure a particularly safe movement for a user of the method, it may be provided that the actual size and shape of the imaged obstacle-free area is determined within the surroundings of the image recording unit,

due to the location of the obstacle-free area in the digital image,

due to the relative position, in particular height and orientation, of the image acquisition unit with respect to the environment,

optionally parameters of the image acquisition unit, in particular resolution and focal length,

such as

- assuming that the obstacle-free area detected in the digital image lies on a given three-dimensional surface, in particular on a plane.

In order to make a method according to the invention particularly flexible for persons with impaired eyesight and limited motor skills, it can be provided that the at least one receptacle is mounted with a walking stick fixed to a person or to an animal, in particular a shoe, glove, harness or walker arranged, image pickup unit is added.

In order to perform a method according to the invention not continuously, but only selectively under predetermined conditions, so that, for example, an image acquisition unit required to carry out the process need not be permanently supplied with voltage, it can be provided that the position and orientation of Image acquisition unit is monitored and the image acquisition unit only creates images or steps b) to e) are performed only

- When the image pickup unit is in a predetermined position, in which both the orientation of the horizontal image axis of the image pickup unit and the viewing direction of the image pickup unit are aligned approximately horizontally, in particular with an angular deviation of at most 30 ° about a horizontal axis, or

- When the image containing unit containing shoe is completely placed on the floor.

In order to specifically start the course of a method according to the invention, it may be advantageous for the digital images to be recorded with an image recording unit mounted in a shoe, the height of the image recording unit being determined above the ground when the shoe is resting on the floor and

the actual size and shape of the imaged obstacle-free area is determined according to the invention on the assumption that the image-recording unit is at the determined height.

A particularly safe locomotion can be made possible for persons with limited vision and reduced motor skills if, in addition to the limit indicator value, an obstacle indicator is also determined in step b) indicating the type of obstacle the obstacle-free area is limited to, and

if the obstacle indicator is determined for individual pixels on the limiting curve and preferably the person is displayed.

For a simple and precise provision of a distance for limiting the obstacle-free area for a user of a method according to the invention, it may be provided that a first distance of the limiting curve or at least one point on the boundary curve of the obstacle-free area is determined by the image recording unit,

- That in addition to recording the digital image by distance measurement by means of ultrasound, a second distance is determined, wherein an ultrasonic sensor used for this purpose is arranged in the region of the image pickup unit and

- Is aligned in the same direction as the image pickup unit, or

a detection area partially overlapping with the recording area of the image recording unit, - That the second distance and the first distance of the obstacle-free area are compared with each other and another distance measurement is created using the first distance and the second distance, preferably in each case deviations in the respective smaller distance is considered as another distance measurement.

The invention further relates to a method which ensures that a person with limited vision and reduced motor skills can move independently in an environment, since the person is warned of obstacles. According to the invention, it is provided that a distance value corresponding to the size of the obstacle-free area is determined or predetermined by one person, and then depending on this distance value at least one body location, in particular in at least one shoe, the person at least one of several sequentially in the direction of the more sensitive Actuator elements, in particular vibration elements, is activated, wherein the selection of the actuator elements is made depending on the distance value and wherein an actuator is selected in a sensitive body region, in particular the closer to the toe is selected, the smaller the Distance value is.

The invention further relates to an apparatus for detecting the image of an obstacle-free area surrounding an image recording unit, comprising an image recording unit and a processing unit connected downstream of the image recording unit. According to the invention, it is provided that the image recording unit is designed to produce at least one image of a subregion of the surroundings of the image recording unit and to transmit the image to the processing unit, and that the processing unit is designed to

to create a digital image based on this at least one image and to examine the digital image thus created for the presence of possible limitations of an obstacle-free region,

determine, for individual pixels of the digital image, in each case a limiting indicator value as a function of the pixel-by-pixel predetermined image data of the digital image within an environment of the respective pixel, which indicates with which probability the image of the boundary of an obstacle-free region is located within a given pixel environment,

to specify a coordinate parameterization for the digital image with two parameters, which corresponds in particular to the pixel coordinates, - To determine a boundary curve of the obstacle-free area, in particular in the form of a train, as a curve on which there are pixels that - compared with pixels in their environment - have a high limit indicator value, and

- Define the image of the obstacle-free area by the limiting curve and, where appropriate, further predetermined curves or lines, in particular the boundary edges of the digital image.

In order to determine a reliable image of the obstacle-free area surrounding the image recording unit, it can be provided that the processing unit is designed to

to select a path of travel as the limiting curve, the route passing through a number of pixels of the digital image, and

for a number of predetermined first parameters, in particular for each image column of the digital image or for a number of image columns each having a predetermined x-coordinate, in each case one pixel, in particular a pixel, through which the path extends.

For particularly reliable determination of the limiting curve can be provided in one embodiment of the invention that the processing unit is adapted to perform an optimization method for determining the boundary curve, wherein the processing unit is adapted for individual eligible boundary curves in each case for individual pixels on the respective Limit curve, in particular for a plurality of pixels each having predetermined first parameters or x-coordinates, each to determine a to be optimized measure value, which is composed of the following Teilmaßwerten:

- the limit indicator value, as well

the difference between the second parameter in the respective point of the second parameter in an adjacent point located on the same boundary curve, the first parameter of which deviates by a predetermined value, and in particular lies in an adjacent column of the digital image, or a value derived from this difference.

In order to ensure a smooth course of the limiting curve preferably near the ground, can be disregarded in the vertically away from the floor extending edges be provided that the processing unit is adapted to perform the determination of the boundary curve, an optimization method, wherein the processing unit is adapted for each eligible boundary curves in each case those pixels on the respective boundary curve, in particular for a plurality of pixels each having predetermined first parameters or x coordinates, select

- Have as high limit indicator values and / or the highest possible values of the second parameter, in particular the y-coordinate, in comparison to other pixels, in particular with the same predetermined first parameter or the same x-coordinate, and / or

- Which have a second parameter, in particular a y-coordinate, which corresponds approximately to the second parameter of the respective neighboring point on the boundary curve or has only slight difference to this second parameter, so that in particular from the position of the pixels results in a limiting curve whose change in Direction of the second parameter, in particular the y-coordinate, is as small as possible.

In order to achieve a high degree of accuracy and reliability in the detection of a boundary of an obstacle-free area, it may be provided that the processing unit is designed to

to calculate the boundary indicator value pixel by pixel by means of a neural network, in particular for all pixels in the same way and independently of the remaining pixels, and

to hand over to the neural network as input values the individual brightness or color values within an environment around the respective pixel, in particular given a predetermined relative position to the respective pixel, in particular given the same for each pixel.

For effectively training the neural networks to provide accurate limit indicator values, it may be provided that the processing unit is configured to

to create the neural network using a variety of predetermined training images, and

for each of the training images in each case to specify a training boundary curve and to train the neural network for the respective training image in the areas lying in the area of the training boundary curve To provide a limit indicator value other than the limit indicator values determined in the rest of the image area.

In order to ensure a safe movement of a user of a device according to the invention in an environment unknown to him, it can be provided that the processing unit is designed to determine the actual size and shape of the imaged obstacle-free area within the environment of the image recording unit,

due to the location of the obstacle-free area in the digital image,

such as

In order to make a device according to the invention for a person with limited vision and motor skills in many ways, it can be provided that the image pickup unit is fixedly mounted on a person or on an animal, especially in a shoe, glove, harness, helmet walking stick or Walker is arranged.

In a particularly energy-saving embodiment of the invention, in which, for example, the image acquisition unit does not have to be permanently supplied with voltage, it can be provided that the processing unit is designed to monitor the position and orientation of the image acquisition unit and to carry out further processing steps only

- When the image containing unit containing shoe is completely placed on the floor

and / or that the image acquisition unit is designed to create images only then

- When the image pickup unit is in a predetermined position, in which both the orientation of the horizontal image axis of the image pickup unit and the viewing direction of the image pickup unit are aligned approximately horizontally, in particular with an angular deviation of at most 30 ° about a horizontal axis, or - When the image containing unit containing shoe is completely placed on the floor.

For selectively starting the acquisition of digital images and determining parameters with the image acquisition unit, it may be provided that the image acquisition unit is mounted in a shoe, wherein the processing unit is adapted to determine the height of the image acquisition unit above the ground when the shoe is on the ground stands and

determine the actual size and shape of the imaged obstacle-free area provided that the image-capturing unit is at the determined height.

For particularly safe locomotion for persons with limited vision and limited motor skills in a foreign environment, it may be provided that the processing unit is designed to

- to identify, in addition to the limit indicator value, an obstacle indicator indicating the type of obstacle limiting the obstacle-free area, and

- To determine the obstacle indicator for each pixel on the boundary curve.

For a simple and precise provision of a distance to limit the obstacle-free area for a user, it can be provided that a device according to the invention comprises an ultrasound sensor arranged in the region of the image recording unit, wherein the ultrasound sensor

- Is aligned in the same direction as the image pickup unit, or

a detection area partially overlapping with the recording area of the image recording unit,

characterized in that the processing unit is adapted to

to determine a first distance of the limiting curve or at least one point on the boundary curve of the obstacle-free region from the image recording unit,

- In addition to recording the digital image by distance measurement using the ultrasonic sensor to determine a second distance, and

compare the second distance and the first distance of the obstacle-free area with each other and create another distance measurement using the first distance and the second distance, and

- Preferably set in each case deviations in the respective smaller distance as another distance measurement.

To display the determined parameters for a user while traveling in an environment unknown to him, it may be provided that an inventive Device comprises a feedback unit, wherein the feedback unit for, in particular wireless, data communication with the processing unit and for displaying the determined distance measured value and / or the determined obstacle indicator value and / or the actual size and shape of the imaged obstacle-free area for the person is formed ,

The invention further relates to a device which allows a person with impaired vision and motor skills to move independently in an unknown environment, as the device warns the person of obstacles. According to the invention, it is provided that the feedback unit is designed as a shoe with actuator elements, and that the processing unit is designed to

- to determine or specify a distance value corresponding to the size of the obstacle-free area around a person, and

- to control the feedback unit depending on this distance value,

to activate in the shoe of the person at least one of several sequentially in the direction of the more sensitive body area, in particular to the toe tip, arranged actuator elements, in particular vibration elements, to make the selection of the actuator depending on the distance value, and to select an actuator in a more sensitive body area, in particular all the more closer to the toe, the smaller the distance value.

Further advantages and embodiment of the invention will become apparent from the description and the accompanying drawings.

Exemplary embodiments of the invention are shown schematically in the drawings and will be described below by way of example with reference to the drawings.

Fig. 1 shows an environment around an image pickup unit. Fig. 2 shows a distribution of the boundary indicator values in an image of the environment. Fig. 2a shows a section of pixels from Fig. 2. Fig. 3 shows schematically the determined boundary curve in an image of the environment. Fig. 4 shows a shoe with actuator elements. Fig. 5 shows the column-by-column calculation of potentials by means of the Viterbi algorithm for the example in Fig. 2a. Fig. 6 shows the calculated potentials for the example in Fig. 2a. Fig. 7 shows the determined course of the limiting curve. Fig. 1 shows an environment 10 in the form of a room in which a person with limited vision wants to move. In one of the walls bounding the space, a window is arranged, wherein the window has a lower edge 12 of the window. Since the person's space is not known, she is dependent on aids that warn her of obstacles, such as two walls in the direction of the person's movement, and recognize, for example, the accessible obstacle-free area.

The person therefore wears a tool, such as a shoe 20, in which an image pickup unit 1 is arranged. The image recording unit 1 mounted in the shoe 20 is a watertight, robust camera, which is frequently used, for example, in sports. Such cameras are small enough that they can also fit in a shoe 20. In the exemplary embodiment shown in FIG. 1, the image recording unit 1 is arranged on the toe in the region of the toes of the wearer of the shoe 20 and aligned in the direction of movement of the wearer of the shoe 20.

In the exemplary embodiment shown in FIG. 1, in addition to the image recording unit 1, a processing unit 4, an energy store 3 and an ultrasonic sensor 2 are arranged in the shoe 20. The image recording unit 1 is designed to make recordings in the form of digital images available and to forward digital images D produced by it to the processing unit 4. The processing unit 4 is designed to process the digital images D transmitted to it and to carry out the inventive method steps described in more detail below. Of course, a plurality of ultrasonic sensors 2 can also be arranged in the shoe 20. Other devices required for carrying out a method according to the invention or for transmitting the detected obstacles, such as a feedback unit, may likewise be arranged in the shoe 20, or else carried by the person on the body or otherwise carried along with the aid independently of the aids.

In order to capture the image of an obstacle-free region which surrounds the image recording unit 1, at least one image of a subregion of the space is first created with the image recording unit 1. There is also the possibility to create several recordings and combine them into a single digital image D, in particular to select one from several different recordings and to create the digital image D based on this shot. Likewise, the digital image D can be created as a result of different intermediate processing steps such as in particular filtering, sharpening, color corrections etc from one or more recordings. Alternatively, these steps may be performed in the processing unit 4 described below.

The thus created digital image D is forwarded from the image acquisition unit 1 to the processing unit 4 and examined by the processing unit 4 for the presence of possible limitations B of the obstacle-free region. This boundary B of the obstacle-free area to be found in the digital image D corresponds in the space shown in FIG. 1 to the wall edge 11.

In order to check whether there is a boundary B of an obstacle-free area in the digital image D, a limiting indicator value Bl is determined by the processing unit 4 for individual pixels P, in particular for all pixels, of the digital image D. For this purpose, an environment around each pixel P is defined for the pixel-wise predetermined image data of the digital image D, within which the boundary indicator value Bl for the respective pixel P is determined. The limit indicator value Bl indicates the probability with which the image of the boundary B of an obstacle-free region is located within a given pixel environment.

In order to determine the areas where there is a boundary B that limits the walk-in area, a neural network is used to determine the boundary indicator value Bl in the embodiment shown. The neural network calculates pixel by pixel, in particular for all pixels P in the same way, the boundary indicator value Bl. As input values, the individual brightness or color values within an environment around the respective pixel P are transmitted to the neural network. The environment around the respective pixel P can advantageously have a square shape with an edge length of 200 pixels, in the center of which the respective pixel P is located. Of course, in the context of the invention, other environments may be used as input values for the neural network.

First, a probability distribution map is computed by means of the neural network, which includes a boundary indicator value Bl for each pixel P. The size of the probability distribution map essentially corresponds to the size of the digital image, in particular except for border areas. The Limiting indicator value Bl indicates a probability as to whether the searched boundary B of the obstacle-free area is imaged in the respective pixel P.

To create the neural network, a multiplicity of predefined training images are used in a training phase, a training boundary curve being predetermined for each of the training images and the neural network being trained for the respective training image in the area adjacent to the training boundary curve Pixels to provide a Begrenzungsindikatorwert Bl, which differs from the rest of the image area determined boundary indicator values Bl.

In order to construct the neural network to provide reliable bound indicator values Bl and to effectively recognize bounds B of the obstacle-free region, training images are used that include both digital images D of indoor and outdoor environments 10, in which various types of indoor environment Limit B of an obstacle-free area are shown. The illustrated boundaries B may be, for example, steps, sidewalk edges, walls or other boundaries B.

For the training of the neural network, digital images D of outdoor environments 10 are also used in different weather conditions. In order to increase the effectiveness of the training of the neural network, additional image transformations such as rotation, horizontal reflection, contrast change, brightness change and blur are applied.

FIG. 2 shows a schematic exemplary representation with a significantly reduced number of pixels for boundary value indicator values Bl determined by the neural network, in each case for the surroundings around pixels P of a section of an image of the space shown in FIG. In this case, white areas correspond to pixels P for which a small limit indicator value Bl has been determined, ie that they contain the image of a boundary of the obstacle-free area only with low probability. Long-hatched areas correspond to pixels P for which a higher limit indicator value Bl has been determined, so that these pixels P are more likely to contain the image of a boundary B. The highest boundary indicator value Bl has been determined in FIG. 2 for longitudinally and crosshatched regions, so that these pixels P are most likely to image a boundary B. As the next method step, a coordinate parameterization with two parameters is specified for the digital image, which corresponds in particular to the pixel coordinates, for example in the form of an x and y coordinate. In the schematic section of the image of the space shown in Fig. 2, the x-coordinate indicates the column number and increases from the left edge of the picture to the right, and the y-coordinate indicates the line number and decreases from the upper edge of the picture.

However, this parameterization is not mandatory. Alternatively, for example, another coordinate system can be selected in which, starting from a defined camera point, lines of the same distance to the camera point in the form of layer lines in the digital image D are predefined.

In particular, in this parameterization it is also not mandatory for the individual coordinate lines thus specified to exist directly in pixels P, i. in pixels for which limit indicator values are present immediately. Rather, it is also possible for the pixels specified by the parameterization to also specify and use limiting indicator values BI created by interpolation from nearby pixels P.

Subsequent to the determination of the limit indicator values Bl, a limiting curve BK of the obstacle-free region, in particular in the form of a path train, is determined by the processing unit 4 as a curve. In this case, the limiting curve BK is selected such that the image pixels are located on it which have a high limiting indicator value Bl compared to pixels in their surroundings. This means that the limiting curve BK is not necessarily selected to run through individual pixels P, but can also be selected to run through pixels interpolated between a plurality of pixels P.

The image of the obstacle-free area, which is safely accessible to the person, is thus determined by the limiting curve BK and optionally further predetermined curves or straight lines, in particular the boundary edges of the digital image D. In the exemplary embodiment shown, to determine the limiting curve BK, the path train is selected such that it runs through a number of pixels P of the digital image D. In this case, in each case a pixel, in particular a pixel P, is selected for a number of predefined first parameters, in particular for each image column of the digital image D or for a number of image columns each having a predetermined x coordinate, through which the path extends. As can be seen in FIG. 2, it is also possible that for two or more pixels P, for example with the same x coordinate, an identically high limit indicator value is determined. In FIG. 2, this is the case, for example, by the window edge 12 likewise depicted in the image of the room in the upper left region of the probability distribution map.

In order to reduce such possible ambiguities in the probability distribution map estimated by the neural network and to find the most probable limitation of the obstacle-free area, an optimization method is applied by the processing unit 4 in the exemplary embodiment shown. In order to determine the limiting curve BK, a respective dimension value to be optimized is determined in each case for individual pixels on the respective limiting curve BK for individual individual boundary curves BK, in particular for a large number of pixels each having predetermined first parameters or x-coordinates. The dimension value consists of the following partial dimension values:

- the limit indicator value Bl, as well as

the difference between the second parameter in the respective point of the second parameter in an adjacent point lying on the same boundary curve BK, the first parameter of which deviates by a predetermined value, and in particular lies in an adjacent column of the digital image D, or a value derived from this difference ,

Since large jumps in the boundary curve BK between two adjacent columns are mostly due to uncertainties in the probability distribution map, it can be determined, for example, by defining transition probabilities between adjacent columns, which possible boundary curve BK is to be preferred. With such transition probabilities can be selected, how much the resulting limiting curve BK should be pulled through, or in how far jumps are allowed.

Alternatively, for example, obstacles closer to or closer to the person may be rated higher than those further from the person and in the background. Alternatively, possible Limits B, which are in the lower region of the digital image D, or a solid boundary B without jumps, are preferred.

For the concrete determination of an optimal boundary curve BK with respect to the aforementioned criteria, it is possible, for example, to use the Viterbi algorithm, which is described, for example, in Viterbi, A., Error bounds for convolutional codes and asymptotically optimum decoding algorithm. In: IEEE Transactions on Information Theory. Vol. 13, No. 2, 1967, pp. 260-269.

The potential Φ to be maximized by the Viterbi algorithm for determining the profile of the limiting curve BK in the digital image D is composed of:

the output of the neural network, ie the determined boundary indicator values Bl or the probabilities that a pixel P contains a boundary B of an obstacle-free area, weighted with a weighting factor ^vl n,

a value depending on the height of the pixel P in the digital image D, for example by its y-coordinate or a value derived therefrom. This

Value is weighted with a weighting factor.

a value which depends on the neighborhood relations of the individual points on the boundary curve <h {Zi, Vi> Xi-fi ^; Vi + i), weighted with a weighting factor

In this case, x, or y, respectively denotes the x-coordinate or the y-coordinate of the ith point P, on the limiting curve BK. A bounding curve with the points Pi or coordinates xi, yi is searched for, for which the following sum expression Φ is maximal for a limiting curve:

Φ = ν> _η φ _η ί Χι, //,) + ^ ll <ph (Xi> Vi) + ^ Vi, X-i + l ?Ji-rl}

In the following, a possible implementation of the determination of the course of the limiting curve BK by means of the Viterbi algorithm is explained in a greatly simplified example with reference to the pixels marked in FIG. 2 and enlarged in FIG. 2a. Typically, the Viterbi algorithm is used to determine the boundary curve BK on the entire image.

The highly simplified example shown (FIG. 2 a) contains a section of the digital image D with columns x = 1... 4, and lines y = 1. 3, of pixels, wherein the Number of columns in counting to the left increases and the number of lines is increased in the count down. For the marked pixels in Table 1, the limit indicator values Bl determined by the neural network are compiled in a tabular form.

Table 1: Determined bound indicator values Bl of the selected pixels.

In determining the course of the boundary curve BK based on complete digital images D, the number of columns and rows may range from 1000 to 5000. In the example shown, the weighting factors or functions listed in Table 2 are specified:

Table 2: Weighting factors and functions used in the example.

To determine the course of the limiting curve BK, the following computations are first carried out pixel by pixel for the column x = 1, which are represented, for example, for the pixel in the row y = 1. In this case, the sum of the limit indicator value Bl = 1, 0 multiplied by the predetermined weighting factor

= 1, 0 and the height y (P ₁ ) = y ₁ = 1 multiplied by the weighting factor ^'1 h = 0.25 for the height formed so that the result is 1, 25. This calculation is then performed in the same way for all pixels P of the column x, = 1. The calculated results for the column x = 1 are shown in FIG.

For the further columns, the following procedure is selected pixel-by-pixel for all pixels P, which is illustrated in more detail by way of example for the pixel in the row y = 1 of the column x = 2; the determination is made in an analogous manner for the remaining pixels or columns. Among all the pixels of the respective preceding column x, = 1, in each case the determined potential is in each case the magnitude of the difference of the y-coordinate values of the respective pixels

| y (Pi ₊ i) -y (Pi) |, weighted by the weighting factor for the neighborhood ^w l ^! = -0.5 added (Figure 5). The maximum value from the respective column is determined under these pixel-by-pixel calculated values. For example, this results in the pixel at the position y = 1 is the maximum value taking into account the pixel at position x, = 1, y = 3.

The limit indicator value Bl for the relevant pixel as well as the height, respectively, are added to the determined maximum value, in each case with their associated weightings, as already described with respect to the pixels of the first row, in order to form the potential value for the relevant pixel: Potential value Φ for the pixel at the position x _{i + 1} = 2, y = 1 results:

4> (P _{i +} i) = max i {Φ (Ρι) + w _b | y (P _{i +} i) -y (Pi) |} + w _h y (P _{i + 1} ) + w _n Φ _π (P _{i +1} )

In this case, instead of the y-coordinate value y (P _{i + 1} ), it is also possible to use a value derived therefrom, for example the result f (y (P _{i + 1} )) of the evaluation of a function f of the y-coordinate value y ( P _{i +} ).

FIG. 6 shows the individual calculated potential values. Likewise, those pixels whose values are used for the determination of the potentials in the respectively following pixels are shown linked to them by arrows, for example the potential value of the pixel becomes x = 1, y = 3 for the determination of the potential value of the pixel used at the point x = 2, y = 2.

The last column looks for the pixel with the highest potential. Starting from this pixel, the pixel which was used in the calculation of the pixel of the potential is now searched backwards, that is to say, according to the decreasing column number.

In Fig. 7, the largest potential in the pixel was found at the position x = 4, y = 2. For the determination of this pixel, the pixel at position x = 3, y = 2 was used because of this, among the pixels in the third column (x = 3), the largest potential component (9.25) results. Starting from this pixel, the pixel in the second column (x = 2) which was used in the calculation of the potential of this pixel is now searched for. This is the pixel at position x = 2, y = 2. Finally will from this pixel, look for the pixel in the first column (x = 1) used in the calculation of the potential of that pixel. It is the pixel at position x = 1, y = 3.

The pixels thus determined are used in the framework of the Viterbi algorithm for determining the curve profile of the limiting curve BK.

In the example shown, it is taken into account as further specifications in the determination of the curve progression that the limiting curve BK has no abrupt jumps and at low altitude in the digital image D, i. should run at high y-coordinates.

For this purpose, for calculating a value derived from the y-coordinate specification for determining the potential of the neighborhood, for example, functions may be used in which small deviations are neglected, such that, for example, absolute differences which are smaller than a predefined threshold value, for example 5, are neglected. This also makes it possible to achieve that the determined limit indicator curve BK has no rapid changes or jumps. In addition, other or further specifications for the determination of the curve can be made.

3 schematically shows the result of the application of such an optimization method for determining the limiting curve BK on the basis of the boundary indicator values Bl determined by the neural network for the pixels P of the section of the image of the environment 10 or of the space shown in FIG. In determining the boundary curve BK shown in FIG. 3, the path length was selected to pass through a number of pixels P of the digital image D.

In this case, with the optimization method described above for each image column of the digital image D, each having a predetermined x coordinate, one pixel each, in particular a pixel P, is selected which is located far down in the image in order to allow the line to pass through. In this way, the lower window edge 12, which is still partly provided by the neural network with limiting indicator values Bl in FIG. 2, can effectively be eliminated as the boundary B of an obstacle-free region.

This procedure is also particularly advantageous, for example, if the user of the invention uses them when climbing stairs, since in this case always the level closest to him is first identified so that he is effectively routed from one level to the next.

After first determining the obstacle-free area boundary B, the actual size and shape of the imaged obstacle-free area within the environment 10 of the image acquisition unit 1 is then determined by the processing unit 4 to determine to the user of the method in which area he is safely traveling can, without encountering obstacles. In the created digital image D corresponds to the size of the imaged obstacle-free area the freely accessible space that extends between the image pickup unit 1 and the wall edge 1 1 in Fig. 1.

In order to determine the shape and size of the obstacle-free region, the position of the obstacle-free region in the digital image D, the relative position, in particular height and orientation, of the image recording unit 1 with respect to the environment 10 and optionally parameters of the image recording unit 1, in particular resolution and focal length, are used. Furthermore, in order to determine the shape and size of the obstacle-free region, it is assumed that the obstacle-free region detected in the digital image D lies on a predetermined three-dimensional surface, in particular on a plane.

In the exemplary embodiment, the position and orientation of the image recording unit 1 is advantageously monitored, so that only images are produced by the image recording unit 1 or method steps according to the invention are carried out when the image recording unit 1 is in a predetermined position. As a predetermined position, for example, both a position can be defined in which both the orientation of the horizontal image axis of the image pickup unit 1, and the viewing direction of the image pickup unit 1 are aligned approximately horizontally, in particular with an angular deviation of at most 30 ° about a horizontal axis, or Also, a layer in which the image pickup unit 1 containing shoe 20 is completely placed on the ground.

In addition, in the embodiment, the height of the image pickup unit 1 mounted in the shoe 20 above the ground occupied by the image pickup unit 1 when the shoe 20 stands on the ground is determined. In this case, for example, the actual size and shape of the imaged obstacle-free area is determined only on condition that the image acquisition unit 1 is at the determined height. Thus is the height can be used as a further criterion for an energy-saving, efficient procedure.

For example, when the person wearing the shoe 20 with built-in image acquisition unit 1 rests on a lying surface, the person's feet and thus their shoes are no longer in a seated position and the viewing direction of the image pickup unit 1 is compared to the predetermined one Position rotates. In this case, a creation of digital images D is not required because the person does not want to move and the image acquisition unit 1 does not need to be powered by the energy storage device 3.

For example, if the shoe 20 with the image pickup unit 1 mounted therein has no contact with the ground and thus is not at the determined height, then no digital images D are advantageously produced by the image pickup unit 1. This is the case, for example, when the person wearing the shoe 20 raises his or her feet while walking, or when the person is resting with his legs up.

Thus, it can be ensured that the image acquisition unit 1 does not permanently consume voltage for the production of digital images D, but only when the image acquisition unit 1 is in a position in which the implementation of the method according to the invention is provided, i. that the person wearing the shoe 20 actually requires assistance in locomotion. In this case, it is also ensured at the same time that the size or shape of the obstacle-free area or the distance to potential obstacles can be reliably determined.

As an improvement of the embodiment shown, in addition to determining the size and shape of the obstacle-free region, a first distance A1 of the boundary curve BK or at least one point on the boundary curve BK of the obstacle-free region can be determined by the image acquisition unit 1. From the known position of the image pickup unit 1 and assuming that the person wearing the shoe 20 with integrated image pickup unit 1 is standing on a plane, the distance from the two-dimensional image representing the digital image D taken by the image pickup unit 1 can be obtained become an obstacle. In addition to the acquisition of the digital image D and the evaluation with respect to the first distance A1, a second distance A2 is determined by measuring the distance by means of ultrasound. In the exemplary embodiment in FIG. 1, an ultrasonic sensor 2 used for this purpose is arranged in the shoe 20 near the image recording unit 1 and aligned in the same direction as the image recording unit 1, so that the recording area of the image recording unit 1 partially overlaps with the detection area of the ultrasonic sensor 2.

Subsequently, the first distance A1 and the second distance A2 of the obstacle-free area are compared with each other, and another distance measurement value A is created using the first distance A1 and the second distance A2. In the case of deviations between the first distance A1 and the second distance A2, in each case the smaller distance is preferably regarded as a further distance measurement value A. In Fig. 1 such determined distance lines are located in determined distances of 10 cm, 50 cm, 100 cm and 150 cm to the image pickup unit 1.

This approach is particularly advantageous because, for example, obstacles made of glass such. B. glass doors or floor-deep glass windows or transparent partitions of an image pickup unit 1 may be insufficiently recorded under certain circumstances, so that they are not recognized correctly in the determination of the boundary curve BK means of the neural network NN. However, in this case, the ultrasonic sensor 2 correctly detects the distance to the obstacle made of glass. The fact that the lower of the two distances A1, A2 determined by different methods is taken as a further distance measurement A offers additional security for the person who uses the invention in locomotion, since the danger of encountering an obstacle is made more difficult by this distance selection. is reduced.

In all embodiments of the invention may further be provided that in addition to the Begrenzungsindikatorwert Bl and an obstacle indicator Hl is determined. The obstacle indicator value H1 indicates the type of obstacle limiting the obstacle-free area. The processing unit 4 or the neural network NN is designed to determine the obstacle indicator H1 for individual pixels on the boundary curve BK and preferably to represent the person. In this way, the person wearing the shoe 20 can be notified, which obstacles or objects such. As sidewalk edges, walls, people, animals or the like, are in their direction of movement, so that they get the opportunity to respond accordingly. In all embodiments of the invention, alternatively or additionally, it can be provided that the at least one receptacle is accommodated with an image recording unit 1 mounted permanently on a person or on an animal, in particular in a glove, belt, helmet, walking stick or walking aid. It is thus possible, for example, for frail persons who are dependent on walking aids, such as, for example, crutches or walking sticks, to mount the image recording unit 1 in crutches.

Alternatively, for example, shoes 20 or collars for assistance dogs can be equipped with such an image recording unit 1, so that persons with impaired vision and reduced motor skills can move particularly safe in an unknown environment, as they assistance from their assistance dog and additionally inventively determined hints and warnings receive.

In all embodiments of the invention, the results obtained, such as the shape and size of an obstacle-free area around the image-capturing unit 1 or the distance of the image-capturing unit 1 to potentially in-progress obstacles, are perceptibly displayed to the user of the invention. The processing unit 4 is designed to forward the determined results, for example, to a feedback unit.

The user may, for example, wear the feedback unit on the body, so that the results obtained can be transmitted to him in a manner that can be perceived by him. For example, if the user of the invention is a person with impaired vision, the feedback unit may be, for example, headphones used to carry the person upside down.

The processing unit 4 is designed to transmit the determined results on the shape and size of the accessible obstacle-free area to the feedback unit and to control the feedback unit such that, for example, as the distance to the boundary B of the obstacle-free area decreases, the sound sequences are louder or higher As soon as the sound is heard, it will be delivered at a speeding rhythm, so that the person can see that it is approaching an obstacle.

It may, for example, also be provided that, depending on whether in which direction an obstacle is in relation to the person, ie left or right of the person or just in front of the person, only sounds are applied to the left or right ear or to both ears of the person to alert them to the location of the obstacle.

4 shows a shoe 20 with an image pickup unit 1, an ultrasound sensor 2, an energy store 3, a processing unit 4, and actuator elements 5 mounted therein. The image pickup unit 1 and the ultrasound sensor 2 are installed in the toe, so that both are aligned in the same direction , The energy storage 3 and the processing unit 4 are arranged in the embodiment in paragraph 21 of the shoe 20. Six actuator elements 5 are arranged in the region of the toes or the footwear of the wearer of the shoe 20 in the shoe sole region 22, wherein the actuator elements 5 are arranged such that they form a line extending from the ball of the foot in the direction of the toe. In the exemplary embodiment, the actuator elements 5 are designed as vibrating elements which vibrate when they are actuated, for example, by the processing unit 4. The actuator elements may, for example, also be designed in the form of a pin and, when activated, press upwards into the foot.

First, a distance corresponding to the size of an obstacle-free area around the image pickup unit 1 arranged in the shoe tip, or a person who carries the shoe 20, is determined by the processing unit 4. For determining the distance value, at least one digital image D and / or the ultrasonic sensor 2 created by the image recording unit 1 is used.

In order to make the determined distance for the person detectable, at least one of a plurality of actuator elements 5 arranged sequentially in the direction of the more sensitive body region, in particular towards the toe, is activated depending on the distance value at at least one body location in the shoe 20 of the person. The selection of the actuator 5 is made depending on the distance value. As the distance to an obstacle becomes smaller, actuator elements 5 are selected in an all the more sensitive body region, in particular the closer to the tip of the toe.

This means that, at a comparatively large distance to an obstacle, first the actuator element 5 is actuated in the area of the person's footing. If the distance to the obstacle becomes smaller, starting from the actuator element 5 in the area of the football, the actuator element 5 located closer to the toe tip is actuated until finally, at a short distance to the obstacle, the actuator element in FIG Area of the person's toes is chosen. This signals to the person that he / she is in danger of encountering an obstacle should it continue to move in the direction it was headed.

However, the invention can not only be used to facilitate the movement in everyday life for persons with limited vision and / or reduced motor skills. Alternatively, the invention can also be used to detect an obstacle-free area around an image pickup unit 1 arranged, for example, in the area of the bumper of a vehicle, and to determine the shape and size of the freely passable area in the direction of travel of the vehicle. This may be of interest, for example, in the area of autonomous driving in order to determine the trafficable environment, or to detect the distance to a preceding vehicle and, for example, to regulate the speed of the autonomously driving vehicle, depending on the distance to the vehicle in front.

Claims

claims:

1 . A method of detecting the image of an obstacle-free area surrounding an image pickup unit (1)

a) with the image recording unit (1) at least one image of a portion of the environment (10) of the image recording unit (1) is created and based on this at least one recording a digital image (D) is created, characterized b) that the digital image thus created (D) is examined for the presence of possible boundaries (B) of an obstacle-free area, wherein for each pixel (P) of the digital image (D) in each case a limit indicator value (Bl) depending on the pixel-wise predetermined image data of the digital image (D) within an environment of respective pixel (P) is determined which indicates with which probability the image of the boundary (B) of an obstacle-free area is located within a given pixel environment,

c) that for the digital image (D) a coordinate parameterization with two parameters is specified, which corresponds in particular to the pixel coordinates,

d) that a boundary curve (BK) of the obstacle-free region, in particular in the form of a train, is determined as a curve on which there are pixels which have a high limiting indicator value (Bl) compared with pixels in their surroundings,

e) that the image of the obstacle-free area by the limiting curve (BK) and possibly further predetermined previously curves or lines, in particular the boundary edges of the digital image (D) is set.

2. The method according to claim 1, characterized in that as a limiting curve (BK) a stretch of tracks is selected, which runs through a number of pixels (P) of the digital image (D), wherein for a number of predetermined first parameters, in particular for each image column of the digital image (D) or for a number of image columns each having a predetermined x-coordinate, in each case one pixel, in particular a pixel (P), is selected, through which the path extends.

3. The method according to claim 1 or 2, characterized in that for the determination of the limiting curve (BK) in step d) an optimization method is performed, in which for individual candidate boundary curves (BK) in each case for individual pixels on the respective limiting curve (BK ), in particular for a plurality of pixels each having predetermined first parameters or x-coordinates, one each value to be optimized, which is composed of the following partial values,

- the limit indicator value (Bl), as well as

the difference between the second parameter in the respective point of the second parameter in an adjacent point located on the same boundary curve (BK), the first parameter of which deviates by a predetermined value, and in particular lies in an adjacent column of the digital image (D), or one of value derived from this difference.

4. The method according to any one of the preceding claims, characterized in that for the determination of the limiting curve (BK) in step d) an optimization method is performed, in which for each eligible boundary curves (BK) in each case those pixels on the respective limiting curve (BK) , in particular for a multiplicity of picture elements, each having predetermined first parameters or x-coordinates,

the highest possible limit indicator values (B1) and / or compared to other pixels, in particular with the same given first parameter or the same x-coordinate

having a second parameter, in particular a y-coordinate, which corresponds approximately to the second parameter of the respective neighboring point on the limiting curve or has only slight difference to this second parameter, in particular so that the position of the pixels results in a limiting curve (BK), whose change in the direction of the second parameter, in particular the y-coordinate, is as small as possible.

5. The method according to any one of the preceding claims, characterized in that the boundary indicator value (Bl) by means of a neural network, in particular for all pixels (P) in the same manner and independently of the remaining pixels (P) is calculated, the neural Network (NN) as input values the individual brightness or color values within a, in particular for each pixel (P) equal predetermined environment around the respective pixel (P), in particular with a predetermined relative position to each pixel (P) receives.

6. The method according to claim 5, characterized in that the neural network is created using a plurality of predetermined training images, wherein for each of the training images in each case a training boundary curve is predetermined and the neural network is trained on it, for the respective training image, in to provide the pixels lying in the region of the training boundary curve with a limiting indicator value (Bl) which deviates from the limiting indicator values (B1) determined in the rest of the image region.

Method according to one of the preceding claims, characterized in that the actual size and shape of the imaged obstacle-free area is determined within the environment (10) of the image-recording unit (1),

due to the location of the obstacle-free area in the digital image (D),

due to the relative position, in particular height and orientation, of the image recording unit (1) with respect to the environment (10),

optionally parameters of the image acquisition unit (1), in particular resolution and focal length,

such as

- assuming that the obstacle-free area detected in the digital image (D) lies on a given three-dimensional surface, in particular on a plane.

8. The method according to any one of the preceding claims, characterized in that the at least one recording with a fixedly mounted on a person or on an animal, in particular in a shoe (20), glove, harness, walking stick or walking aid arranged, image pickup unit is added ,

9. The method according to claim 8, characterized in that the position and orientation of the image recording unit is monitored and the image recording unit (1) only creates images or steps b) to e) of claim 1 are carried out only

- When the image pickup unit (1) is in a predetermined position, in which both the orientation of the horizontal image axis of the image pickup unit (1) and the viewing direction of the image pickup unit (1) approximately horizontally, in particular with an angular deviation of at most 30 ° about a horizontal axis , are aligned, or

- When the image pickup unit (1) containing shoe (20) is completely placed on the ground.

10. The method according to claim 8 or 9, characterized in that the digital images are recorded with an in a shoe (20) mounted image pickup unit (1), wherein the height of the image pickup unit (1) above the ground is determined when the shoe (20) is on the ground and

the actual size and shape of the imaged obstacle-free area according to claim 6 is determined on condition that the image pickup unit (1) is at the determined height.

1 1. Method according to one of the preceding claims, in particular according to one of claims 8 to 10, characterized in that in step b) in addition to the Begrenzungsindikatorwert (Bl) and an obstacle indicator is determined which indicates the type of obstacle from the obstacle-free area is limited and

the obstacle indicator is determined for individual pixels on the limiting curve (BK) and preferably displayed to the person.

12. The method according to any one of the preceding claims, characterized

- That a first distance (A1) of the limiting curve (BK) or at least one point on the boundary curve (BK) of the obstacle-free area of the image pickup unit (1) is determined,

- That in addition to recording the digital image (D) by distance measurement by means of ultrasound, a second distance (A2) is determined, wherein an ultrasonic sensor used for this purpose (2) in the region of the image pickup unit (1) is arranged and

- Is aligned in the same direction as the image pickup unit (1), or

has a detection area partially overlapping with the recording area of the image recording unit (1),

- That the second distance (A2) and the first distance (A1) of the obstacle-free area are compared with each other and another distance measurement (A) using the first distance (A1) and the second distance (A2) is created, preferably in case of deviations in each case the respective smaller distance is regarded as a further distance measured value (A).

13. Method, in particular according to one of the preceding claims, characterized in that a size of the obstacle-free area corresponding distance value is determined or specified by a person, and then depending on this distance value at least one body location, in particular in at least one shoe (20). in which the person of at least one of several sequentially in the direction of the more sensitive body area, in particular towards the toe tip, arranged actuator elements (5), in particular vibration elements is activated, wherein the selection of the actuator elements (5) is made depending on the distance value and wherein an actuator element (5) is selected in a body region that is all the more sensitive, in particular, the closer the foot tip is selected, the smaller the distance value is.

14. An apparatus for detecting the image of an obstacle-free area surrounding an image recording unit (1) comprising an image recording unit (1) and a processing unit (4) connected downstream of the image recording unit (1),

wherein the image recording unit (1) is designed to produce at least one image of a subregion of the environment (10) of the image recording unit (1) and to transmit the image to the processing unit (4), and

wherein the processing unit (4) is designed to

to create a digital image (D) based on this at least one image and to examine the digital image (D) thus created for the presence of possible boundaries (B) of an obstacle-free region,

- For each pixel (P) of the digital image (D) in each case a boundary indicator value (Bl) depending on the pixel-wise predetermined image data of the digital image (D) within an environment of each pixel (P) to determine which indicates the probability within a predetermined pixel environment is the image of the boundary (B) of an obstacle-free area,

for the digital image (D) to specify a coordinate parameterization with two parameters, which corresponds in particular to the pixel coordinates,

- To determine a boundary curve (BK) of the obstacle-free area, in particular in the form of a train, as a curve on which there are pixels that - compared with pixels in their environment - have a high limiting indicator value (Bl), and

- Define the image of the obstacle-free area by the limiting curve (BK) and possibly further predetermined curves or lines, in particular the boundary edges of the digital image (D).

15. The device according to claim 14, characterized in that the processing unit (4) is designed to

to choose a path of travel as the limiting curve (BK), the path of the route passing through a number of pixels (P) of the digital image (D), and

for a number of predefined first parameters, in particular for each image column of the digital image (D) or for a number of image columns each having a predetermined x coordinate, in each case one pixel, in particular a pixel (P), through which the path extends ,

16. The apparatus of claim 14 or 15, characterized in that the processing unit (4) is adapted to perform the determination of the limiting curve (BK) an optimization method, wherein the processing unit (4) is adapted for individual eligible boundary curves ( BK) in each case for individual pixels on the respective boundary curve (BK), in particular for a multiplicity of pixels each having predetermined first parameters or x coordinates, respectively to determine a dimension value to be optimized, which is composed of the following partial values:

- the limit indicator value (Bl), as well as

17. Device according to one of claims 14 to 16, characterized in that the processing unit (4) is adapted to perform to determine the limiting curve (BK) an optimization method, wherein the processing unit is adapted to individual for individual limiting curves (BK ) in each case select those pixels on the respective boundary curve (BK), in particular for a multiplicity of pixels each having predetermined first parameters or x coordinates,

- Have the highest possible boundary indicator values (Bl) and / or the highest possible values of the second parameter, in particular the y-coordinate, in comparison to other pixels, in particular with the same predetermined first parameter or the same x-coordinate, and / or

- Which have a second parameter, in particular a y-coordinate, which corresponds approximately to the second parameter of the respective neighboring point on the boundary curve or has only slight difference to this second parameter, so that in particular from the position of the pixels results in a limiting curve (BK), whose change in the direction of the second parameter, in particular the y-coordinate, is as small as possible.

18. Device according to one of claims 14 to 17, characterized in that the processing unit (4) is adapted to

to calculate the boundary indicator value (Bl) pixel by pixel by means of a neural network, in particular for all pixels (P) in the same way and independently of the remaining pixels (P), and

- To the neural network as input values, the individual brightness or color values within a, in particular for each pixel (P) equal predetermined environment around the respective pixel (P), in particular with predetermined relative position to each pixel (P) to pass.

19. Device according to one of claims 14 to 18, characterized in that the processing unit (4) is adapted to

for each of the training images in each case to specify a training boundary curve and to train the neural network to supply a limiting indicator value (Bl) for the respective training image in the areas lying in the area of the training boundary curve, which is determined by the limiting indicator values determined in the rest of the image area (Bl) deviates.

20. Device according to one of claims 14 to 19, characterized in that the processing unit (4) is adapted to determine the actual size and shape of the imaged obstacle-free area within the environment (10) of the image pickup unit (1),

due to the location of the obstacle-free area in the digital image (D),

such as

21. Device according to one of claims 14 to 20, characterized in that the image recording unit (1) is fixedly mounted on a person or on an animal, in particular in a shoe (20), glove, harness, helmet walking stick or walker is arranged.

22. The device according to claim 21, characterized in that the processing unit (4) is adapted to monitor the position and orientation of the image pickup unit (1) and to carry out further processing steps only

- When the image pickup unit (1) containing shoe (20) is completely placed on the ground

and / or that the image acquisition unit (1) is designed to create images only then

Device according to claim 22, characterized in that the image pickup unit (1) is mounted in a shoe (20), wherein the processing unit (4) is adapted to detect the height of the image pickup unit (1) above the ground when the Shoe (20) is standing on the floor and

determine the actual size and shape of the imaged obstacle-free area on condition that the image pickup unit (1) is at the determined height.

24. Device according to one of claims 14 to 23, characterized in that the processing unit (4) is adapted to

- to determine, in addition to the limit indicator value (BI), an obstacle indicator indicating the type of obstacle limiting the obstacle-free area, and

- For each pixel on the limiting curve (BK) to determine the obstacle indicator.

25. Device according to one of claims 14 to 24 comprising a in the region of the image pickup unit (1) arranged ultrasonic sensor (2), wherein the ultrasonic sensor (2)

- Is aligned in the same direction as the image pickup unit (1), or

characterized in that the processing unit (4) is designed to

to determine a first distance (A1) of the boundary curve (BK) or at least one point on the boundary curve (BK) of the obstacle-free area from the image recording unit (1),

- In addition to recording the digital image (D) by distance measurement by means of the ultrasonic sensor (2) to determine a second distance (A2), and

- Compare the second distance (A2) and the first distance (A1) of the obstacle-free area with each other and create a further distance measurement (A) using the first distance (A1) and the second distance (A2), and

- Specify in each case in deviations respectively the respective smaller distance as a further distance measurement value (A).

26. Device according to one of claims 14 to 25, characterized by a feedback unit, wherein the feedback unit for, in particular wireless, data communication with the processing unit (4) and for displaying the determined distance measured value and / or the determined obstacle indicator value and / or the actual size and shape of the imaged obstacle-free area is designed for the person.

27. Device, in particular according to one of claims 14 to 26, preferably a shoe (20), comprising a feedback unit with actuator elements (5) and a processing unit (4) which is designed to

- to control the feedback unit depending on this distance value,

in the shoe (20) of the person, activating at least one of several actuator elements (5), in particular vibration elements, arranged sequentially in the direction of the more sensitive body region, in particular towards the foot tip, making the selection of the actuator elements (5) dependent on the distance value, and an actuator element (5) to select in a more sensitive body region, in particular, the closer to the tip of the toe, the smaller the distance value.