DE102016207209A1 - STIXEL ESTIMATION AND SEGMENTATION OF THE TRANSPORT PROCESS USING "DEEP LEARNING" - Google Patents

Abstract

Methods and systems are available to recognize an object in an image. In one embodiment, a method comprises: receiving, via a processor, from a single sensor, the data representing an image; the division of the image by the processor into vertical partial images; the processing of the vertical part images by the processor based on deep learning models; and the recognition of an object by the processor based on the processing.

Description

TECHNICAL PART

The technical field generally refers to object recognition systems and methods. In particular, it involves object recognition systems and procedures that recognize objects based on the so-called "deep learning" ("profound or profound learning experience").

BACKGROUND

Different systems process data to detect objects near the system. For example, some vehicle systems detect objects in the vicinity of the vehicle and use the information about the object to alert the driver to the object and / or control the vehicle. The vehicle systems recognize the object based on the sensors, which are distributed throughout the vehicle. For example, multiple cameras are mounted behind, on the sides and / or the front of the vehicle to detect objects. Images from multiple cameras are used to recognize the object based on stereo vision. Mounting multiple cameras in a vehicle or any system increases the overall cost.

Accordingly, it is desirable to provide methods and systems that recognize objects in an image from a single camera. Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

SUMMARY

Methods and systems are available to recognize an object in an image. In one embodiment, a method includes: receiving, via a processor, from a single sensor, the data representing an image; the division of the image by the processor into vertical partial images; the processing of the vertical part images by the processor based on deep learning models; and the recognition of an object by the processor based on the processing.

In one embodiment, a system includes a non-transitory computer-readable medium. The non-transitory computer-readable medium includes a first computer module that receives data from a single sensor via a processor, the data representing an image. The non-transitory computer-readable medium includes a second computer module that divides the image into vertical sub-images via the processor. The non-transitory computer-readable medium includes a third computer module that processes, via the processor, the vertical sub-images based on deep-learning models, and that detects an object through the processor based on the processing.

DESCRIPTION OF THE DRAWINGS

The embodiments will be described below in conjunction with the following figures (drawings), wherein like numerals denote like elements, and wherein:

1 FIG. 11 is an illustration of a vehicle incorporating an object recognition system in accordance with various embodiments; FIG.

2 FIG. 11 is a data flow diagram illustrating an object recognition module of the object recognition system in accordance with various embodiments; FIG.

3 Figure 12 is an illustration of a deep learning model in accordance with various embodiments;

4 - 6 Fig. 10 are illustrations of image scenes in accordance with various embodiments; and

7 FIG. 10 is a flowchart illustrating an object recognition method that may be performed by the object recognition system in accordance with various embodiments.

DETAILED DESCRIPTION

The following detailed description is by way of example only and is not intended to limit the application and uses. Furthermore, there is no intention to be bound by any theory expressed or implied in the preceding technical field, background, brief summary or the following detailed description. It should be noted that the same reference numbers refer to the same or corresponding parts and features throughout the drawings. The term "module" as used herein refers to an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group processor), and a memory containing one or more software or firmware programs, a combinatorial logic circuit, and / or others performs suitable components that provide the described functionality.

Here with reference to 1 shows a vehicle 10 , which is an object recognition system 12 in accordance with various embodiments. As can be seen, the shown and described object recognition system 12 be implemented in different systems, including non-mobile platforms or mobile platforms, such as: As automobiles, trucks, buses, motorcycles, trains, ships, aircraft, rotorcraft and the like. Exemplary is the disclosure in the context of the object recognition system 12 implemented in the vehicle 10 discussed. Thus, while the schematic diagrams herein depict exemplary arrangements of elements, additional intervening elements, devices, functions, or components may be present in a current embodiment. It must also be noted that 1 merely illustrative and may not be drawn to scale.

The object recognition system 12 includes a single sensor 14 , the object recognition module 16 assigned. As shown, the single sensor captures 14 observable conditions near the vehicle 10 , The single sensor 14 can be any sensor of observable conditions near the vehicle 10 feels, for. As a camera, a lidar, a radar, etc. As an example, the disclosure is discussed in a context in which the individual sensor 14 A camera is the visual images of a scene outside the vehicle 10 generated.

The single sensor 14 can be anywhere within or outside the vehicle 10 be attached, including at the front of the vehicle 10 , the left side of the vehicle 10 , the right side of the vehicle 10 and the back of the vehicle 10 , As can be seen, several individual sensors 14 at the vehicle 10 be attached, one for each or a combination of the front of the vehicle 10 , the left side of the vehicle 10 , the right side of the vehicle 10 and the back of the vehicle 10 , As an example, the disclosure is discussed in a context that the vehicle 10 only a single sensor 14 Has.

The single sensor 14 captures an area of the vehicle 10 is assigned and generates sensor signals based thereon. In various embodiments, the sensor signals include image data. The object recognition module 16 receives the signals and processes them to recognize an object. In various embodiments, the object recognition module generates 16 selectively signals based on recognition of the object. The signals are from a control module 18 and / or an alarm module 20 received to the vehicle 10 to selectively control and / or to alert the driver to the vehicle 10 to control.

In various embodiments, the object recognition module detects 16 the object based on an image processing method that processes the image data with deep learning models. Deep learning models may include neural networks, such as convolution networks, or other deep learning models, such as deep-belief networks. The deep learning models are pre-trained based on a variety of sample image data.

In various embodiments, the object recognition module processes 16 the image data using the deep learning model to obtain obstacles and other traffic elements in the image. The object recognition module 16 uses the detected elements to detect, for example, road sharing, stixels, and / or objects within a scene.

With reference to here 2 A data flow diagram illustrating various embodiments of the object recognition module 16 of the object recognition system 12 ( 1 ). The object recognition module 16 processes image data 30 Matching with different embodiments. As can be seen, various embodiments of the object recognition module 16 comprise any number of sub-modules in accordance with the present disclosure. For example, the submodules shown in 2 combined and / or further subdivided to similarly process an image and generate signals based on the processing. Entries in the object recognition module 16 can through the single sensor 14 of the vehicle 10 to be received ( 1 ) received by other control modules (not shown) of the vehicle 10 ( 1 ) and / or detected by other sub-modules (not shown) of the object recognition module 16 , In various embodiments, the object recognition module comprises 16 a model data store 32 , an image processing module 34 , a deep learning module 36 , a Stixelerkennungsmodul 38 , an object determination module 40 , a road-sharing module 42 and / or a signal generation module 44 ,

The model data store 32 stores one or more deep learning models 46 , An exemplary deep learning model 46 is shown for example in 3 , The exemplary deep learning module 46 is a Folding network model. The convolutional network model includes multiple layers, including one filtering layer and multiple pooling layers. The deep learning model 46 is learned based on a variety of sample image data. In various embodiments, the example data may represent particular scenes or object types associated with the vehicle.

With further reference to 2 receives the image processing module 34 as input the image data 30 representing an image taken by a single sensor 14 has been recorded ( 1 ). The image processing module 34 divide the picture into a variety of drawing files 48 on. The multitude of partial images 48 contains, for example, vertical sections or vertical strips of the original image. As can be seen, the image processing module 34 Divide the picture in different ways. The disclosure becomes exemplary in connection with the division of the image into vertical sections or stripes by the image processing module 34 be discussed.

The image processing module 34 further determines the position data 50 of the drawing files 48 in the picture. For example, the image processing module 34 every drawing 48 position data 50 based on the position of the partial image in the original image. For example, the position assigned to the vertical section corresponds to position X along the X axis in the picture.

The deep learning module 36 receives the fields as input 48 and the corresponding position data X 50 , The deep learning module 36 processes every partial image 48 by making a deep learning model 46 used in the model data store 32 is stored. Based on the processing generates the deep learning module 36 Position data Y 52 pointing to the edge of the traffic elements (below and / or above each element) in each field 48 clues.

The stixel detection module 38 receives as input the plurality of fields 48 , the position data X 50 and the position data Y 52 , The stixel detection module 38 Further, each of the many fields is processed to determine a second position Y in the field. The second position Y indicates an endpoint of the object in the field. The stixel detection module 38 determines the second position Y in the field based on a deep learning module 46 from the model data memory 32 and / or other image processing techniques.

The stixel detection module 38 defines a stixel based on the position X, the first position Y and the second position Y of a field. For example, as shown in 4 , the stixels begin at the designated ground truth (position Y) and end at the designated second position Y. For example, if the first position Y and the second position Y are in the vicinity thereof, then a stixel may not be defined. The stixel determination module 38 generates stix data 54 based on the defined pixels in the image.

With further reference to 2 receives the object determination module 40 as input the multitude of partial images 48 , the position data X 50 and the position data Y 52 , The object determination module 40 determines the presence of an object based on the subpicture data 48 and the position data Y 52 , For example, the object determination module processes 40 the captured image based on additional processing techniques (eg, estimating the optical flow or other methods) to determine if an object is present in the image above the detected position Y. As shown in 5 , generates the object determination module 40 object data 56 which indicate the position X and the position Y of the detected objects in the sub-images.

With further reference to 2 receives the road partitioning module 42 as input the multitude of partial images 48 , the position data X 50 and the position data Y 52 , The road-partitioning module 42 estimates the subpicture data 48 and the position data Y 52 to find an outline of a street in a scene. For example, as shown in 6 , estimates the road splitting module 42 each row of the field and defines the road layout based on the first and last position X in the series having a contiguous position Y. The road-partitioning module 42 generates roadmap data 58 based on the first and last position X of all rows of the image.

With further reference to 2 receives the signal generator module 44 as input the Stixeldaten 54 , the object data 56 and / or the road distribution data 58 , The signal generator module 44 rates the stix data 54 , the object data 56 and / or the road distribution data 58 and selectively generates an alarm signal 60 and / or a control signal 62 , based on the rating. If, for example, a rating of the Stixeldaten 54 and / or the object data 56 indicating that the object is a threat, then an alarm signal 60 and / or a control signal 62 generated. Another example would be that if the assessment of the road distribution data 58 indicating that the vehicle 10 comes off the street, then a warning signal 60 and / or a control signal 62 is generated. As can be seen, the Stixeldaten 54 , the object data 56 and / or the road distribution data 58 be rated. Signals based on criteria other than those described are merely examples.

With reference to here 7 and with further reference to 1 and 2 1, a flowchart shows an object recognition method 100 which is executed by the object recognition system 12 of the 1 and 2 in accordance with various embodiments. As can be seen in the disclosure, the order of operation is within the method 100 not limited to the sequential order as in 7 but may be implemented in one or more variations, as appropriate and consistent with the present disclosure.

As can further be seen, the method may look 7 be scheduled to operate at predetermined time intervals during operation of the vehicle 10 is performed and / or scheduled to run based on predetermined events.

In one example, the method may start at 105 , The image data 30 become at 110 receive. From the image data 30 become at 120 the drawing files 48 determined, and at 130 the position data X 50 of the drawing files 48 , The drawing files 48 be using a deep learning model 46 at 140 processed to the position data Y 52 to investigate. The drawing files 48 , the position data X 50 and the position data Y 52 then join 150 . 160 and or 170 processed to at least one of the Stixeldaten 54 , the object data 56 and / or the road distribution data 58 to investigate. The Stixeldaten 54 , the object data 56 and / or the road distribution data 58 become at 180 evaluated and used to selectively control the control signals 62 and / or warning signals 60 at 190 to create. After that, the method can be used 200 end up.

While at least one embodiment has been presented in the foregoing detailed description, it should be understood that there are a variety of variations. It is further understood that the embodiment or embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of this disclosure in any way. The foregoing detailed description rather provides those skilled in the art with a convenient roadmap for implementing the embodiment or embodiments. It is understood that various changes can be made in the function and arrangement of elements without departing from the scope of the disclosure as set forth in the appended claims and their legal equivalents.

Claims (10)

A method of recognizing an object, comprising: Receiving by a processor data from a single sensor, the data representing an image; Division of the image by the processor into vertical sub-images; Processing of the vertical part images by the processor based on deep learning models; and Detection of an object by the processor based on the processing. The method of claim 1, further comprising assigning position data to each level of the vertical fields based on a location of the vertical fields in the image. The method of claim 2, wherein the position data includes a position X along an X-axis of the image. The method of claim 1, wherein the processing of the vertical fields further includes processing the vertical fields using the deep learning model to determine boundaries of road elements in the vertical fields. The method of claim 4, wherein each boundary of a road element includes at least a lower boundary, an upper boundary, and a top and bottom boundary. The method of claim 4, wherein each boundary includes a position Y along a Y-axis of the vertical field. The method of claim 4, further comprising processing data above the boundaries using an image processing technique to determine if one or more objects are present in the vertical subpicture above the boundaries. The method of claim 4, further comprising determining an outline of a road in the image based on the boundaries and the vertical fields. The method of claim 1, further comprising determining stix data based on the vertical fields and the deep learning models. The method of claim 9, wherein the detection of the object is based on the stix data.

Images