KR102655999B1 - Apparatus and method for detecting distance values per instance object using a stereo camera - Google Patents

Abstract

The present invention discloses an apparatus and method for detecting distance values for each instance object using a stereo camera. The apparatus and method for detecting distance values for each instance object using a stereo camera of the present invention include: a stereo camera image receiver that receives left and right images from a stereo camera; A data extraction unit that classifies instance objects from the left and right images received from the stereo camera image receiver using a preset object classification algorithm, extracts distance values for each instance object, and extracts data for deep learning; And through a preset deep learning network algorithm, the data extracted from the data extraction unit is concatenated with related data and compressed into one image, and class information for each instance object and between the instance object and the own vehicle are compressed. It is characterized by including a deep learning unit that generates a two-channel array containing distance values.

Description

Apparatus and method for detecting distance values for each instance object using a stereo camera {APPARATUS AND METHOD FOR DETECTING DISTANCE VALUES PER INSTANCE OBJECT USING A STEREO CAMERA}

The present invention relates to an apparatus and method for detecting distance values for each instance object using a stereo camera. More specifically, the present invention relates to detecting the distance value between the own vehicle and surrounding instance objects by utilizing data detectable from the video image of the stereo camera for deep learning. This relates to an apparatus and method for detecting distance values for each instance object using a stereo camera.

With the recent rapid development of science and technology, digital image processing has been put to practical use in many application fields such as space observation, medical use, photo interpretation, design, various pattern recognition, and vehicle operation. In particular, recently, services that combine image or video recognition technology with AI are emerging, and image recognition technology using deep learning is accelerating technology development as it can be linked to various fields such as medicine, self-driving cars, and commerce.

In other words, image recognition refers to analyzing image signals input from a camera to segment objects and extract features. In particular, in this embodiment, when driving a vehicle, image recognition is performed to identify the own vehicle and instances around the vehicle. It is about finding the distance value between objects.

There are various ways to perform image recognition to extract a specific object from the background, but as shown in Figure 1, depth map and bounding box (bbox) information are used to identify objects present in the video image. You can obtain the relative distance value between instance objects and the own vehicle. More specifically, to calculate the relative distance for each bounding box, the smallest depth value within the bounding box can be used, or the distance value can be calculated by calculating the correlation between the left image and the right image.

Meanwhile, there are various objects that need to be detected in an image. Among these, objects with the same properties that are grouped and defined are called classes, and the form of the class can be variable depending on the application to which it is applied. At this time, depending on the type of class defined, it may be the same object or a different object. For example, in the case of the sky class, two or more isolated areas of the sky may appear in the image, and these areas can be classified as the same object. On the other hand, in the case of different objects, if the boundaries are ambiguous and difficult to distinguish, only the class is defined without distinguishing the objects. For example, when defining classes such as cars, pedestrians, and bicycles, the boundaries are clear and may be necessary depending on the time. Since it is important information to distinguish and recognize, it is essential to distinguish each object. At this time, each object distinguished is called an instance object.

In the prior art shown in FIG. 1, the distance value between the own vehicle and the instance object is calculated using all pixel information in the bounding box. Accordingly, there was a problem in which accurate distance calculation could not be performed because the information was contaminated with pixels unrelated to the instance object. In addition, as the depth map was calculated to calculate the relative distance for each bounding box, the amount of calculation increased significantly, leading to a decrease in performance.

The background technology of the present invention includes Korean Patent Publication No. 10-2014-0142554 (publication date: December 12, 2014) titled “Moving object detection device, method, and computer-readable recording medium using depth information.” .

According to one aspect of the present invention, the present invention was created to improve the above problems, and the stereo camera detects the distance value between the own vehicle and surrounding instance objects by utilizing data detectable in the video image of the stereo camera for deep learning. The purpose is to provide an apparatus and method for detecting distance values for each instance object using a camera.

An apparatus for detecting a distance value for each instance object using a stereo camera according to an aspect of the present invention includes: a stereo camera image receiver that receives left and right images from the stereo camera; A data extraction unit that classifies instance objects from the left and right images received from the stereo camera image receiver through a preset object classification algorithm and extracts data for deep learning by extracting distance values for each instance object. ; And through a preset deep learning network algorithm, the data extracted from the data extraction unit is concatenated with related data and compressed into one image form, and class information for each instance object and the instance object are combined. It is characterized by including a deep learning unit that generates a two-channel array containing distance values between the own vehicles.

In the present invention, the data extraction unit performs mask segmentation for each instance, indicating whether each pixel corresponds to a specific instance object and a bounding box in the left and right images received from the stereo camera image receiver. a mask segmentation generator that detects; and a region extraction unit that divides the mask segmentation for each instance detected by the mask segmentation generator into regions, calculates the origin of the bounding box, and extracts class information for each instance object.

In the present invention, the deep learning unit lists instance objects corresponding to each category of the mask segmentation area for each left and right image, the origin of the bounding box, and class information for each object, structures them in an array, and concatenates them, A combination structuring unit that converts input data for performing deep learning; An encoding unit for encoding the input data converted by the combination structuring unit for each category of the mask segmentation area, the origin of the bounding box, and class information for each object into a preset deep learning network algorithm; an image compression unit that concatenates the data for each category encoded by the encoding unit and compresses it into a single image; and an array generator that inputs the image data compressed by the image compression unit back into a preset deep learning network algorithm to generate a two-channel array including class information for each instance object and a distance value between the instance object and the own vehicle; It is characterized by including.

In the present invention, the data extraction unit detects a bounding box through a preset object classification algorithm in each of the left and right images received from the stereo camera image receiver, and creates a feature map of the corresponding position of the detected bounding box. is extracted by aligning it using the RoIAlign method, and classifies the class of the object from the extracted feature map and detects the mask of the object at the same time.

In the present invention, the preset object classification algorithm is characterized as a region-based deep learning algorithm including a mask region-based convolutional neural network (R-CNN).

In the present invention, the preset deep learning network algorithm is characterized as a deep learning network algorithm based on a convolution neural network (CNN).

A method of detecting a distance value for each instance object using a stereo camera according to another aspect of the present invention includes the steps of a stereo camera image receiving unit receiving left and right images from the stereo camera; The data extraction unit classifies instance objects through a preset object classification algorithm in each of the left and right images received from the stereo camera image receiver, extracts distance values for each instance object, and extracts data for deep learning. step; And a deep learning unit compresses the data extracted from the data extraction unit into one image form by concatenating related data through a preset deep learning network algorithm, and class information for each instance object and the Generating a two-channel array containing distance values between the instance object and the host vehicle.

In the step of extracting data for deep learning of the distance value extraction for each instance object of the present invention, the data extraction unit extracts a bounding box and a pixel from each left and right image received from the stereo camera image receiver. Detecting mask segmentation for each instance indicating whether it corresponds to a specific instance object, dividing the detected mask segmentation for each instance into regions, calculating the origin of the bounding box, and extracting class information for each instance object. It is characterized by

In the step of generating a two-channel array including class information for each instance object and a distance value between the instance object and the host vehicle of the present invention, the deep learning unit creates a mask segmentation area and bounding for each image on the left and right. Origin of box and class information for each object Lists instance objects corresponding to each category, structures them in an array form and concatenates them, converts them into input data for deep learning, and sets the origin of the mask segmentation area and bounding box. And the input data converted for each category of class information for each object is entered into a preset deep learning network algorithm and encoded, and the encoded data for each category is concatenated and compressed into one image form. , Characterized by inputting the data of the compressed image back into a preset deep learning network algorithm to generate a two-channel array containing class information for each instance object and a distance value between the instance object and the own vehicle.

In the step of extracting data for deep learning of distance value extraction for each instance object of the present invention, the data extractor detects a bounding box through a preset object classification algorithm in each of the left and right images received from the stereo camera image receiver. The feature map of the corresponding position of the detected bounding box is aligned and extracted using the RoIAlign method, and the class of the object is classified from the extracted feature map and the mask of the object is detected at the same time. It is characterized by

In the present invention, the preset object classification algorithm is characterized as a region-based deep learning algorithm including a mask region-based convolutional neural network (R-CNN).

In the present invention, the preset deep learning network algorithm is characterized as a deep learning network algorithm based on a convolution neural network (CNN).

The apparatus and method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention utilizes data detectable in the video image of the stereo camera for deep learning to detect the distance value between the own vehicle and surrounding instance objects, Accuracy can be improved, and since a depth map is not obtained, performance can be improved by reducing the amount of calculation and hardware storage capacity, and improving speed.

In addition, the apparatus and method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention can be easily implemented using only a simple data structure using an array and a CNN algorithm, thereby improving convenience.

In addition, in the apparatus and method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention, the number of required Flops is drastically reduced due to a decrease in calculation capacity, and the size of the corresponding calculation chip is reduced, thereby reducing the size of the target board. is reduced, making it possible to reduce weight, and reducing the number of parts and reducing weight has the effect of reducing costs.

Figure 1 is a diagram for explaining a method of calculating distance for each object using a conventional stereo camera.
Figure 2 is a block diagram showing a distance value detection device for each instance object using a stereo camera according to an embodiment of the present invention.
Figure 3 is a flowchart illustrating a method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating data extraction by a distance value detection device for each instance object using a stereo camera according to an embodiment of the present invention.
Figures 5a and 5b are diagrams for explaining the deep learning process of a distance value detection device for each instance object using a stereo camera according to an embodiment of the present invention.

Hereinafter, an apparatus and method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention will be described with reference to the attached drawings. In this process, the thickness of lines or sizes of components shown in the drawing may be exaggerated for clarity and convenience of explanation.

In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, definitions of these terms should be made based on the content throughout this specification.

Figure 2 is a block diagram showing an apparatus for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention, and Figure 4 is a block diagram showing distance value detection for each instance object using a stereo camera according to an embodiment of the present invention. This is a diagram for explaining data extraction of the device, and Figures 5a and 5b are diagrams for explaining the deep learning process of the distance value detection device for each instance object using a stereo camera according to an embodiment of the present invention. With reference to this, the stereo The distance value detection device for each instance object using a camera is explained as follows.

As shown in FIG. 2, the apparatus for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention includes a stereo camera image receiver 10, a data extractor 20, and a deep learning unit 30. Includes.

First, according to one embodiment, the present invention relates to an invention that determines objects around (e.g. in front of) the own vehicle and calculates the distance for each object in an environment using a stereo camera.

Accordingly, the stereo camera image receiver 10 receives the left and right images taken from the stereo camera, and the received left and right images are sent to the data extractor 20 and other ECUs in the vehicle. It can be transmitted, etc.

The data extraction unit 20 includes a mask segmentation generation unit 22 and a region extraction unit 24, and uses a preset object classification algorithm in each of the left and right images received from the stereo camera image reception unit to create an instance object. Classifies (instance object), extracts distance values for each instance object, and extracts data for deep learning.

In other words, the data extractor 20 detects a bounding box in each of the left and right images received from the stereo camera image receiver 10 through a preset object classification algorithm, and determines the corresponding position of the detected bounding box. The feature map is aligned and extracted using the RoIAlign method, and the class of the object can be classified from the extracted feature map and at the same time the mask of the object can be detected.

Here, the bounding box refers to a rectangular box that includes all instance objects within the image. It is generally defined by the position coordinates of the origin of the bounding box or the diagonal extreme values, and the origin and diagonal extreme values can be easily obtained from each other.

As shown in FIG. 4, the mask segmentation generator 22 displays a bounding box in each of the left and right images received from the stereo camera image receiver 10 and whether each pixel corresponds to a specific instance object. Detects mask segmentation for each instance. At this time, (a) is the mask segmentation result area, and (b) is the class information of the detected object and the origin coordinates of the bounding box in the image.

Meanwhile, mapping pixels within an image to a predefined class is called segmentation, and at this time, pixels segmented from the same instance or class are grouped and imaged as a mask. It is called segmentation (mask segmentation). More precisely, when grouping instances together, it is called instance mask segmentation, and when grouping between classes, it is called class mask segmentation. However, in this embodiment, since the target is to recognize instance objects, only instance mask segmentation is applicable, so instance mask segmentation is simply called mask segmentation. can do.

Then, the region extraction unit 24 divides the mask segmentation for each instance detected by the mask segmentation generation unit 22 into regions, calculates the origin of the bounding box, and extracts class information for each instance object.

Meanwhile, in this embodiment, the preset object classification algorithm is a region-based deep learning algorithm including mask region-based convolutional neural network (Mask R-CNN).

Here, Mask R-CNN is one of the algorithms that recognizes instance objects in the image plane. Numerous bounding boxes are proposed (proposal-based), and within the proposed bounding boxes, the most important instance objects are displayed on a pixel-wise basis using a CNN (Convolutional Neural Network)-based deep learning algorithm. If the finding result is correct, all pixels corresponding to the instance object in the image will be found.

In addition, Mask R-CNN extracts the feature map of the corresponding location from the proposed bounding box (also known as RoI) using the RoIAling method, classifies the class of the object from the extracted feature map, and simultaneously classifies the object's mask. can be obtained, and multiple objects can be recognized as individual instances. In other words, Mask R-CNN is an algorithm that separates RoIAlign and class-specific masks, and may not perform object detection and semantic segmentation (theoretical segmentation) as separate tasks. And RoIAlign aligns the feature map and uses specific values corrected using an interpolation method such as bilinear interpolation. In other words, RoIAlign can accurately align the RoI area of the feature map by using bilinear interpolation without using rounding.

In other words, Mask R-CNN can independently predict each class when predicting the mask of an object in the RoI area. In other words, if there are k classes, k masks can be predicted and trained using only the loss of the mask channel corresponding to the actual class. In a conventional segmentation network, in order to predict which class a pixel in an image corresponds to, it is assumed that all classes are mutually exclusive. Therefore, all classes are in competition for one pixel.

However, Mask R-CNN separates predicting what class the object in RoI is from predicting the mask of that object. Like other networks, object class predictions compete with each other assuming mutual exclusion, but when predicting the object's mask, k masks for the object are predicted, and the k masks do not compete with each other because loss calculations are performed independently. It may not be possible.

Next, the deep learning unit 30 includes a combination structuring unit 32, an encoding unit 34, an image compression unit 36, and an array generation unit 38, as shown in FIGS. 5A and 5B. As shown, the data extracted from the data extraction unit is compressed into one image form by concatenating related data through a preset deep learning network algorithm, and class information for each instance object and the instance Create a two-channel array containing the distance values between the object and the host vehicle.

First, the combination structuring unit 32 lists instance objects corresponding to each category of the mask segmentation area, origin of the bounding box, and object-specific class information for each of the left and right images, structures them in an array, and concatenates them, Convert it to input data for deep learning.

Meanwhile, in this embodiment, concatenation is a term used in deep learning and is based on a multi(n)-dimensional or multi-channel array, and means directly stacking the array based on a specific channel. Because of this, concatenation is only possible if all channels except one channel in the n-dimensional array are unconditionally identical. For example, if you concatenate a 5*5*3 array and 5*5*10 channels, the arrays will be stacked based on the 3rd channel to create 5*5*13 channels, but the 5*5*3 array and 5*3 *In the case of 10 channels, concatenation cannot occur because the sizes of two or more channels are different.

And the encoding unit 34 encodes the input data converted by the combination structuring unit for each category of the mask segmentation area, the origin of the bounding box, and the class information for each object into a preset deep learning network algorithm.

Additionally, the image compression unit 36 can concatenate the data for each category encoded by the encoding unit 34 and compress it into one image.

Finally, the array generator 38 inputs the image data compressed by the image compression unit 36 back into a preset deep learning network algorithm to include class information for each instance object and a distance value between the instance object and the own vehicle. You can create a two-channel array. That is, the array generator 38 can finally detect class information for each instance object and the distance value between the instance object and the own vehicle.

In this embodiment, the preset deep learning network algorithm is a convolutional neural network (CNN)-based deep learning network algorithm.

CNN is an abbreviation for Convolution Neural Network, and is defined through basic deep learning building blocks such as convolution filter, pooling, relu/tanh/absolute/sigmode function, etc., and can be used in tens of thousands of ways depending on how to combine convolution filter, pooling, relu function, etc. There may be a way. In CNN, due to pooling, data containing relevant information from images are combined and the size of the array is reduced. Pooling is also called encoding because the features are compressed as above.

Meanwhile, the above-described content is about the forward pass of the distance value detection device for each instance object using a stereo camera, and the backward pass can be operated to learn the CNN.

That is, in this embodiment, the weights of the CNN can be learned by calculating the difference with the distance information predicted above as loss using the correct answer sheet between the real object and the own vehicle. At this time, the answer key can be obtained through a lidar sensor (not shown) in addition to a stereo camera or a high-precision distance sensor (not shown). At this time, L2 loss, etc. may be used to calculate loss. Additionally, in the backward pass, learning can include Mask R-CNN, and learning can only be done for the CNN network after Mask R-CNN. If Mask R-CNN is not trained, the corresponding weights can be excluded from the update target during training.

Figure 3 is a flowchart for explaining a method of detecting a distance value for each instance object using a stereo camera according to an embodiment of the present invention. With reference to this, the method for detecting a distance value for each instance object using a stereo camera will be described as follows.

First, the stereo camera image receiver 10 receives left and right images from the stereo camera (S10).

Next, the data extractor 20 generates a bounding box in each of the left and right images received from the stereo camera image receiver 10 and a mask segmentation for each instance indicating whether each pixel corresponds to a specific instance object ( Detect mask segmentation (S20).

Then, the data extraction unit 20 divides the mask segmentation for each instance detected in step S20 into regions, calculates the origin of the bounding box, and extracts class information for each instance object (S30).

That is, the data extractor 20 classifies instance objects from each of the left and right images received from the stereo camera image receiver 10 through a preset object classification algorithm and extracts distance values for each instance object. Data for deep learning can be extracted.

In other words, the data extractor 20 detects a bounding box in each of the left and right images received from the stereo camera image receiver 10 through a preset object classification algorithm, and determines the corresponding position of the detected bounding box. The feature map is aligned and extracted using the RoIAlign method, and the class of the object can be classified from the extracted feature map and at the same time the mask of the object can be detected.

Meanwhile, in this embodiment, the preset object classification algorithm is a region-based deep learning algorithm including mask region-based convolutional neural network (Mask R-CNN).

Here, Mask R-CNN is one of the algorithms that recognizes instance objects in the image plane. Numerous bounding boxes are proposed (proposal-based), and within the proposed bounding boxes, the most important instance objects are displayed on a pixel-wise basis using a CNN (Convolutional Neural Network)-based deep learning algorithm.

Next, the deep learning unit 30 lists the instance objects corresponding to each category of the mask segmentation area for each image on the left and right, the origin of the bounding box, and class information for each object, structures them in an array, and concatenates them. , Convert it to input data for deep learning (S40).

Then, in step S40, the deep learning unit 30 inputs the converted input data for each category of the mask segmentation area, the origin of the bounding box, and the class information for each object into a preset deep learning network algorithm and encodes it. (S50).

Next, in step S50, the deep learning unit 30 concatenates the encoded data for each category and compresses it into one image (S60), and the data of the compressed image is again subjected to a preset deep learning process. It is input into the network algorithm to generate a two-channel array containing class information for each instance object and the distance value between the instance object and the host vehicle (S70).

That is, the deep learning unit 30 compresses the data extracted from the data extraction unit into one image form by concatenating the data extracted from the data extraction unit through a preset deep learning network algorithm, and classifies each instance object ( class) information and a distance value between the instance object and the own vehicle are created, that is, class information for each instance object and the distance value between the instance object and the own vehicle can be finally detected.

In this embodiment, the preset deep learning network algorithm is a convolutional neural network (CNN)-based deep learning network algorithm.

CNN is an abbreviation for Convolution Neural Network, and is defined through basic deep learning building blocks such as convolution filter, pooling, relu/tanh/absolute/sigmode function, etc., and can be used in tens of thousands of ways depending on how to combine convolution filter, pooling, relu function, etc. There may be a way.

As described above, the apparatus and method for detecting the distance value for each instance object using a stereo camera according to an embodiment of the present invention utilizes data detectable in the video image of the stereo camera for deep learning to determine the distance between the own vehicle and surrounding instance objects. By detecting the value, accuracy can be improved, and since the depth map is not obtained, the amount of calculation and hardware storage capacity can be reduced, and performance can be improved due to increased speed.

In addition, the apparatus and method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention can be easily implemented using only a simple data structure using an array and a CNN algorithm, thereby improving convenience.

In addition, in the apparatus and method for detecting distance values for each instance object using a stereo camera according to an embodiment of the present invention, the number of required Flops is drastically reduced due to a decrease in calculation capacity, and the size of the corresponding calculation chip is reduced, thereby reducing the size of the target board. is reduced, making it possible to reduce weight, and reducing the number of parts and reducing weight has the effect of reducing costs.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those skilled in the art will recognize that various modifications and other equivalent embodiments are possible therefrom. You will understand.

Therefore, the true technical protection scope of the present invention should be determined by the scope of the patent claims below.

10: Stereo camera image receiver
20: data extraction unit
22: Mask segmentation generation unit
24: Area extraction unit
30: Deep learning department
32: Combined structural unit
34: encoding unit
36: Image compression unit
38: Array generation unit

Claims (12)

a stereo camera image receiving unit that receives left and right images from a stereo camera;
A mask segmentation generator that detects a bounding box and mask segmentation for each instance from each of the left and right images received from the stereo camera image receiver, and divides the detected mask segmentation for each instance into regions, calculates the origin of the bounding box, and A data extraction unit that classifies instance objects using a preset object classification algorithm, including a region extraction unit that extracts class information for each object, extracts distance values for each instance object, and extracts data for deep learning. ; and
Through a preset deep learning network algorithm, the data extracted from the data extraction unit is compressed into one image form by concatenating related data, class information for each instance object, and the instance object and its own difference. It includes a deep learning unit that generates a two-channel array containing distance values between
The deep learning unit,
Mask segmentation area for each image on the left and right, origin of bounding box, and class information for each instance object. Instance objects corresponding to each category are listed, structured and combined in an array form, and converted into input data for deep learning. A distance value detection device for each instance object using a stereo camera including a combination structure unit.
delete According to clause 1,
The deep learning unit,
An encoding unit for encoding the input data converted by the combination structuring unit for each category of the mask segmentation area, the origin of the bounding box, and class information for each object into a preset deep learning network algorithm;
an image compression unit that concatenates the data for each category encoded by the encoding unit and compresses it into a single image; and
An array generator that inputs the image data compressed in the image compression unit back into a preset deep learning network algorithm to generate a two-channel array containing class information for each instance object and a distance value between the instance object and the own vehicle; A distance value detection device for each instance object using a stereo camera, further comprising:
According to clause 1,
The data extraction unit,
A bounding box is detected through a preset object classification algorithm in each of the left and right images received from the stereo camera image receiver, and the feature map of the corresponding position of the detected bounding box is aligned using the RoIAlign method. A distance value detection device for each instance object using a stereo camera, characterized in that it extracts, classifies the class of the object from the extracted feature map, and simultaneously detects the mask of the object.
According to clause 1,
The preset object classification algorithm is,
A distance value detection device for each instance object using a stereo camera, characterized as a region-based deep learning algorithm including mask R-CNN (mask R-CNN: mask region-based convolutional neural network).
According to clause 1,
The preset deep learning network algorithm is,
A distance value detection device for each instance object using a stereo camera, characterized by a deep learning network algorithm based on a convolution neural network (CNN).
A stereo camera image receiving unit receiving left and right images from a stereo camera;
The data extraction unit classifies instance objects through a preset object classification algorithm in each of the left and right images received from the stereo camera image receiver, extracts distance values for each instance object, and extracts data for deep learning. step; and
The deep learning unit compresses the data extracted from the data extraction unit into one image form by concatenating related data through a preset deep learning network algorithm, and provides class information for each instance object and the instance. Including; generating a two-channel array containing distance values between the object and the own vehicle,
In the step of extracting the data, the data extraction unit,
Detect bounding boxes and mask segmentation for each instance from the left and right images, divide the detected mask segmentation for each instance into regions, calculate the origin of the bounding box, and extract class information for each instance object,
In the step of generating the two-channel array, the deep learning unit,
Mask segmentation area for each image on the left and right, origin of bounding box, and class information for each instance object. Instance objects corresponding to each category are listed, structured and combined in an array form, and converted into input data for deep learning. A method for detecting distance values for each instance object using a stereo camera.
delete According to clause 7,
In the step of generating a two-channel array containing class information for each instance object and a distance value between the instance object and the own vehicle, the deep learning unit,
The input data converted for each category of the mask segmentation area, the origin of the bounding box, and the class information for each object are input into a preset deep learning network algorithm and encoded, and the encoded data for each category is concatenated. ) is compressed into a single image format, and the data of the compressed image is input back into a preset deep learning network algorithm to generate a two-channel array containing class information for each instance object and the distance value between the instance object and the own vehicle. A method for detecting distance values for each instance object using a stereo camera.
According to clause 7,
In the step of extracting data for deep learning, extracting distance values for each instance object, the data extraction unit,
A bounding box is detected through a preset object classification algorithm in each of the left and right images received from the stereo camera image receiver, and the feature map of the corresponding position of the detected bounding box is aligned using the RoIAlign method. A method for detecting distance values for each instance object using a stereo camera, characterized in that it extracts, classifies the class of the object from the extracted feature map, and simultaneously detects the mask of the object.
According to clause 7,
The preset object classification algorithm is a region-based deep learning algorithm including a mask region-based convolutional neural network (R-CNN). Method for detecting distance values for each instance object using a stereo camera. .
According to clause 7,
The preset deep learning network algorithm is a deep learning network algorithm based on a convolution neural network (CNN). A method for detecting distance values for each instance object using a stereo camera.

Images