WO2020085653A1

WO2020085653A1 - Multiple-pedestrian tracking method and system using teacher-student random fern

Info

Publication number: WO2020085653A1
Application number: PCT/KR2019/012101
Authority: WO
Inventors: 고병철; 남재열; 김상준
Original assignee: 계명대학교 산학협력단
Priority date: 2018-10-26
Filing date: 2019-09-18
Publication date: 2020-04-30
Also published as: KR102164950B1; KR20200052429A

Abstract

According to a multiple-pedestrian tracking method and system using a teacher-student random fern provided in the present invention, by extracting the feature values of pedestrians by means of tiny YOLO which is a type of depth network, and learning a random fern by means of the extracted feature values, real-time learning is made possible and erroneous tracking due to changes in the shapes and sizes of the pedestrians can be minimized.

Description

Multiple pedestrian tracking method and system using teacher-student random fun

The present invention relates to a plurality of pedestrian tracking methods and systems, and more particularly, to a plurality of pedestrian tracking methods and systems using a teacher-student random fun.

In the Intelligent Transportation System (ITS) and Advanced Driver Assistance System (ADAS), to prevent pedestrians and vehicles from colliding, the technology to detect and track pedestrians is essential.

In order to achieve the appropriate level of safety in an active intelligent traffic system, the state-of-the-art driver assistance system must track all pedestrians in motion to identify pedestrians at risk of entering the road in advance.

Among various methods of tracking pedestrians, there is a method of tracking pedestrians using a Kalman filter. The Kalman filter is a recursive filter that tracks the dynamic state of noise, and is based on measurements made over time. The Kalman filter repeatedly performs state prediction and measurement update when the motion model and measurement model are linear, or when the motion model and measurement model follow the Gaussian distribution, but cannot be used unless the above two cases are true. have.

Recently, a method of tracking pedestrians using a convolutional neural network (CNN) has been studied, but it is easy to track a single pedestrian using a synthetic product neural network, but a large amount of traffic is required to track multiple pedestrians. There are problems such as not being suitable for a real-time tracking environment because parameters are required and the amount of data to be processed is large.

Accordingly, there is a need to develop a plurality of pedestrian tracking methods and systems suitable for a real-time tracking environment using a small amount of parameters.

On the other hand, as a prior art related to the present invention, Patent No. 10-1588648 (name of the invention: pedestrian detection and tracking method for intelligent video surveillance) has been disclosed.

The present invention is proposed to solve the above problems of the previously proposed methods, and extracts a feature value of a pedestrian using tiny YOLO, a type of deep network, and uses a random fun ( Random Fern) aims to provide a number of pedestrian tracking methods and systems using teacher-student random fun that enable real-time learning to minimize false tracking due to pedestrian shape change and size change. Is done.

In addition, the present invention, by using a teacher-student random fun (Teacher-Student Random Ferns) to reduce the number of Ferns (Ferns) to enable real-time tracking, it is possible to quickly and accurately track multiple pedestrians in real time. Another object is to provide a plurality of pedestrian tracking methods and systems using teacher-student random fun.

In order to achieve the above object, a plurality of pedestrian tracking methods using a teacher-student random ferns according to the features of the present invention,

As a number of pedestrian tracking methods,

(1) taking an image including a plurality of pedestrians from a camera installed in a moving vehicle;

(2) detecting a plurality of pedestrians from the image taken in step (1);

(3) extracting feature values by inputting an image including a plurality of pedestrians detected in step (2) into a deep network;

(4) learning a teacher-student random ferns using the feature values extracted in step (3); And

And (5) tracking a plurality of pedestrians using the teacher-student random ferns learned in step (4).

Preferably, the step (2),

(2-1) distinguishing a pedestrian and a non-pedestrian from the image photographed in step (1); And

(2-2) detecting the pedestrians classified in step (2-1) as the plurality of pedestrians.

Preferably, the deep network in step (3) is,

It can be a synthetic product neural network.

More preferably, the synthetic product neural network may be tiny YOLO.

Even more preferably, the tiny YOLO,

It may be composed of 9 convolution layers, 6 max pooling layers, and 1 fully connected layers.

Preferably, in step (3),

Feature values may be extracted for each of the plurality of pedestrians detected in step (2).

Preferably, the step (4),

(4-1) learning a teacher random fun using the feature values extracted in step (3); And

(4-2) Using the teacher random fun (Teacher Random Fern) learned in step (4-1) may include the step of learning a student random fun (Student Random Fern).

More preferably, the Teacher Random Ferns (Teacher Random Ferns),

You can have multiple Ferns.

More preferably, the Student Random Ferns (Student Random Ferns),

The number of Ferns may be less than that of the Teacher Random Ferns.

Preferably, in step (5),

A plurality of pedestrians can be tracked by reducing the number of Ferns using the learned Teacher-Student Random Ferns.

A plurality of pedestrian tracking systems using a teacher-student random fun according to the features of the present invention for achieving the above object,

As a number of pedestrian tracking systems,

A camera unit for photographing images including a plurality of pedestrians from a camera installed in a moving vehicle;

A detection unit that detects a plurality of pedestrians from the image taken by the camera unit;

An extraction unit that extracts feature values by inputting an image containing a plurality of pedestrians detected by the detection unit into a deep network;

A learning unit for learning a teacher-student random ferns using the feature values extracted from the extraction unit; And

It is characterized in that it comprises a tracking unit for tracking a plurality of pedestrians using a teacher-student random fun (Teacher-Student Random Ferns) learned from the learning unit.

Preferably, the detection unit,

A segmentation module that distinguishes pedestrians and non-pedestrians from the images captured by the camera unit; And

It may include a detection module for detecting the pedestrians identified in the classification module to the plurality of pedestrians.

Preferably, the deep network may be a synthetic product neural network.

More preferably, the synthetic product neural network may be tiny YOLO.

Even more preferably, the tiny YOLO,

Preferably, the extraction unit,

Feature values may be extracted for each of the pedestrians detected by the detection unit.

Preferably, the learning unit,

A first learning module for learning a teacher random ferns using the feature values extracted from the extraction unit; And

It may include a second learning module for learning the student random fun (Student Random Ferns) using the teacher random fun (Teacher Random Ferns) learned in the first learning module.

More preferably, the Teacher Random Ferns (Teacher Random Ferns),

You can have multiple Ferns.

More preferably, the Student Random Ferns (Student Random Ferns),

The number of Ferns may be less than that of the Teacher Random Ferns.

Preferably, the tracking unit,

According to a plurality of pedestrian tracking methods and systems using a teacher-student random fun proposed in the present invention, a feature value of a pedestrian is extracted using tiny YOLO, a type of deep network, and a random value is extracted using the extracted feature value By learning the Fern (Random Ferns), real-time learning is possible, minimizing mistracking due to pedestrian shape change and size change.

In addition, according to a plurality of pedestrian tracking methods and systems using the teacher-student random fun proposed in the present invention, to reduce the number of ferns (Ferns) to enable real-time tracking, teacher-student random fun (Teacher-Student) Random Ferns), it is possible to quickly and accurately track multiple pedestrians in real time.

1 is a flowchart illustrating a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a multi-layer perceptron (MLP) network among deep networks.

3 is a view showing a detailed flow of step S200 in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

FIG. 4 is a view illustrating step S210 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

FIG. 5 is a view illustrating step S300 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

6 is a view showing the detailed flow of step S400 in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

7 is a diagram illustrating an overall process of learning a teacher random ferns in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

8 is a diagram showing an algorithm for learning a student random fun in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention.

9 is (a) tracking a plurality of pedestrians using QuadMOT and (b) tracking a plurality of pedestrians using a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention Drawings for comparison.

10 is a view showing the configuration of a plurality of pedestrian tracking systems using a teacher-student random fun according to an embodiment of the present invention.

11 is a view showing the detailed configuration of a detection unit in a plurality of pedestrian tracking systems using a teacher-student random fun according to an embodiment of the present invention.

12 is a diagram showing the detailed configuration of a learning unit in a plurality of pedestrian tracking systems using a teacher-student random fun according to an embodiment of the present invention.

10: Multiple pedestrian tracking system

100: camera unit

200: detection unit

210: classification module

220: detection module

300: extraction unit

400: learning department

410: first learning module

420: second learning module

500: tracker

S100: Step of shooting an image containing a plurality of pedestrians from a camera installed in a moving car

S200: detecting a plurality of pedestrians in the image taken in step S100

S210: Step to distinguish pedestrian and non-pedestrian from the image taken in step S100

S220: detecting pedestrians classified in step S210 as a plurality of pedestrians

S300: extracting feature values by inputting an image including a plurality of pedestrians detected in step S200 into a deep network

S400: Learning a teacher-student random ferns using the feature values extracted in step S300.

S410: Learning a teacher random fun using the feature values extracted in step S300.

S420: Step of learning student random fun using teacher random ferns learned in step S410.

S500: Tracking multiple pedestrians using the teacher-student random ferns learned in step S400

Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily implement the present invention. However, in the detailed description of a preferred embodiment of the present invention, when it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, the same or similar reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, in the entire specification, when a part is said to be 'connected' with another part, it is not only 'directly connected', but also 'indirectly connected' with another element in between. Includes. In addition, "including" a component means that other components may be further included instead of excluding other components, unless otherwise stated.

Each step of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention may be performed by a computer device. Hereinafter, for convenience of description, the subject may be omitted in each step.

1 is a flowchart illustrating a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. As illustrated in FIG. 1, a method for tracking a plurality of pedestrians using a teacher-student random fun according to an embodiment of the present invention includes photographing images including a plurality of pedestrians in a camera installed in a moving vehicle (S100) , Detecting a plurality of pedestrians from the image taken in step S100 (S200), extracting feature values by inputting an image including a plurality of pedestrians detected in step S200 into a deep network (S300), in step S300 Using the extracted feature value to learn a teacher-student random fun (Teacher-Student Random Ferns) step (S400), and using a teacher-student random fun (Teacher-Student Random Ferns) learned in step S400 a plurality of It may be implemented, including the step of tracking the pedestrian (S500).

Hereinafter, before explaining each step of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention, the deep network and random fun used in the present invention will be first described in detail. Do it.

Artificial Neural Network (ANN) is a statistical learning algorithm used in machine learning and cognitive science, inspired by the neural network of biology (especially the brain of the animal's central nervous system). The artificial neural network refers to an entire network that has problem-solving ability by changing the strength of synaptic binding through learning by artificial neurons (nodes) that form a network through synaptic binding. In a narrow sense, it may refer to a multi-layer perceptron using error back propagation, but this is a misuse, and the artificial neural network is not limited thereto.

A deep network or a deep neural network (DNN) is an artificial neural network composed of several hidden layers between an input layer and an output layer. Deep networks can model complex non-linear relationships, just like a normal artificial neural network. For example, in a deep network structure for an object identification model, each object may be represented by a hierarchical configuration of basic elements of an image, where additional layers can aggregate features of progressively gathered lower layers. This feature of the deep network allows modeling of complex data with fewer units than a similarly performed artificial neural network.

FIG. 2 is a diagram illustrating a multi-layer perceptron (MLP) network among deep networks. As shown in FIG. 2, the MLP network is a neural network in which one or more intermediate layers exist between the input layer and the output layer, and the intermediate layer between the input layer and the output layer is called a hidden layer. The network is connected to the input layer, the hidden layer, and the output layer, and there is no direct connection from each layer to the input layer from the output layer.

The MLP network has a structure similar to that of the single-layer perceptron, but improves the network capability by overcoming the input / output characteristics of the middle layer and each unit to overcome various disadvantages of the single-layer perceptron. In the MLP network, as the number of layers increases, the characteristics of the crystal region formed by perceptrons become more advanced. More specifically, in the case of a single layer, the pattern space is divided into two sections, and in the case of the second floor, a convex open zone or a concave closed zone is formed, and in the case of the third floor, any type of zone may be formed in theory.

In general, when input data is presented to each unit of the input layer, this signal is converted from each unit and transmitted to the middle layer, and finally output to the output layer. The direction of comparing the output value with the desired output value to reduce the difference By adjusting the connection strength, you can train the MLP network.

The Convolutional Neural Network (CNN) is a type of MLP network designed to use minimal preprocessing. The synthetic product neural network is a neural network composed of one or several convolutional layers, a pooling layer, and a fully connected layer, and has a structure suitable for learning two-dimensional data. Since it can be trained through a backpropagation algorithm, it can be widely used in various application fields such as object classification in image and object detection.

The convolution layer can serve to extract features from the input data. The convolution layer may consist of a filter that functions to extract features and an activation function that converts the values extracted from the filter into nonlinear values.

Synthetic product neural networks can be trained through gradient descent and backpropagation algorithms. At this time, the gradient descent method is an optimization algorithm for first-order approximation values. It is a method of finding the gradient (slope) of a function and continuously moving the gradient to the lower side and repeating it until an extreme value is reached. The backpropagation algorithm is used for multi-layer perceptron learning It refers to a statistical technique, which is a method of adjusting individual weights so that a desired value is output for the same input layer.

Random Ferns, a method proposed by Ozuysal in 2007, is a modification of Bayes' theory. Random Ferns overcomes the limitations of Bayes' theory by considering the correlation between feature functions. Also, it is possible to perform simple and fast calculations by implementing a feature function using the difference between two pixels. The performance of Random Ferns has better classification performance than the Random Tree, and has the same performance as the object recognition rate of SIFT and a faster computation speed than SIFT.

Random Ferns can be defined through the following process.

H multiple classes c _z , z = 1,… Defined as, H, and N feature extraction functions f _j , j = 1,… When defined as, K, classifying as a probability can be defined as in Equation 1 below.

Equation 1

Equation 1 can be defined as Equation 2 by using Bayes definition.

Equation 2

Assuming P (C) and P (f ₁ , f ₂ ,…, f _k ) in Equation 2 as predetermined probability values, the multi-class c _z may be defined as in Equation 3 below.

Equation 3

In order to obtain P (f ₁ , f ₂ ,…, f _k | C = c _z ) in Equation 3, each feature extraction function can be calculated as in Equation 4 below.

Equation 4

However, in order to take account of this, in general, since some feature extraction functions show dependent features, Random Ferns assumes that there is a correlation between the feature extraction functions and bundles the correlated feature extraction functions. Referred to as a fern. Equation (4) can be modified as shown in Equation (5) by using random ferns.

Equation 5

In Equation 5, F _k = {f _{a (k, 1)} , f _{a (k, 2)} ,… , f _{a (k, S)} } = 1,… , M denotes the kth ferns, and contains S feature extraction functions inside. a (k, j) has a range of 1,… The random permutation function, N, is selected randomly from the S feature extraction functions. Through this, the random fern (Random Ferns) performs classification using the results of M ferns.

Hereinafter, each step of the plurality of pedestrian tracking methods using the teacher-student random fun proposed in the present invention, using the deep network and the random fun (Random Ferns) as described above, will be described in detail.

In step S100, an image including a plurality of pedestrians may be photographed by a camera installed in a moving vehicle. In order for the active intelligent traffic system to achieve an appropriate level of safety, in the advanced driver assistance system, all pedestrians in motion must be tracked to identify pedestrians at risk of entering the road in advance. Through the camera, images containing multiple pedestrians can be captured.

In step S200, a plurality of pedestrians may be detected from the image photographed in step S100. 3 is a view showing a detailed flow of step S200 in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. As illustrated in FIG. 3, step S200 of a method for tracking a plurality of pedestrians using a teacher-student random fun according to an embodiment of the present invention may include distinguishing a pedestrian and a non-pedestrian from the image photographed in step S100 ( S210), and detecting the pedestrians identified in step S210 as a plurality of pedestrians (S220).

4 is a diagram illustrating steps S210 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. As shown in FIG. 4, in step S210, a pedestrian and a non-pedestrian may be distinguished from the image photographed in step S100. At this time, the pedestrian may be a person, and the non-pedestrian may be a power pole, a tree, a building, or the like.

In step S220, the pedestrians classified in step S210 may be detected as a plurality of pedestrians. In the method of tracking a plurality of pedestrians using a teacher-student random fun according to an embodiment of the present invention, since it is necessary to detect a plurality of pedestrians and input them to a deep network, feature values of each pedestrian must be extracted, in step S220, step The pedestrians classified in S210 may be detected as a plurality of pedestrians, and input to the deep network of step S300 described below to extract feature values of each pedestrian.

5 is a view illustrating a step S300 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. As shown in FIG. 5, in step S300 of a method for tracking multiple pedestrians using a teacher-student random fun according to an embodiment of the present invention, an image including a plurality of pedestrians detected in step S200 is input to a deep network The feature values can be extracted.

More specifically, in step S300 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention, as a deep network, using tiny YOLO, which is a kind of synthetic multiplicity neural network, in step S200 Feature values may be extracted for each pedestrian from an image including a plurality of detected pedestrians.

The tiny YOLO may consist of 9 Convolution layers, 6 Max pooling layers, and 1 fully connected layers. At this time, the feature value of the pedestrian can be extracted through the last connection layer, which is the last layer of tiny YOLO.

In step S400, a teacher-student random ferns may be learned using the feature values extracted in step S300. 6 is a view showing the detailed flow of step S400 in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. As illustrated in FIG. 6, step S400 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention uses a teacher random fun using the feature values extracted in step S300. ) Learning (S410), and using the teacher random fun (Teacher Random Ferns) learned in step S410 to learn the student random fun (Student Random Ferns) (S420).

Teacher-Student Random Ferns is a tracker composed of Random Ferns. Teacher Random Ferns are constructed based on a large amount of training data, so they have high tracking performance, but tracking speed may be slow and it may be difficult to track pedestrians in real time. Accordingly, a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention use a student random fun to maintain a tracking performance of a teacher random ferns while maintaining a tracking performance. By reducing the number of (Ferns), pedestrians can be tracked faster and more accurately than before.

7 is a diagram illustrating an overall process of learning a teacher random ferns in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. As illustrated in FIG. 7, in a method for tracking a plurality of pedestrians using a teacher-student random fun according to an embodiment of the present invention, a teacher random ferns detects and detects a plurality of pedestrians in step S200. In step S300, an image including a plurality of pedestrians may be input to a deep network and learned using the extracted feature values. At this time, the teacher random fern (Teacher Random Ferns) may have a plurality of ferns (Fern), for example, 1 to L (L is a natural number) may have L ferns (Fern).

In step S410, a teacher random fun can be learned using the feature values extracted in step S300. More specifically, the teacher random fun can be learned using the feature value of the pedestrian extracted in step S300 and tiny YOLO, which is one of the synthetic product neural networks.

In step S420, the student random fun can be learned using the teacher random ferns learned in step S410. More specifically, by using the teacher random fun (Teacher Random Ferns) learned in step S410, it is possible to learn by dividing the case where the pedestrian first or twice appeared.

8 is a diagram illustrating an algorithm for learning a student random fun in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention. Through the algorithm of FIG. 8, in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention, a student random fun can be learned when a pedestrian first appears. At this time, the above algorithm can be repeated as many as the number of pedestrians detected through step S200 to learn student random ferns.

When a pedestrian appears more than once, if the pedestrian detected in the current frame matches the pedestrian learned by the Student Random Ferns of the previous frame, a data connection is performed between the two frames, and the Student Random Fun (Student) Random Ferns) has been updated to learn.

As shown in the last part of the algorithm of FIG. 8, the number of Ferns in Student Random Ferns in a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention It can be less than the Teacher Random Ferns.

In step S500, a plurality of pedestrians may be tracked using a teacher-student random ferns learned in step S400. More specifically, in step S500 of a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention, using a teacher-student random ferns learned in step S400 By reducing the number of ferns, multiple pedestrians can be tracked.

9 is (a) tracking a plurality of pedestrians using QuadMOT and (b) tracking a plurality of pedestrians using a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention It is a drawing to compare the appearance. In this case, if the Student Random Ferns are judged to be the same pedestrian in the next frame, they are displayed as boxes of the same color. Referring to (a) of FIG. 9, when tracking a large number of pedestrians using QuadMOT, if the movement of the camera is large or if pedestrians overlap each other, pedestrian tracking is missing (yellow arrow in the fourth image). , There is a problem tracking other pedestrians (red arrow in the third image).

However, referring to (b) of FIG. 9, when a pedestrian is tracked using a plurality of pedestrian tracking methods using a teacher-student random fun according to an embodiment of the present invention, a student random fun By learning by updating and updating, pedestrian tracking is not omitted or the phenomenon of tracking another pedestrian in the middle does not occur.

10 is a view showing the configuration of a plurality of pedestrian tracking system 10 using a teacher-student random fun according to an embodiment of the present invention. As shown in Figure 10, a plurality of pedestrian tracking system 10 using a teacher-student random fun according to an embodiment of the present invention, the camera unit 100, the detection unit 200, the extraction unit 300 , It may be configured to include a learning unit 400 and the tracking unit 500.

More specifically, a plurality of pedestrian tracking systems 10 using a teacher-student random fun according to an embodiment of the present invention includes a camera unit 100 for photographing images including a plurality of pedestrians from a camera installed in a moving vehicle. ), A detection unit 200 for detecting a plurality of pedestrians from an image captured by the camera unit 100, and inputting an image including a plurality of pedestrians detected by the detection unit 200 into a deep network to extract feature values Extractor 300, a learning unit 400 for learning a teacher-student random fun using feature values extracted from the extracting unit 300, and a teacher trained in the learning unit 400 -It may be configured to include a tracker 500 that tracks a plurality of pedestrians using a student-fund random (Teacher-Student Random Ferns).

11 is a diagram showing the detailed configuration of a detection unit in a plurality of pedestrian tracking systems 10 using a teacher-student random fun according to an embodiment of the present invention. As shown in FIG. 11, in a plurality of pedestrian tracking systems 10 using a teacher-student random fun according to an embodiment of the present invention, the detection unit 200 may be used in an image captured by the camera unit 100. It may be configured to include a detection module 220 for detecting the pedestrians separated from the pedestrians and non-pedestrians, and the pedestrians classified in the partitioning module 210 as the plurality of pedestrians.

12 is a view showing the detailed configuration of the learning unit 400 in a plurality of pedestrian tracking systems 10 using a teacher-student random fun according to an embodiment of the present invention. As shown in FIG. 12, in a plurality of pedestrian tracking systems 10 using a teacher-student random fun according to an embodiment of the present invention, the learning unit 400, feature values extracted from the extraction unit 300 A first learning module 410 for learning a teacher random fun by using, and a student random fun by using a teacher random ferns learned in the first learning module 410 Ferns) can be configured to include a second learning module 420.

The plurality of pedestrian tracking systems 10 using a teacher-student random fun according to an embodiment of the present invention have been sufficiently described in connection with a plurality of pedestrian tracking methods using a teacher-student random fun, and thus detailed description will be omitted. Shall be

As described above, according to a plurality of pedestrian tracking methods and systems 10 using a teacher-student random fun proposed in the present invention, a feature value of a pedestrian is extracted using tiny YOLO, a type of deep network, By learning the random ferns using the extracted feature values, real-time learning is possible, thereby minimizing mistracking due to pedestrian shape change and size change. In addition, according to the present invention, in order to reduce the number of Ferns (Ferns) to enable real-time tracking, it is possible to quickly and accurately track multiple pedestrians in real time using a Teacher-Student Random Ferns. have.

The present invention described above can be variously modified or applied by a person having ordinary knowledge in the technical field to which the present invention belongs, and the scope of the technical idea according to the present invention should be defined by the following claims.

Claims

As a number of pedestrian tracking methods,

(1) taking an image including a plurality of pedestrians from a camera installed in a moving vehicle;

(2) detecting a plurality of pedestrians from the image taken in step (1);

(3) extracting feature values by inputting an image including a plurality of pedestrians detected in step (2) into a deep network;

(4) learning a teacher-student random ferns using the feature values extracted in step (3); And

(5) tracking a plurality of pedestrians using the teacher-student random ferns learned in step (4), characterized in that it comprises a plurality of teachers-student random fun Pedestrian tracking method.
The method of claim 1, wherein the step (2),

(2-1) distinguishing a pedestrian and a non-pedestrian from the image photographed in step (1); And

(2-2) A method of tracking a plurality of pedestrians using a teacher-student random fun, characterized in that it comprises the step of detecting the pedestrians separated in step (2-1) as the plurality of pedestrians.
According to claim 1, wherein the deep network in step (3),

A method of tracking multiple pedestrians using a teacher-student random fun, characterized in that it is a synthetic product neural network.
The method of claim 3, wherein the synthetic product neural network,

A method of tracking multiple pedestrians using a teacher-student random fun, characterized in that it is tiny YOLO.
The method of claim 4, wherein the tiny YOLO,

Multiple pedestrian tracking method using teacher-student random fun, characterized by consisting of 9 convolution layers, 6 max pooling layers and 1 fully connected layers .
The method of claim 1, wherein in step (3),

Characterized in that the feature value is extracted for each of the pedestrians detected in the step (2), a plurality of pedestrian tracking method using a teacher-student random fun.
The method of claim 1, wherein the step (4),

(4-1) learning a teacher random fun using the feature values extracted in step (3); And

(4-2) characterized in that it comprises the step of learning a student random fun (Student Random Ferns) using the teacher random fun (Teacher Random Ferns) learned in step (4-1), teacher-student random Multiple pedestrian tracking methods using fun.
The method of claim 7, wherein the teacher random fun (Teacher Random Ferns),

A method of tracking a plurality of pedestrians using a teacher student random fun, characterized by having a plurality of ferns.
The method of claim 7, wherein the student random fun (Student Random Ferns),

A method of tracking a plurality of pedestrians using a teacher-student random fun, characterized in that the number of ferns is less than that of the teacher random ferns.
The method of claim 1, wherein in step (5),

A method of tracking a plurality of pedestrians using a teacher-student random fun, characterized by tracking a plurality of pedestrians by reducing the number of ferns using the learned teacher-student random ferns.
As a plurality of pedestrian tracking system 10,

A camera unit 100 for photographing images including a plurality of pedestrians from a camera installed in a moving vehicle;

A detection unit 200 for detecting a plurality of pedestrians from the image taken by the camera unit 100;

An extraction unit (300) for extracting feature values by inputting an image containing a plurality of pedestrians detected by the detection unit (200) into a deep network;

A learning unit 400 for learning a teacher-student random ferns using the feature values extracted from the extraction unit 300; And

Characterized in that it comprises a tracking unit 500 for tracking a plurality of pedestrians using a teacher-student random fun (Teacher-Student Random Ferns) learned from the learning unit 400, using a teacher-student random fun Multiple pedestrian tracking systems.
The method of claim 11, wherein the detection unit 200,

A segmentation module 210 that distinguishes pedestrians and non-pedestrians from the images captured by the camera unit 100; And

And a detection module 220 that detects the pedestrians classified in the classification module 210 as the plurality of pedestrians, a plurality of pedestrian tracking systems using teacher-student random fun.
The method of claim 11, wherein the deep network,

Multiple pedestrian tracking system using a teacher-student random fun, characterized in that it is a synthetic product neural network.
14. The method of claim 13, The synthetic product neural network,

Characterized by a tiny YOLO, multiple pedestrian tracking system using a teacher-student random fun.
The method of claim 14, wherein the tiny YOLO,

Multiple pedestrian tracking system using teacher-student random fun, characterized by consisting of 9 convolution layers, 6 max pooling layers and 1 fully connected layers .
The method of claim 11, wherein the extraction unit,

Characterized by extracting the feature value for each of the pedestrians detected by the detection unit, a plurality of pedestrian tracking system using a teacher-student random fun.
The method of claim 11, wherein the learning unit 400,

A first learning module 410 for learning a teacher random ferns using the feature values extracted from the extraction unit 300; And

Characterized in that it comprises a second learning module (420) for learning a student random fun (Student Random Ferns) using the teacher random fun (Teacher Random Ferns) learned in the first learning module 410, teacher- Multiple pedestrian tracking system using student random fun.
The method of claim 17, wherein the Teacher Random Ferns (Teacher Random Ferns),

A plurality of pedestrian tracking system using a teacher-student random fun, characterized by having a plurality of ferns (Fern).
The method of claim 17, wherein the student random fun (Student Random Ferns),

A plurality of pedestrian tracking system using a teacher-student random fun, characterized in that the number of ferns (Fern) less than the teacher random fun (Teacher Random Ferns).
The method of claim 11, wherein the tracking unit 500,

A plurality of pedestrian tracking system using a teacher-student random fun, characterized in that to track a plurality of pedestrians by reducing the number of fun (Fern) using the learned teacher-student random fun (Teacher-Student Random Ferns).