KR20230060214A - Apparatus and Method for Tracking Person Image Based on Artificial Intelligence - Google Patents

Abstract

An apparatus and method for tracking an image object based on artificial intelligence include: performing transfer learning for a weight obtained by estimating a human object by pre-leaning and a weight obtaining by estimating the human object by learning a top-view video frame; and estimating the presence of a human object using two weights when extracting the feature of an input image frame. The present invention detects and tracks an object using a weight obtained by estimating a human object by pre-leaning and a weight obtaining by estimating the human object by learning a top-view video frame, thereby achieving a very excellent effect of increasing the tracking accuracy (96%) and detection accuracy (95%) of object tracking technology. The apparatus includes: a first object detection part; a second object detection part; and a transfer learning model part.

Description

Apparatus and Method for Tracking Person Image Based on Artificial Intelligence}

The present invention relates to an apparatus and method for tracking a video object, and more particularly, to transfer learning of weights for estimating a human object by prior learning and weights for estimating a human object by learning a top-view video frame, and An artificial intelligence-based image object tracking device and method for estimating the existence of a human object using two weights when extracting features.

Today, 5G is having a profound impact on video surveillance and monitoring services by processing video streams at high speed with high reliability, high bandwidth and secure network connectivity.

Deep learning-based tracking technology by multi-person tracking and detection framework using this 5G infrastructure is being researched a lot.

In video surveillance, human tracking is important to distinguish various environmental components, such as the deformable nature of the human body, occlusion, lighting and background conditions, especially the visual appearance of a person from others.

Current object tracking technologies have disadvantages in that object detection accuracy and tracking accuracy are somewhat low, or expensive solutions, longer analysis time, and enormous bandwidth requirements occur.

Korean Patent Publication No. 10-2021-0067498

In order to solve this problem, the present invention transfer-learns the weights for estimating the human object by prior learning and the weights for estimating the human object by learning the top-view image frame, and when extracting the features of the input image frame, two An object of the present invention is to provide an artificial intelligence based video object tracking device and method for estimating the existence of a human object using weights.

An artificial intelligence-based video object tracking device according to the features of the present invention for achieving the above object,

A first feature map of an image frame is extracted from a learning dataset storing a training dataset, a verification dataset, and a test dataset, and at least one region in which the existence of a human object is estimated based on the extracted first feature map a first object detection unit that extracts and displays as a first bounding box;

A top-view image frame is received using a camera unit installed in the top-view, a second feature map of the top-view image frame is extracted, and at least one region in which the existence of a human object is estimated is extracted from the image based on the extracted second feature map. a second object detection unit displaying a second bounding box; and

A first weight is generated for a result learned from the learning model for which pretraining has been completed from the first object detector, and a second weight is generated for a result learned from a learning model using a top-view image frame from the second object detector. It includes a transfer learning model unit that does.

An artificial intelligence-based image object tracking method according to the features of the present invention,

The first object detection unit extracts a first feature map of an image frame from a learning dataset storing a training dataset, a verification dataset, and a test dataset, and estimates the presence of a human object in the image based on the extracted first feature map. extracting at least one area to be displayed as a first bounding box;

The second object detection unit receives a top-view image frame using a camera unit installed in the top-view, extracts a second feature map of the top-view image frame, and at least one object for which the existence of a human object is estimated in the image based on the extracted second feature map. extracting an area of and displaying it as a second bounding box; and

The transfer learning model unit generates a first weight for a result learned from a learning model for which pre-learning has been completed from the first object detector, and a second weight for a result learned from a learning model using a top-view image frame from the second object detector. 2 includes generating weights.

According to the configuration described above, the present invention detects and tracks an object using weights estimated for a human object by learning in advance and weights estimated for a human object by learning a top-view image frame in advance, thereby achieving tracking accuracy (96%) of the object tracking technology. ) and detection accuracy (95%) can achieve very good effects.

1 and 2 are diagrams illustrating the configuration of an apparatus for tracking a top-view image object based on transfer learning according to an embodiment of the present invention.
3 and 4 are diagrams schematically illustrating the configuration of a bounding box detection unit according to an embodiment of the present invention.
5 is a diagram illustrating an example of detecting a human object in a top-view image using YOLO.
6 is a block diagram briefly illustrating the internal configuration of an object tracking unit according to an embodiment of the present invention.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

1 and 2 are diagrams showing the configuration of a transfer learning-based top-view image object tracking device according to an embodiment of the present invention, and FIGS. 3 and 4 are schematic diagrams of the configuration of a bounding box detection unit according to an embodiment of the present invention. is the drawing shown.

The present invention briefly shows a series of processes of generating a top-view image sequence from a top-view and performing object detection and object tracking in the top-view image sequence.

The apparatus 100 for tracking an object in a top-view image based on transfer learning according to an embodiment of the present invention includes a bounding box detection unit 200 and an object tracking unit 300.

The bounding box detection unit 200 according to an embodiment of the present invention includes a COCO dataset database unit 210, a first object detection unit 220, an input unit 230, a second object detection unit 240, and a transfer learning model. It includes a unit 250, an object detection model unit 260, a training image input unit 270, and an output unit 280.

A top-view image generator (not shown) generates a top-view image sequence using a camera unit installed in the top-view. Here, the top view is a view looking down from above.

The top-view image generator converts the generated top-view image sequence into an image frame.

The COCO dataset database unit 210 is a dataset created for tasks in the field of computer vision, such as object detection, segmentation, and keypoint detection, and stores training datasets, verification datasets, and test datasets.

The input unit 230 receives an image frame from the top-view image generator.

The first object detection unit 220 collects a training data set, a verification data set, and a test data set from the COCO data set database unit 210, and extracts a feature map of an image frame using pre-trained YOLOv3. Based on the feature map, at least one region in which the presence of a human object is estimated is extracted from the image.

The second object detection unit 240 collects the image frames acquired by the top-view image generator from the input unit 230, extracts a feature map of the top-view image frame using YOLOv3, and extracts a person from the image based on the extracted feature map. At least one region in which the existence of an object is estimated is extracted.

The first object detection unit 220 and the second object detection unit 240 are configured to use deep neural networks (DNNs), convolutional deep neural networks (CNNs), recurrent neural networks (RNNs), and A feature map is extracted from an input image using one of deep belief networks (DBN).

The first object detection unit 220 and the second object detection unit 240 may generate a feature map using a model that has already been learned by the learning unit based on deep learning. Deep learning is a set of machine learning (Machine Learning) algorithms that attempt a high level of abstraction (Abstractions, the task of summarizing key contents or functions in large amounts of data or complex data) through a combination of several nonlinear transformation methods. is defined

The first object detection unit 220 and the second object detection unit 240 extract an area in which an object is estimated to exist in an image frame, and extract a feature map indicating a feature from the extracted area.

The first object detection unit 220 and the second object detection unit 240 extract at least one region in which the existence of a human object is estimated from the image based on the extracted feature map. A method for extracting a region may include, for example, faster RCNN, SSD (Single Shot MultiBox Detector), YOLO (You Only Look Once), and the like, and the present invention takes the YOLO object recognition module (YOLOv3) as an example.

The first object detection unit 220 and the second object detection unit 240 select a feature map including coordinates of classes for each region of the image from among the feature maps, and identify coordinates for distinguishing regions from the selected feature maps. Then, the identified coordinates can be extracted as a region in which the entity's existence is estimated.

The first object detection unit 220 and the second object detection unit 240 may set one or two or more human objects.

The first object detection unit 220 and the second object detection unit 240 may display each of the at least one extracted region as a bounding box surrounding the outermost periphery of the object.

Each bounding box indicates that there is a possibility that an object exists at the location of the corresponding bounding box in the image.

The YOLOv3 model uses a single network architecture to predict class probabilities and corresponding bounding boxes for the entire input image.

The YOLOv3 model contains 24 convolutional layers and 2 fully connected layers. Convolutional layers are used for image extraction, while fully connected layers compute class predictions and probabilities.

While the first object detection unit 220 and the second object detection unit 240 detect a person, the model divides the input image frame into S×S regions, also referred to as grid cells, as shown in FIG. 5 . These grid cells are linked to bounding beat predictions and class probabilities. Each cell predicts the probability of whether or not the center of a person is in a grid cell. If the prediction is positive, the bounding box and confidence value for each positive detection is predicted.

The first object detection unit 220 and the second object detection unit 240 define a confidence value (Conf(person)) representing the degree of the detected bounding box as a person by Equation 1.

Here, Pr (person) indicates whether there is a person in the predicted bounding box (eg: 1, no: 0), and IOU (Pred, Truth) is the overlapping area of the predicted bounding box and the actual bounding box, which is the sum of the two areas. The IoU overlap ratio divided by the width.

Here, BoxT represents the ground truth in the training set (Truth), and BoxP represents the predicted bounding box (Pred).

Ground truth means the actual correct answer level (Label), not the value predicted by the neural network model. That is, it represents the actual location of an object.

IoU indicates the location of an object with a bounding box in object detection, at this time, the areas of the predicted bounding box and the actual ground true bounding box are compared, and when the two areas are compared, the overlapping area is divided by the combined area of the two areas IoU overlap When the ratio is a predetermined threshold value (0.5 or more), it is said to be matched.

If the IoU value is greater than 0.5, the Region is viewed as an object, and labeled with the same class as Ground True.

The first object detection unit 220 and the second object detection unit 240 select and predict a region suitable for human detection in the top-view image frame, and obtain a desired bounding box using a reliability value after prediction.

The first object detection unit 220 and the second object detection unit 240 predict five values for all bounding boxes including h, w, x, y and reliability values. Here, the width and height are expressed as w and h, and the coordinates of the center of the bounding box are expressed as x and y.

The first object detection unit 220 and the second object detection unit 240 define a threshold value, discard low-scoring confidence values, process the remaining multiple high-confidence bounding boxes, and use non-maximal suppression to determine the final location parameters. derive a variable

The first object detection unit 220 and the second object detection unit 240 calculate a loss function for the detected bounding box.

The first object detection unit 220 and the second object detection unit 240 learn the characteristics of the image frame so that the total sum of the calculated loss functions is minimized.

The loss function is the sum of the regression loss and the classification loss. The first object detection unit 220 and the second object detection unit 240 consider only one object, that is, a person, and the loss function for this task is given as Equation 3 below.

는 예측된 바운딩 박스의 좌표 손실을 나타내고,

는 실제 바운딩 박스의 좌표 손실을 계산하는데 사용된다. 탑뷰 영상 프레임에서 사람의 크기가 다르며 YOLOv3 손실 특징은 모든 바운딩 박스에 대해 동일한 손실을 계산한다. 그러나 손실 함수에 대한 크고 작은 객체의 영향은 전체 이미지에 대해 다릅니다. 따라서, 좌표의 손실 함수를 개선하기 위해서 대조(Contrast) 정규화가 사용된다. 좌표

의 손실함수는 하기의 수학식 4로 주어진다.here,

denotes the coordinate loss of the predicted bounding box,

is used to calculate the coordinate loss of the actual bounding box. The size of the person in the top-view video frame is different, and the YOLOv3 loss feature calculates the same loss for all bounding boxes. However, the influence of large and small objects on the loss function is different for the entire image. Therefore, contrast normalization is used to improve the coordinate loss function. coordinate

The loss function of is given by Equation 4 below.

는 바운딩 박스의 좌표 예측에 사용하는 스케일 매개변수이고(

),

는 i번째 셀에서 감지된 바운딩 박스의 예측 위치이고,

는 i번째 셀에서 바운딩 박스의 실제 위치이다.here,

is the scale parameter used to predict the coordinates of the bounding box (

),

is the predicted position of the bounding box detected in the i-th cell,

is the actual location of the bounding box in the i-th cell.

Equation 4 calculates the loss function associated with the predicted bounding box with coordinate values x and y.

는 j번째 바운딩 박스에서 감지된 사람의 가능성을 보여준다.

는 일정한 상수이다.

shows the probability of a person detected in the j-th bounding box.

is a constant constant.

는 (j = 0 내지 B)를 각 그리드 셀(i = 0 내지 S²)에 대한 예측 변수로 사용하여 각 바운딩 박스에 대한 합계를 계산한다.

calculates the sum for each bounding box using (j = 0 to B) as a predictor for each grid cell (i = 0 to S ² ).

의

는 하기의 수학식 5로 계산된다.

of

Is calculated by Equation 5 below.

는 분류 오류를 나타내고,

와

는 예측된 바운딩 박스의 i번째 그리드 셀의 신뢰도 값과 원본 슬라이딩 윈도우의 i번째 그리드 셀의 신뢰도 값이고,

는 그리드 셀 i의 j번째 바운딩 박스에서 사람이 감지되는지 여부를 나타낸다. 대상 사람이 j번째 경계 상자와 i번째 격자 셀에 있는 경우 수학식 5의 함수는 1과 같고, 그렇지 않으면 0이 된다.here,

represents the classification error,

and

is the confidence value of the ith grid cell of the predicted bounding box and the confidence value of the ith grid cell of the original sliding window,

represents whether a person is detected in the j-th bounding box of grid cell i. The function of Equation 5 is equal to 1 if the target person is in the j-th bounding box and the i-th grid cell, and 0 otherwise.

는 그리드 셀 i의 j번째 바운딩 박스에서 사람이 감지되지 않는지 여부를 나타낸다.

represents whether a person is not detected in the j-th bounding box of grid cell i.

The transfer learning model unit 250 generates a first weight for a result learned from the learning model for which pre-learning has been completed from the first object detection unit 220, and learns using the top-view image frame from the second object detection unit 240. A second weight for a result learned from the model is generated.

The training image input unit 270 generates a training image frame that is a top-view image.

The object detection model unit 260 receives a training image frame, which is a top-view image, and extracts a feature map of the input training image frame using the first and second weights generated by the transfer learning model unit 250, Based on the extracted feature map, at least one region in which the presence of a human object is estimated is extracted from the image, displayed as a bounding box, and output to the output unit 280.

6 is a block diagram briefly illustrating the internal configuration of an object tracking unit according to an embodiment of the present invention.

The object tracking unit 300 applies deep SORT, a tracking algorithm based on deep learning, to track a human object in a top-view image frame.

The object tracking unit 300 includes an input module 310, a bounding box tracking unit 320, a controller 330, a CNN model unit 340, a distance calculation unit 350, and a tracking result unit 360.

The input module 310 receives an image frame displaying a human object as a bounding box from the object detection model unit 260 of the bounding box detection unit 200 .

The bounding box tracking unit 320 uses a Kalman filter to track bounding box information.

The bounding box tracking unit 320 tracks bounding box information included in a plurality of temporally received image frames by using a Kalman filter.

A Kalman filter is a recursive filter that estimates a linear state based on measurements (observations) that contain errors (noise).

The Kalman filter predicts the current position using the previously accumulated past tracking data, and if there is matching observed data among the newly received detected data, the ratio of the variance value between the predicted data and the observed data is used to determine which of the two The final state is determined by the ratio of dispersion.

The Kalman filter predicts the state (position and size) of an object through the Kalman filter from previously accumulated tracking data. The Kalman filter is a well-known tracking algorithm, and its detailed description is omitted.

Kalman filtering uses spatial coordinate information to track objects in the current video frame.

Spatial coordinate information is as follows.

는 감지된 바운딩 박스의 각 좌표의 추적 속도이고,

는 바운딩 박스의 위치이다.

is the tracking speed of each coordinate of the detected bounding box,

is the location of the bounding box.

, 높이 h이다.u, v are the bounding box center positions, and the aspect ratio

, with height h.

The bounding box tracking unit 320 performs tracking prediction using spatial coordinate information of the bounding box.

The control unit 330 receives space information and tracking information of the bounding box from the bounding box tracking unit 320 and transmits them to the CNN model unit 340.

The CNN model unit 340 includes a convolutional neural network (CNN) and a plurality of filters, and the CNN may include convolutional layers designed to perform convolutional operations.

A convolution layer constituting a CNN may perform a convolution operation related to an input using a kernel.

The CNN model unit 340 extracts a feature vector including shape (appearance) information of the bounding box using CNN.

The distance calculation unit 350 uses the Mahalanobis distance to calculate the distance between the previous detected object tracking and the next detected object tracking.

The cost matrix is used to represent the shape in the Deep SORT algorithm.

The distance calculation unit 350 indicates spatial similarity between new detection and tracking using the two distance values calculated by Equation 6.

는

투영 추적이 측정 공간이고,

를 위한 새로운 탐지

가 사용됨을 나타낸다. T는 전치행렬을 나타낸다.here,

Is

the projection trace is the measurement space,

new detection for

indicates that is used. T represents a transposed matrix.

는 각 바운딩 박스 탐지를 나타낸다.

denotes each bounding box detection.

와 측정된 위치의

추적(Track) 간의 차이를 계산하여 마할라노비스 거리(Mahalanobis Distance)를 획득한다.The distance calculation unit 350 detects a new

and the measured position of

The difference between tracks is calculated to obtain the Mahalanobis distance.

) 추적(Track)과 j번째(

) 탐지 사이의 마할라노비스 거리 임계값을 이용하여 다음의 수학식 7에 의해 결정 지표를 계산한다.The distance calculation unit 350 calculates the distance value of Equation 6 and the i-th (

) track and the jth (

) Calculate the decision index by the following Equation 7 using the Mahalanobis distance threshold between detections.

Here, t ⁽¹⁾ is a preset Mahalanobis distance threshold.

The distance calculating unit 350 evaluates to 1 when there is a correlation between the i-th traces, and the j-th detection is permitted.

The distance calculation unit 350 calculates a second distance value representing shape information by Equation 8 below.

추적(Track)과

탐지(Detection) 사이의 최소 코사인 거리를 계산한다.The second distance value (d ⁽²⁾ (i, j)) is as shown in Equation 8,

Track and

Calculate the minimum cosine distance between detections.

과

은 모양 디스크립터(Apperance Descript)를 나타내고,

는 i번째 추적에 있는 100개 이상의 객체(사람)의 모양을 나타내는데 사용된다. 연관 추적 사이의 임계값을 설정하기 위해서는 다음의 수학식 9를 사용한다.here,

class

Represents an appearance descriptor (Apperance Descript),

is used to represent the shape of 100 or more objects (people) in the ith trace. In order to set the threshold value between the association traces, the following Equation 9 is used.

의 갤러리(Gallery)(

) 상태를 유지한다.The distance calculation unit 350 is the last associated with the shape descriptor for each tracking k.

's Gallery (

) state.

The distance calculating unit 350 uses Equation 9 below to set a threshold value between association tracking.

If the distance value is small, it becomes 1, and if the distance value is large, it becomes 0. The distance calculation unit 350 combines the cost function (cost matrix, C _i,j ) of Equation 9 by Equation 10 below. In other words, to create an association problem, two metrics are combined using a weighted sum (Equation 10).

를 통해 제어할 수 있다.The impact of each metric on the combined associated cost is a hyperparameter

can be controlled through

The distance calculation unit 350 defines a gate function (gate matrix, b _i,j ) of Equation 11 below to match spatial information and shape information.

An association is said to be acceptable if it falls within the gating domain of the two metrics in Equation 11.

The value of Equation 11 is equal to 0 if the shape and space gate functions are not the same, and equal to 1 if they are the same.

It also indicates that (i, j) is a true match between shape and spatial information. Therefore, detection and tracking in every new video frame involves utilizing the above cost and gate functions.

The tracking result unit 360 tracks in the video sequence and continues tracking in the next new video frame if the new detection is effectively associated with the current tracking.

The tracking result section 360 is set to 0 if a new detection and tracking are connected or do not match.

Therefore, in this case, the tracking result unit 360 fails to detect a new one. In this case, new detections are not linked to existing detections in frame f. A new detection is then initialized with a temporary trace.

Therefore, in the tracking result unit 360, when the threshold value between related tracking is set (Equation 9), the Deep SORT algorithm continues to verify, and the following (f + 1), (f + 2), ... (f + t) Connect with the new detection in the temporary frame, and if successfully connected, the corresponding trace is trace confirmed and updated. Otherwise, it is deleted immediately.

The tracking result unit 360 is immediately deleted if a threshold value between related tracking is not set.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

100: top view video object tracking device
200: bounding box detection unit
300: object tracking unit

Claims (10)

A first feature map of an image frame is extracted from a learning dataset storing a training dataset, a verification dataset, and a test dataset, and at least one region in which the existence of a human object is estimated based on the extracted first feature map a first object detection unit that extracts and displays as a first bounding box;
A top-view image frame is received using a camera unit installed in the top-view, a second feature map of the top-view image frame is extracted, and at least one region in which the existence of a human object is estimated is extracted from the image based on the extracted second feature map. a second object detection unit displaying a second bounding box; and
A first weight is generated for a result learned from the learning model for which pretraining has been completed from the first object detector, and a second weight is generated for a result learned from a learning model using a top-view image frame from the second object detector. An artificial intelligence-based video object tracking device comprising a transfer learning model unit that The method of claim 1,
A training image frame, which is a top-view image, is input, and a third feature map of the input training image frame is extracted using the first weight and the second weight generated by the transfer learning model unit, and the extracted third feature map is based on the extracted third feature map. An artificial intelligence based image object tracking apparatus further comprising an object detection model unit extracting at least one region in which the existence of a human object is estimated from the raw image and displaying the extracted region as a third bounding box. The method of claim 2,
Bounding for receiving image frames in which a human object is displayed as a bounding box from the object detection model unit and tracking bounding box information included in a plurality of temporally received image frames by using a Kalman filter to track the bounding box information box tracking; and
Composed of a Convolutional Proposal Network (CNN) and a plurality of filters, spatial information and tracking information of the bounding box are received from the bounding box tracking unit, and feature vectors including shape (appearance) information of the bounding box are extracted. An artificial intelligence-based image object tracking device further comprising an object tracking unit consisting of a CNN model unit to The method of claim 1,
The first object detection unit and the second object detection unit track an artificial intelligence-based image object defining a reliability value (Conf(person)) representing the degree of the detected bounding box as a person by Equations 1 and 2 below. Device.
[Equation 1]

Here, Pr (person) indicates whether there is a person in the predicted bounding box (eg: 1, no: 0), and IOU (Pred, Truth) is the overlapping area of the predicted bounding box and the actual bounding box, which is the sum of the two areas. IoU overlap ratio divided by width.
[Equation 2]

Here, BoxT represents the ground truth in the training set (Truth), and BoxP represents the predicted bounding box (Pred). The method of claim 1,
The first object detection unit and the second object detection unit learn characteristics of an image frame such that a total sum of loss functions of Equations 3, 4, and 5 below is minimized.
[Equation 3]

here,

denotes the coordinate loss of the predicted bounding box,

is used to calculate the coordinate loss of the actual bounding box.
[Equation 4]

here,

is the scale parameter used for predicting the coordinates of the bounding box (

),

is the predicted position of the bounding box detected in the i-th cell,

is the actual position of the bounding box in the i-th cell.
[Equation 5]

here,

represents the classification error,

and

is the confidence value of the ith grid cell of the predicted bounding box and the confidence value of the ith grid cell of the original sliding window,

Indicates whether a person is detected in the j-th bounding box of grid cell i,

Indicates whether a person is not detected in the j-th bounding box of grid cell i,

calculates the sum for each bounding box using (j = 0 to B) as a predictor for each grid cell (i = 0 to S ² ). The first object detection unit extracts a first feature map of an image frame from a learning dataset storing a training dataset, a verification dataset, and a test dataset, and estimates the presence of a human object in the image based on the extracted first feature map. extracting at least one area to be displayed as a first bounding box;
The second object detection unit receives a top-view image frame using a camera unit installed in the top-view, extracts a second feature map of the top-view image frame, and at least one object for which the existence of a human object is estimated in the image based on the extracted second feature map. extracting an area of and displaying it as a second bounding box; and
The transfer learning model unit generates a first weight for a result learned from a learning model for which pre-learning has been completed from the first object detector, and a second weight for a result learned from a learning model using a top-view image frame from the second object detector. 2 An artificial intelligence-based video object tracking method comprising generating weights. The method of claim 6,
The object detection model unit receives a training image frame that is a top-view image, extracts a third feature map of the input training image frame using the first and second weights generated by the transfer learning model unit, and extracts the extracted third feature map. The method of tracking an image object based on artificial intelligence further comprising extracting at least one region in an image in which the presence of a human object is estimated based on the feature map and displaying the extracted region as a third bounding box. The method of claim 7,
The bounding box tracking unit receives an image frame in which a human object is displayed as a bounding box from the object detection model unit, and uses a Kalman filter to track the bounding box information, and the bounding box information included in the plurality of temporally received image frames. tracking; and
The CNN model unit consists of a Convolutional Proposal Network (CNN) and a plurality of filters, receives spatial information and tracking information of the bounding box from the bounding box tracking unit, and features including shape (appearance) information of the bounding box. An artificial intelligence-based image object tracking method further comprising the step of extracting a vector. The method of claim 6,
Artificial intelligence including the step of defining a confidence value (Conf(person)) representing the degree of the detected bounding box as a person by the first object detection unit and the second object detection unit by Equation 1 and Equation 2 below. Based video object tracking method.
[Equation 1]

Here, Pr (person) indicates whether there is a person in the predicted bounding box (eg: 1, no: 0), and IOU (Pred, Truth) is the overlapping area of the predicted bounding box and the actual bounding box, which is the sum of the two areas. IoU overlap ratio divided by width.
[Equation 2]

Here, BoxT represents the ground truth in the training set (Truth), and BoxP represents the predicted bounding box (Pred). The method of claim 6,
The first object detection unit and the second object detection unit learning characteristics of image frames such that a sum of loss functions of Equations 3, 4, and 5 below is minimized. Object tracking method.
[Equation 3]

here,

denotes the coordinate loss of the predicted bounding box,

is used to calculate the coordinate loss of the actual bounding box.
[Equation 4]

here,

is the scale parameter used to predict the coordinates of the bounding box (

),

is the predicted position of the bounding box detected in the i-th cell,

is the actual position of the bounding box in the i-th cell.
[Equation 5]

here,

represents the classification error,

and

is the confidence value of the ith grid cell of the predicted bounding box and the confidence value of the ith grid cell of the original sliding window,

Indicates whether a person is detected in the j-th bounding box of grid cell i,

Indicates whether a person is not detected in the j-th bounding box of grid cell i,

calculates the sum for each bounding box using (j = 0 to B) as a predictor for each grid cell (i = 0 to S ² ).

Images