KR20160078561A

KR20160078561A - Real Time Image Recognition Method in Low Performance Video Device

Info

Publication number: KR20160078561A
Application number: KR1020140187887A
Authority: KR
Inventors: 윤주홍; 황영배; 최병호; 김정호
Original assignee: 전자부품연구원
Priority date: 2014-12-24
Filing date: 2014-12-24
Publication date: 2016-07-05
Also published as: KR101641647B1; WO2016104831A1

Abstract

Provided is a real-time image recognition method in a low performance video device. The image recognition method according to an embodiment of the present invention detects an interested object in an image on a first cycle, and tracks the interested object in the image on a second cycle. The first cycle is longer than the second cycle. The image recognition method sets an operation frequency of a detector and a tracker adaptively, and can perform real-time image recognition even in a low performance embedded environment.

Description

TECHNICAL FIELD [0001] The present invention relates to a real-time image recognition method for a low-

The present invention relates to an image recognition technology, and more particularly, to an image recognition method for detecting and tracking an object of interest in real time from an image captured by a video device.

Real-time (per frame) image recognition is indispensable to perform functions such as human interaction based on motion recognition, personal information protection, and vehicle collision prevention.

Conventional image recognition algorithms show good performance in high-end devices such as PCs, but it is very difficult to operate in real-time in embedded environments such as mobile phones, tablet PCs, and automobile black boxes due to their high computational complexity.

Therefore, in a vehicle black box equipped with a low-end embedded platform, the false detection rate of objects (pedestrians, faces, license plates, vehicles) is relatively high in an actual driving environment.

This can degrade the mosaic processing function and the vehicle collision prevention function that are based on image recognition, which is more problematic. Therefore, it is required to search for a method for realizing stable image recognition in real time in a low-end video device such as a black box for a vehicle.

It is an object of the present invention to adaptively operate a detector and a tracker to enable real-time image recognition even in a low-end embedded environment.

It is another object of the present invention to provide a real-time image recognition method capable of adaptively adjusting an operation plan of a detector and a tracker using camera information and image information.

It is another object of the present invention to provide an image processing apparatus and method that can be used to extract unique information (skins, borders, etc.) in an image, geometrical structure information of a scene viewed by a camera, And a motion estimation unit for estimating a motion of the object based on the estimated movement information.

According to an aspect of the present invention, there is provided an image recognition method including: detecting an object of interest in a first period in an image; And tracking the object of interest in a second period in the image, wherein the first period is longer than the second period.

The method of recognizing an image according to an embodiment of the present invention may further comprise setting the first period and the second period with reference to at least one of feature information of the image and motion information of a camera that captures the image ; &Lt; / RTI >

According to another aspect of the present invention, there is provided a method of recognizing an image, comprising: setting an object of interest through an interpolation method in a frame in which detection and tracking are not performed among frames constituting an image.

The detecting step may detect an object of interest including at least one of a face and a license plate using at least one of skin information and edge information.

Also, the detecting step may detect the object of interest only in an area where the skin density and the edge density are equal to or more than the reference.

The detection step and the tracking step may set the detection and tracking area by referring to the geometric structure information of the scene viewed by the camera that captures the image.

In addition, the tracking step may set a tracking area based on an object motion size per area.

According to another aspect of the present invention, there is provided an image recognition method including: a detector for detecting an object of interest in a first period in an image; A tracker for tracking the object of interest in a second period in the image; And a setting unit setting the first period to be longer than the second period.

As described above, according to the embodiments of the present invention, the number of operations of the detector and the tracker is adaptively set, and real-time image recognition is possible even in a low-end embedded environment.

In addition, it is possible to use the inherent information (skin, boundary line, etc.) in the image, the geometrical structure information of the scene viewed by the camera, the size information of the object due to the structural features of the image, motion restriction information, The search area can be reduced to a minimum, the amount of calculation can be minimized, and the false detection rate of the object can be drastically reduced.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating a video apparatus according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a result of setting an image recognition operation plan;
3 is a diagram illustrating a face detection result using skin information,
4 is a diagram illustrating a result of search area setting using skin information,
5 is a diagram illustrating a result of minimizing a search area using edge information,
6 is a diagram illustrating a result of license plate detection using edge information,
7 is a diagram illustrating a result of setting a license plate search area using geometric structure information of a video scene,
8 is a diagram illustrating search area setting results for vehicle and pedestrian detection using geometric structure information of a video scene,
9 is a diagram illustrating a sliding window search and a similarity map obtained as a result,
10 is a diagram illustrating motion information of an object and motion information of a scene.

Hereinafter, the present invention will be described in detail with reference to the drawings.

One. Low Embedded Real-time image recognition in platform environment

A real-time image recognition method according to an embodiment of the present invention realizes detection / tracking of an object of interest in a low-end platform with respect to an image captured through a camera.

Particularly, the real-time image recognition method according to the embodiment of the present invention can prevent personal information (mosaic processing of face and license plate), human interaction system, recognition of a crime vehicle, prevention of a vehicle collision through recognition of a pedestrian / And it seeks detection / tracking of all kinds of objects, not one kind of object detection / tracking.

In the real-time image recognition method according to the embodiment of the present invention, a detector having a relatively large amount of computation is operated at a minimum, a tracker with a small amount of computation is supplemented with interpolation, .

1 is a view for explaining a video apparatus according to an embodiment of the present invention. As shown in FIG. 1, an imaging apparatus according to an embodiment of the present invention includes an operation plan setting unit 110, a detector 120, and a tracker 130.

The detector 120 detects an object of interest from the image photographed through the camera using an object detection algorithm and the tracker 130 tracks the object of interest detected by the detector 120 using an object tracking algorithm.

The operation plan setting unit 110 sets the operation period of the detector 120 and the tracker 130. When setting the operation period of the detector 120 and the tracker 130, the operation plan setting unit 110 can refer to the feature information (skin information, edge information, etc.) of the image and the camera information (position information, speed information, have.

2. Setting the image recognition operation (object detection / tracking) plan

Detection performance may be good if the object 120 is detected by the detector 120 for every frame constituting the image, but it is impossible in a low-end platform due to high calculation amount.

Accordingly, the operation plan setting unit 110 operates both the detector 120 and the tracker 130 with a relatively small amount of computation to assist both the speed and the detection rate.

For frames in which both the detector 120 and the tracker 120 are not operated, the object position is interpolated and filled with motion information that can be calculated from the tracking results.

2 is a diagram illustrating the result of the image recognition operation plan setting. In FIG. 2, the detector 120 operates once every 30 frames, the tracker 130 acts once every three frames to replace the role of the detector 10, and the frames in which the tracker 130 does not operate are moved It can be confirmed that the object position is interpolated and filled with information based link.

In order to more effectively set the operation period of the detector 120 and the tracker 130, the controller 120 refers to the feature information (skin information, edge information, etc.) and camera information (position information, 120 and the tracker 130 can be adaptively set according to the situation.

For example, when the position of the camera (vehicle) is a highway or when the speed of the camera (vehicle) is equal to or higher than a constant speed, the face detection function of the detector 120 is stopped. When the speed of the expressway or the camera (vehicle) is high, the image blur due to the high speed occurs and the face information is severely distorted. Therefore, the face detection is not necessary for protecting the information.

On the other hand, when the complexity of the image is high, it is possible to reduce the calculation amount by setting the detection period to be longer than in the case where the image complexity is high. In addition, it is also possible to set the detection period to be short at a time point where the scene change is fast (for example, at the time of rotation), thereby increasing the detection accuracy.

3. Reduce search area using image features

Retrieving the entire image area for object detection not only increases the amount of computation that is unnecessary but also causes another increase in false detection occurrence. In order to reduce the amount of computation and to minimize the false positives, it is necessary to utilize the independent feature information of the object which can distinguish the object from the background as well as the data obtained by learning the object.

In order to improve the speed of the detector 120, the characteristic feature information of each object is used in the image while minimizing the false detection rate of the image recognition. Specifically, skin color (skin color) information, which is color information that is common to faces, is utilized, and edge information existing around an object is utilized in the case of license plates, vehicles, and pedestrians.

The detector 120 does not search the entire area of the image but searches only the area where the skin information and the boundary information exist for the object of interest.

In the case of using the skin color information, as shown in FIG. 3, it is possible to effectively remove the background, which is not the skin color, as shown in FIG. 4, thereby significantly reducing the face detection area, do.

When the edge information is utilized, the background can be effectively removed as shown in FIG. 5, and the object detection area is significantly reduced, thereby reducing the calculation amount. 6 illustrates the result of plate detection using edge information.

Since the area designated as the background is not searched, the false detection rate occurring in the background area can be greatly reduced. On the other hand, skin / edge density for each detection result can be additionally considered to further reduce false positives.

The skin / edge density is a value obtained by dividing the number of skin color / edge pixels in the detected object by the width of the detected object, and the detection result in which this value is equal to or less than the threshold value is treated as low reliability.

4. Reduction of search area using geometric structure information of video scene

The detector 120 and the tracker 130 set an effective search area using the geometric structure information of the scene viewed by the camera. That is, as shown in FIGS. 7 and 8, the area where the object is located in the image can be roughly determined according to the object size in the image due to the structure of the scene. By using this, it is possible to eliminate the unnecessary calculation amount consumed in the search considering only the size of the object suited to the area, rather than searching the object considering various sizes in all areas.

5. Shrinking the search area using the appearance model and feature information of the object

The tracker 130 performs a search using an external appearance model of an object within a predetermined area, as shown in FIG. 9, using the detected result. As shown in FIG. 9, the position of the pixel having the highest similarity in the similarity map obtained after the search of the sliding window is the result of tracking.

In addition, the tracking performance can be improved by utilizing the feature information of the object in addition to the appearance model of the object. In case of face, skin density information obtained from face detection is used, and edge density information such as license plate is used to divert to the background area during tracking to prevent tracking failure.

FIG. 10 shows a method of improving the object tracking performance by using motion information in a specific region of the image as a prior information, on the assumption that the movement of objects moves constantly for a short time. As shown in FIG. 10, since a large motion is generated in the left / right area of the image, the search range is enlarged, and the large range of motion is not generated in the center area of the image.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

110: Operation plan setting unit
120: detector
130: Tracker

Claims

Detecting an object of interest in a first period in an image; And
And tracking the object of interest in a second period in the image,
Wherein the first period is longer than the second period.

The method according to claim 1,
And setting the first period and the second period with reference to at least one of feature information of the image and motion information of a camera that captures the image.

The method according to claim 1,
And setting an object of interest through interpolation in a frame in which detection and tracking are not performed among frames constituting an image.

The method according to claim 1,
Wherein the detecting step comprises:
Wherein at least one of face information and edge information is used to detect an object of interest including at least one of a face and a license plate.

The method of claim 4,
Wherein the detecting step comprises:
Wherein the object of interest is detected only in an area where the skin density and the edge density are equal to or more than the reference.

The method of claim 1,
Wherein the detecting and tracking comprises:
Wherein a detection and tracking area is set by referring to geometrical structure information of a scene viewed by a camera that photographs the image.

The method according to claim 1,
Wherein the tracking step comprises:
And setting a tracking area based on the size of the object motion per area.

A detector for detecting an object of interest in a first period of the image;
A tracker for tracking the object of interest in a second period in the image; And
And a setting unit configured to set the first period to be longer than the second period.