KR20110035662A - Intelligent image search method and system using surveillance camera - Google Patents

Abstract

PURPOSE: An intelligent image search system using a surveillance camera is provided to offer a multiple pedestrian sensing and tracing method. CONSTITUTION: An intelligent image search system comprises a plurality of surveillance cameras(102) for recording a monitoring target area, an image processing server(104) extracts pedestrian characteristic information based on the image recorded by the surveillance cameras and detects the image in which the pedestrian characteristic information matched with the pedestrian characteristic information, and a user interface(108) for displaying the detected image or inputting the pedestrian characteristic information for searching.

Description

Intelligent image search method and system using surveillance cameras {INTELLIGENT IMAGE SEARCH METHOD AND SYSTEM USING SURVEILLANCE CAMERA}

The present invention relates to an image retrieval system and a method thereof, and more particularly to an intelligent image retrieval system and method using a surveillance camera.

Video surveillance systems that detect the occurrence of a specific activity or event are the most important of all research studies using human tracking. The reason for this is that as society develops, the importance of the safety of individuals and facilities in public spaces as well as in private spaces is highly recognized.

As the modern society becomes more information, unmanned, automated, and computerized, warnings about the safety and security of the property and safety of the company and the workplace continue to emerge. As a result, efforts are being made to protect and manage the property and safety of themselves and the workplace, and the importance and scope of security is expanding to major facilities, government offices, schools, businesses, and homes. The need for

In particular, CCTV technology and IP camera network technology, which is a video surveillance and security system, have recently been rapidly developed and intelligent. There are three ways for a video surveillance system to receive artificial intelligence and become an intelligent system.

The first is to implement a function of collecting a plurality of images from a separate server here, and analyzing the collected images in software form. The second method is to add intelligent functions, which are one step ahead of motion detection, to DVRs that store, record and play back images captured by CCTV cameras. Finally, there is a way to run the software in the camera itself, in the chip in the encoder equipment, or via a network connection.

Among these, an intelligent video surveillance system that runs software on a separate server to enable various intelligent functions is in line with the development of computer software. As software for computers began to be applied to video image processing, images were analyzed, tracked, and classified. The emergence of such intelligent surveillance system goes beyond simply checking image data input from each camera, and establishes a video surveillance system that detects and tracks objects of interest in the image data through computer technology and image processing software. It also means that you can.

In order to efficiently implement such an intelligent video surveillance system, stable detection and tracking of multiple pedestrians from camera images, feature extraction of pedestrians, and pedestrian feature-based image retrieval techniques are required.

The present invention provides a multi-pedestrian detection and tracking method for supporting an intelligent search for the surveillance camera image.

The present invention also provides a method for extracting the characteristics of multiple pedestrians input from a surveillance camera and searching for pedestrians desired by a user based on the extracted multiple pedestrian characteristics.

The present invention extracts pedestrian characteristic information based on a plurality of surveillance cameras photographing a surveillance target area and images captured by the plurality of surveillance cameras, and matches the pedestrian characteristic information to be searched among the extracted pedestrian characteristic information. An image processing server for detecting an image from which pedestrian characteristic information is extracted and a user interface unit for inputting the pedestrian characteristic information to be searched or displaying the detected image on an image search screen are provided.

The present invention also provides a process of receiving images captured from a plurality of surveillance cameras, extracting characteristic information of a pedestrian based on the received image, and searching for input from the user among the extracted pedestrian characteristic information. An intelligent image retrieval method includes detecting an image from which pedestrian characteristic information matching the pedestrian characteristic information is extracted and displaying the detected image on an image search screen.

In addition, the present invention is an intelligent image retrieval system having a plurality of surveillance cameras, to extract the characteristic information of the pedestrian based on the images captured by the plurality of surveillance cameras, and to search among the extracted pedestrian characteristic information An intelligent image retrieval system including an image processing apparatus for detecting an image from which pedestrian characteristic information matching the pedestrian characteristic information is extracted is provided.

The present invention can stably detect and track multiple pedestrians from the surveillance camera image to extract the characteristics of the pedestrians.

In addition, the present invention receives a real-time image from a plurality of surveillance cameras installed in the surveillance target area, if the user who wants to search the online or offline image inputs the pedestrian characteristic information to search for detecting the pedestrian corresponding to the characteristic information It can be displayed on an online video image or an offline representative frame image.

In the following description of the present invention, detailed descriptions of well-known functions or configurations will be omitted if it is determined that the detailed description of the present invention may unnecessarily obscure the subject matter of the present invention. Terms to be described later are terms defined in consideration of functions in the present invention, and may be changed according to intentions or customs of users or operators. Therefore, the definition should be based on the contents throughout this specification.

An embodiment of the present invention provides an image retrieval technology based on the characteristics of pedestrians by stably detecting and tracking multiple pedestrians and extracting the characteristics of each of the multiple pedestrians to support intelligent search for surveillance camera images.

Specifically, an embodiment of the present invention provides a method for detecting multiple pedestrians, a method for tracking the multiple pedestrians, and a method for extracting characteristics of each of the multiple pedestrians based on a real-time image input from a surveillance camera. We can search pedestrian to do efficiently.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a surveillance camera image-based intelligent search system according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the system 100 includes a surveillance camera 102, an image processing server 104, an image data storage unit 106, and a user interface unit 108. It will be apparent to those skilled in the art that the image processing server 104 may be implemented in an image processing apparatus (not shown), and the image processing apparatus may include the surveillance camera 102. Hereinafter, other components of the system 100 that are not related to the contents of the present invention will be omitted.

The surveillance camera 102 is installed in the surveillance target area to photograph the position and movement of the pedestrian.

The image processing server 104 detects and tracks pedestrians based on a real-time image input from the surveillance camera 102 and extracts characteristics of pedestrians. In addition, the image processing server 104 performs a function of searching for a pedestrian desired by the user by comparing the pedestrian characteristic information input by the user with the pedestrian characteristic information input from the surveillance camera. Meanwhile, the image processing server 104 receives a real time image from the surveillance camera 102, extracts representative frames, and stores the extracted representative frames in the image data storage 106.

When the pedestrian characteristic information is input by the user who wants to search the online or offline image, the user interface unit 108 provides an image search window for displaying the pedestrian matching the pedestrian characteristic information on the online video image or the offline representative frame image. do. The image search window may be divided into a real time image search mode and an offline representative image search mode. In addition, the image search window is provided with a query input window so that the user can input the characteristic information of the pedestrian to search.

Hereinafter, the configuration and operation of the image processing server 104 will be described in detail with reference to the accompanying drawings.

2 is a block diagram of an image processing server according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the image processing server 104 includes three main modules, an image and query inputter 202, a pedestrian tracking and feature extractor 204, and a feature matching and result outputter 206.

The image and query inputter 202 may include a surveillance camera image inputter 208, a still image / metadata inputter 210, and a query inputter 212.

The surveillance camera image inputter 208 receives a video signal in the form of a data stream from the surveillance camera 102 and transmits the image signal to the pedestrian tracking and feature extractor 204. The surveillance camera image inputter can simultaneously check images of up to two surveillance cameras, and can access a surveillance camera given an arbitrary authority according to IP address designation.

The still image / metadata input unit 210 reads a still original image, a background subtraction image, and pedestrian area information for each frame image from the image data storage unit 106 and provides the pedestrian tracking and feature extractor 204.

The query inputter 212 may provide the pedestrian characteristic information to the characteristic matching and result outputter 206 when the characteristic information of the pedestrian to be searched from the user interface unit 108 is input. The characteristic information of the pedestrian may be information such as the height of the pedestrian, the hat color, and the color of the clothes to be worn.

The pedestrian tracking and feature extractor 204 may include an optical flow analyzer 214, an adaptive background subtractor 216, a pedestrian tracker 218, a representative frame extractor 220, and a pedestrian feature extractor 222. .

The pedestrian tracking and feature extractor 204 extracts the motion information of the object through the optical flow analyzer 214 and combines it with the adaptive background subtractor 216 to stably extract the area of the pedestrian. do. It also quickly accommodates background changes in outdoor environments where lighting conditions change frequently. The pedestrian tracking and feature extractor 204 quickly removes a pedestrian recognized as a background, and provides a function of stably extracting an area even in the case of a pedestrian temporarily not moving.

First, the optical flow analyzer 214 extracts motion information (ie, a motion vector for each pixel) of an object by using an image provided from the surveillance camera image inputter 208, and then adaptive adaptive median background subtractor 216. As input information.

In this case, the optical flow refers to a vector indicating a moving direction of a pixel represented by brightness by comparing a current image and a previous image in several consecutive images. The optical flow indicating the dynamic movement of the object determines the motion information of the object in the image that changes over time, and further enables stable separation of the background from the background.

Representative methods for calculating the optical flow are the Gradient-based Lucas-Kanade method and the Horn-Schunck method. Both of these methods propose a solution to solve the lens aperture problem.

The first method, the Lucas-Kanade method, is the most widely used method, which is fast, noise-resistant and simple, making it suitable for applications requiring real-time execution. Basically, the Lucas-Kanade method is based on the well-known motion constraint equation of Equation 1 below.

, 여기서 I는 (x,y,t)에서 명도(intensity)이며, 는 이미지 속도이다.

Where I is the intensity at (x, y, t) and is the image speed.

However, this limitation alone is difficult to estimate the movement position due to the aperture problem caused by the local limited interpretation of the optical flow. For this reason, Lucas and Kanade assume a constant optical flow v in the local window as a constant, and in that window minimize the weighted sum-of-squares of the error function E, which is defined as Suggest.

, 여기서 w는 가중함수이다.

Where w is the weighting function.

Therefore, the optical flow v can be obtained by the least square method as shown in Equation 3 below.

The second method, the Horn-Schunck method, is based on the optical flow constraint equation defined by Equation 4 below.

, 여기서 , 는 옵티컬 플로우 벡터를 이루는 구성 성분으로서 속도 벡터이고, 이다.

Where is the velocity vector as a component of the optical flow vector.

는 영상으로부터 측정되는 값이며, 이를 이용하여 u와 v를 계산하게 된다. 그런데 방정식은 하나이고 구해야 할 미지수는 두 개이므로, 다른 가정이 없는 한, 한 점에서 u, v의 값을 결정할 수 없다. 이 를 렌즈 구경 문제라고 부른다. Horn과 Schunck는 상기 렌즈 구경 문제를 해결하기 위해 변분법(variation of calculus)에 의해 오차함수(error function)를 최소화시킴으로써 옵티컬 플로우 필드(optical flow field)를 얻는 방법을 제안한다.In Equation 4

Is the value measured from the image, and u and v are calculated using this. However, since there is only one equation and two unknowns, we cannot determine the values of u and v at one point unless there are other assumptions. This is called a lens aperture problem. Horn and Schunck propose a method of obtaining an optical flow field by minimizing an error function by a variation of calculus to solve the lens aperture problem.

)와 옵티컬 플로우 구속 방정식 상의 오차(

)는 각각 하기 수학식 5와 같이 표현된다.That is, assuming that the optical flow field is flat, Equation 4 can calculate the solution by the variation method. To do this, a smoothness measure of the velocity field

) And the error in the optical flow constraint equation (

Are each expressed as in Equation 5 below.

를 표현하면,

이 된다. 그리고, 상기

를 최소화함으로써 u, v를 구하면 하기 수학식 6과 같이 표현된다.Using a weighting constant λ, the error function by combining the equation (5)

If you express

Becomes And,

When u and v are obtained by minimizing, it is expressed as in Equation 6 below.

, 여기서 이다.

, Is here.

The optical flow analyzer 214 may extract the motion information of the object by using any one of the two methods described above. That is, the optical flow analyzer 214 determines the static object as a stationary object when the size of the motion vector for each pixel is less than or equal to a predetermined threshold value and displays the value as 0, and displays the binary image represented as 255 only for pixels greater than or equal to the predetermined threshold value. Create

The adaptive background subtractor 216 performs background modeling by applying a median modeling technique without a separate preprocessing process for background learning. At this time, the motion binary image generated by the output of the optical flow analyzer 214 is read, and in the case of pixels without motion information, the background model update process is performed except for the background model update for pixels with motion information. The adaptive background subtractor 216 generates a difference image (ie, a binary image) between the median background model and the currently input frame image.

In the case of outdoor video, lighting is not stable and the background is also characterized by a lot of movement such as trees, waves, and fountains. In addition, it is impossible to secure a background frame without any dynamic objects such as pedestrians or vehicles for a certain period of time for background modeling. Therefore, there is a need for a method capable of adaptively updating background information and extracting dynamic object areas stably in response to changes in illumination over time. In other words, an efficient background subtraction technique is required together with the above-described optical flow technique.

The background deduction technique includes a Running Gaussian Average (RGA) method, a Gaussian Mixture Model (GMM) method, a GMM with adaptive number of Gaussians based on Gaussian of variation, and an approximation. Median filtering-based background subtraction (AMF).

Hereinafter, a preferred embodiment of the present invention will be described a method of modeling a background using an approximate median filtering-based background subtraction (AMF) technique among the background subtraction techniques. However, it will be apparent to those skilled in the art that the background may be implemented by a background subtraction technique other than the approximate median filtering-based background subtraction (AMF) technique.

The approximate median filtering-based background subtraction (AMF) technique utilizes the optical flow information as the base information for the background update, thereby adaptively adapting the pedestrian area included in the background as well as stable update of the background without motion. Can be removed from the background image.

That is, the basic background modeling technique uses an approximate median background modeling technique that is strong against noise. When the background update area is selected, motion history information including optical flow image information of a predetermined period is queried, and the background update is performed only for an area without the motion history information.

For example, FIG. 3 illustrates an original image (left) and a background average image (right) at one time point. Referring to FIG. 3, it can be seen that the background average image at any one point includes the pedestrian area as the background image.

4 illustrates an original image by time (left), a motion history image (middle), and a background subtraction result image (right). Referring to FIG. 4, it can be seen that a background subtraction result is generated by applying an optical flow image-based approximate median background modeling technique according to an embodiment of the present invention. That is, the area of the pedestrian that was initially included in the background image is updated with the original background information, so that the correct pedestrian area is generated as a result of background subtraction as time passes.

5 illustrates a background update procedure according to a preferred embodiment of the present invention. Referring to FIG. 5, in step 502, the optical flow analyzer extracts a motion history image based on the original image.

In operation 504, the adaptive background subtractor performs a background update based on the motion history image. That is, the adaptive background subtractor inquires the motion history information and performs background update only for the region without the motion history information. When the step 504 is completed, in step 506 the adaptive background subtractor outputs a background difference image based on the original image and the background update image.

By repeating the background update process of step 504, the adaptive background subtractor may adaptively update a background that changes with time to obtain a background difference image.

As described above, multiple pedestrian areas may be stably detected by using the above-described optical flow analyzer and the adaptive background subtractor. When the multi-pedestrian area is detected, an area of interest of a pedestrian (hereinafter referred to as ROI) is detected, and the movement of the pedestrian is tracked.

The pedestrian tracker 218 divides the grid-based area by inputting the background difference image generated from the adaptive background subtractor 216, calculates the foreground pixel ratio to the background pixel for each grid patch, and thus, calculates the foreground area. Extracts the region of interest for the object region.

6 illustrates a pedestrian ROI detection procedure according to a preferred embodiment of the present invention. Referring to FIG. 6, in step 602, the optical flow analyzer extracts a motion history image based on an original image.

In operation 604, the adaptive background subtractor performs a background update based on the motion history image. In operation 606, the adaptive background subtractor outputs a background subtraction image using the original image and the background update image. In operation 608, the adaptive background subtractor performs noise reduction on the background subtraction image and outputs the noise to the pedestrian tracker.

When the step 608 is completed, in step 610 the pedestrian tracker performs grid-based region segmentation with the input of the background difference image. In operation 612, the pedestrian tracker calculates a foreground pixel ratio to a background pixel for each grid patch to detect a region of interest for the foreground region, that is, the dynamic object region. In step 614, the pedestrian tracker updates the foreground ROI using the motion history image.

As described above, the multi-pedestrian area may be detected based on an image input in real time from the surveillance camera through the above-described process, and the ROI of the detected pedestrian may be detected.

Meanwhile, in the process of detecting the ROI of the pedestrian, although the dynamic object is removed from the background as the dynamic object included in the initial background screen, the ghost does not disappear immediately from the background image and is expressed as a dynamic object area. This is called the Ghost phenomenon. In order to eliminate this phenomenon, if the motion image area corresponding to each area tracked as the foreground area is inspected and the ratio of motion pixels is less than a predetermined threshold value, the foreground area is determined as a ghost area and the object unit information is excluded from the dynamic object candidate. Perform the matching process. For example, referring to FIG. 7, the process of extracting the final dynamic object region based on the background subtraction image and the motion image may be confirmed through an example screen.

When the foreground ROI detection of the pedestrian is completed, the pedestrian tracker 218 performs a function of tracking the same pedestrian by matching adjacent ROIs of the minimum distance between frames. That is, the pedestrian tracker 218 tracks areas of multiple pedestrians by performing track update, track generation, track merging, and track division based on the size and location of the ROI in the same frame.

Tracking multiple pedestrians based on silhouettes or areas is one of the most important technology elements in intelligent surveillance systems. One track is allocated for each extracted foreground blob, and a track is created and removed according to the movement of the object, thereby tracking a plurality of pedestrian areas. Even if an object is divided into two or more areas, if they move together for a certain period in consideration of the size and distance of the foreground blob, they are managed as one track. In addition, each track has color information of a moving object, so that the process of checking by color information can be performed when tracking between frames.

8 illustrates a multi-pedestrian zone detection and tracking procedure according to a preferred embodiment of the present invention.

Referring to FIG. 8, in steps 802 to 814, a foreground ROI of a pedestrian is detected. Since the operations 802 to 814 perform the same operations as the operations 602 to 614 described above, a detailed description of the operations will be omitted below.

When the foreground ROI of the pedestrian is detected through the steps 802 to 814, the pedestrian tracker performs steps 816 to 822 to track the multiple pedestrians.

That is, the pedestrian tracker allocates one track for each extracted foreground blob.

If the foreground blob with the assigned track is consistently matched within a range from the previous frame of the camera and there is continuous movement at a certain interval, the pedestrian tracker moves to step 816 to perform track update.

If the foreground blob is newly generated (ie, a new pedestrian appears), the pedestrian tracker moves to step 818 to create a new track.

On the other hand, when two or more pedestrians recognized within one camera move, some pedestrians sometimes disappear after being hidden by other pedestrians and appear again. In this case, when different pedestrians overlap, the pedestrian tracker moves to step 820 to perform track merging, and if the different pedestrians fall, moves to step 822 to perform track sharing.

After the steps 816 to 822 are completed through the above-described processes and the steps 816 to 822 are completed, the pedestrian tracker updates the foreground area by using the tracking result. Thereafter, the updated foreground region is fed back to step 804 for background updating.

As described above, multiple pedestrians may be stably detected based on an image input in real time from the surveillance camera through the above-described process, and the detected pedestrians may be tracked.

The representative frame extractor 220 selects different representative frames having a difference between the frames from the image output from the pedestrian tracker 218 above a predetermined threshold value, and includes the original image, the foreground image, pedestrian region information, photographing camera number, and photographing. The time and the like are extracted and stored in the image data storage unit 106.

The pedestrian characteristic extractor 222 extracts a pedestrian's characteristic using a pedestrian ROI provided from the pedestrian tracker 218 and still images and metadata provided from the still image / metadata inputter 210. That is, the pedestrian characteristic extractor 222 is divided into a head region, an upper body region, and a lower body region according to a human body ratio in the ROI of the pedestrian, and extracts color characteristics for each region. In addition, the pedestrian characteristic extractor 222 estimates the height of the pedestrian using triangulation and uses the pedestrian characteristic information.

For example, FIG. 9 illustrates a background subtraction image (left) and an original image (right) for dividing the same pedestrian ROI into detailed regions according to an exemplary embodiment of the present invention.

Referring to FIG. 9, when a separate ROI of a pedestrian is extracted, the head region, the upper body region, and the lower body region of the pedestrian are classified based on the height of the ROI. As a characteristic of the head area of the pedestrian, when the overall height of the ROI is h, the color of the part belonging to the center circle of the part corresponding to the upper h / 6 is analyzed. As a characteristic of the upper body region, the color of the foreground pixel is analyzed in the region of the lower h / 2 position in the upper h / 6 region. As a characteristic of the lower region, the color corresponding to the foreground pixel in the lower h / 2 region is analyzed.

The feature matching and result output unit 206 may include a pedestrian feature matcher 224 and a query result output unit 226.

The pedestrian characteristic matcher 224 extracts pedestrians having similar characteristics by comparing pedestrian characteristic information input through the query inputter 212 with pedestrian characteristic information extracted from the pedestrian characteristic extractor 222. That is, the pedestrian characteristic matcher 224 detects an image including pedestrian characteristic information matching the pedestrian characteristic information to be searched among the pedestrian characteristic information extracted from the pedestrian characteristic extractor 222.

The query result output unit 226 performs a function of providing the user interface unit 108 with an image detected as a result of matching the pedestrian characteristic information.

The user interface unit 108 outputs the detected image to an image search window and displays a pedestrian to be searched by the user in the image search window. That is, in the case of an online video, an annotation is displayed on the pedestrian, and in the case of an offline still image, the camera information, the shooting time, and the like are displayed together with the pedestrian annotation.

For example, FIG. 10 illustrates an image search screen according to a preferred embodiment of the present invention. FIG. 10A illustrates the configuration of the real-time video search screen, and FIG. 10B illustrates the configuration of the representative frame search screen.

Referring to FIG. 10, when a user inputs information of a pedestrian's key, upper color, lower color, and hat color, which are to be searched through a query input window on an image search screen, the user interface 108 identifies a corresponding pedestrian. The search is displayed on the real-time video search screen or the representative frame search screen.

As described above, through the operation of the surveillance camera image-based intelligent retrieval system described above, the user can efficiently search for pedestrians to be searched from images input in real time from a plurality of surveillance cameras installed in the surveillance target area in an online video image or an offline representative frame image. You can search by.

11 illustrates an intelligent image retrieval process using a surveillance camera according to a preferred embodiment of the present invention.

Referring to FIG. 11, in operation 1102, a plurality of surveillance cameras are installed in a surveillance target area and provide a captured image to a surveillance camera image input unit in the form of a data stream. The surveillance camera image input unit provides an image captured by the surveillance camera to a pedestrian tracking and feature extractor.

In step 1104, the pedestrian tracking and feature extractor extracts the motion information of the object through an optical flow analyzer and combines it with the adaptive background subtractor to detect the area of the pedestrian. In step 1106, the pedestrian tracking and feature extractor detects the detected ROI of the pedestrian and tracks the movement of the pedestrian.

When the step 1106 is completed, in step 1108, the pedestrian tracking and feature extractor extracts the pedestrian characteristics using the pedestrian ROI provided from the pedestrian tracker and the still image and metadata provided from the still image / metadata inputter.

In operation 1110, it is determined whether the characteristic information of the pedestrian to be searched through the user interface is input. If there is no search request by the user, the image search process ends.

On the other hand, if there is a search request by the user, go to step 1112. In operation 1112, the feature matching and result outputter compares the pedestrian characteristic information input through the query inputter with the pedestrian characteristic information extracted through the pedestrian tracking and feature extractor and extracts pedestrians having similar characteristics. That is, the pedestrian characteristic matcher 224 detects an image including pedestrian characteristic information matching the pedestrian characteristic information to be searched among the pedestrian characteristic information extracted from the pedestrian characteristic extractor 222.

When the step 1112 is completed, in step 1114, the feature matching and result output unit provides the detected image to the user interface unit. The user interface unit 108 outputs the detected image to an image search window and displays a pedestrian to be searched by the user in the image search window.

As described above, through the above-described surveillance camera image-based intelligent retrieval method, the user can efficiently search for pedestrians to be searched from the video inputted in real time from a plurality of surveillance cameras installed in the surveillance target area from the online video image or the offline representative frame image. can do.

While specific embodiments of the present invention have been described above, various modifications may be made without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims below, but also by those equivalent to the claims.

1 is a block diagram of a surveillance camera image-based intelligent search system according to an embodiment of the present invention;

2 is a block diagram of an image processing server according to an exemplary embodiment of the present invention;

3 is a diagram showing an original image (left) and a background average image (right) at a specific time point;

4 is a diagram showing an original image (left), a motion history image (middle), and a background subtraction result image (right) according to time;

5 is a flowchart of a background update procedure according to a preferred embodiment of the present invention;

6 is a flowchart of a pedestrian ROI detection procedure according to a preferred embodiment of the present invention;

FIG. 7 illustrates a procedure of extracting a final dynamic object region based on a background subtraction image and a motion image; FIG.

8 is a flowchart of a multi-pedestrian zone detection and tracking procedure according to a preferred embodiment of the present invention.

9 is a diagram illustrating a background subtraction image (left) and an original image (right) for dividing the same pedestrian ROI into detailed regions;

10 is a block diagram of an image search screen according to an embodiment of the present invention;

11 is a flowchart of an intelligent image search process using a surveillance camera according to a preferred embodiment of the present invention.

Claims (16)

A plurality of surveillance cameras for photographing a surveillance target area; An image processing server extracting pedestrian characteristic information based on the images photographed by the plurality of surveillance cameras, and detecting an image from which the pedestrian characteristic information matching the pedestrian characteristic information to be searched is extracted from the extracted pedestrian characteristic information; And And a user interface to input the pedestrian characteristic information to be searched or to display the detected image on an image search screen. The image processing server of claim 1, wherein the image processing server comprises: A pedestrian tracking and feature extractor for detecting a pedestrian based on the images captured by the plurality of surveillance cameras, tracking the detected movement of the pedestrian, and extracting the characteristic information of the tracked pedestrians; And And a feature matching and result output unit for outputting the extracted image of the pedestrian characteristic information matching the pedestrian characteristic information to be searched from the extracted pedestrian characteristic information to the user interface unit. The method of claim 2, wherein the pedestrian tracking and feature extractor, An optical flow analyzer for outputting motion history information from images captured by the plurality of surveillance cameras; An adaptive background subtractor for outputting a background difference image by combining the output motion history information with the images captured by the plurality of surveillance cameras; A pedestrian tracker which performs grid-based area segmentation based on the background difference image, calculates a foreground pixel ratio to a background pixel for each grid patch, detects a pedestrian interest area, and tracks the detected pedestrian ROI; And And a pedestrian feature extractor for extracting pedestrian feature information using the pedestrian ROI provided from the pedestrian tracker and the still image and metadata provided from the image data storage unit. The method of claim 3, wherein The adaptive background subtractor outputs a background difference image using an approximate median filtering-based background subtraction technique. The method of claim 2, wherein the property matching and result output unit, A pedestrian characteristic matcher which detects an image including pedestrian characteristic information matching the pedestrian characteristic information to be searched among the pedestrian characteristic information provided from the pedestrian tracking and characteristic extractor; And And an inquiry result output unit for outputting the detected image to the user interface unit. The method of claim 1, The characteristic information of the pedestrian is an intelligent image retrieval system, characterized in that at least one of the height of the pedestrian, hat color, the presence of glasses, the color of the clothes of the pedestrian. Receiving images captured by a plurality of surveillance cameras; Extracting characteristic information of a pedestrian based on the received image; Detecting an image from which the pedestrian characteristic information matching the pedestrian characteristic information input from the user is extracted from the extracted pedestrian characteristic information; And Intelligent image search method comprising the step of displaying the detected image on the image search screen. The process of claim 7, wherein the extracting of the characteristic information of the pedestrian is performed. Outputting motion history information from the received image; Combining the output motion history information with the received image to output a background difference image; Detecting a pedestrian interest area based on the output background difference image and tracking the detected pedestrian ROI; And And extracting the pedestrian characteristic information by using the tracked pedestrian ROI, the still image, and the metadata. The method of claim 8, wherein the detecting of the pedestrian ROI comprises: Grid-based region segmentation is performed based on the output background difference image, and the ratio of foreground pixels to grid pixels for each grid patch is calculated to detect the pedestrian ROI. The method of claim 7, wherein The characteristic information of the pedestrian is an intelligent image retrieval method, characterized in that at least one of the height of the pedestrian, the hat color, the presence or absence of glasses, the color of the pedestrian costume. In an intelligent video retrieval system having a plurality of surveillance cameras, Image processing for extracting the pedestrian characteristic information based on the images captured by the plurality of surveillance cameras, and detecting an image from which the pedestrian characteristic information matching the pedestrian characteristic information to be searched is extracted from the extracted pedestrian characteristic information. Intelligent video retrieval system comprising a device. The method of claim 11, And a user interface for inputting the pedestrian characteristic information to be searched or displaying the detected image on an image search screen. The apparatus of claim 11, wherein the image processing apparatus comprises: A pedestrian tracking and feature extractor for detecting a pedestrian based on the images captured by the plurality of surveillance cameras, tracking the detected movement of the pedestrian, and extracting the characteristic information of the tracked pedestrians; And And an attribute matching and result output unit for detecting an image from which the pedestrian characteristic information matching the pedestrian characteristic information to be searched is extracted from the extracted pedestrian characteristic information. The method of claim 13, wherein the pedestrian tracking and feature extractor, An optical flow analyzer for outputting motion history information from images captured by the plurality of surveillance cameras; An adaptive background subtractor for outputting a background difference image by combining the motion history information with the images captured by the plurality of surveillance cameras; A pedestrian tracker which performs grid-based area segmentation based on the background difference image, calculates a foreground pixel ratio to a background pixel for each grid patch, detects a pedestrian interest area, and tracks the detected pedestrian ROI; And And a pedestrian feature extractor for extracting pedestrian feature information using the pedestrian ROI provided from the pedestrian tracker and the still image and metadata provided from the image data storage unit. 15. The method of claim 14, The adaptive background subtractor outputs a background difference image using an approximate median filtering-based background subtraction technique. The method of claim 13, wherein the characteristic matching and result output unit comprises: A pedestrian characteristic matcher which detects an image including pedestrian characteristic information matching the pedestrian characteristic information to be searched among the pedestrian characteristic information provided from the pedestrian tracking and characteristic extractor; And And an inquiry result outputter for outputting the detected image.

Images