KR101394242B1 - A method for monitoring a video and an apparatus using it - Google Patents

Abstract

A video surveillance method according to an embodiment of the present invention includes: receiving an image to be monitored; Obtaining a background of the received image;
Dividing the background of the acquired image into a plurality of background areas, and allocating predetermined semantic information to each of the divided background areas;
Extracting a moving object from the received image;
Determining a type of the extracted moving object;
Comparing the type of the determined moving object with semantic information corresponding to a background area in which the moving object is located to determine a risk level; And
And performing a monitoring process according to the determined risk.

Description

TECHNICAL FIELD [0001] The present invention relates to a video surveillance apparatus and a video surveillance method,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an image surveillance apparatus and an image surveillance method capable of efficiently performing image scene recognition in an intelligent video surveillance system

It is easy for humans to apply the ability to perceive in context to understanding what a person sees. It can be said that there is a lot of experiential learning about the location and plausible scenery where there is an object. For example, we will assume that if there is any moving object in the lake, it will be limited to fish or ship, and people will recognize that it is a dangerous situation. Due to the recent application of these concepts and the rapid development of cameras and computer hardware, systems using various algorithms of computer vision and pattern recognition parts have been used in combination with human computing interfaces, surveillance images, and medical imaging.

Pedestrian recognition technologies including face recognition and behavior recognition, which are improved in terms of speed and accuracy, have been actively researched and achieved a certain level of performance. In particular, pedestrian recognition technology is a very important part of the overall system performance evaluation in the surveillance system.

Especially, the intelligent video surveillance system which recently emerged recognizes the images displayed by the fixed camera manually or by recognizing objects in still images based on a large amount of learned data.

However, such a general video surveillance system is inconvenient and complicated to acquire background information, and has a problem that it can not process a lot of information. For example, in the conventional video surveillance system, it is inconvenient to specify a passive background boundary line for distinguishing the background, and it is difficult to classify the meaning for each background.

In addition, the general video surveillance system does not discriminate the background, but only analyzes the information on the area by designating the area of interest.

In addition, a typical video surveillance system may lack the accuracy to recognize danger situations. There is a way to create a general pattern by tracing a moving object separately, or a method of creating a pattern by distinguishing a normal situation from a dangerous situation with respect to the whole image. However, it is necessary to collect a huge amount of information according to the installation position of the camera, , The accuracy is very low, so the efficiency of the current technology is very low and may not be useful. In particular, in the case of a video surveillance method using a currently used PTZ (pan, tilt, zoom) camera, it is difficult to apply the method for the above reasons.

An object of the present invention is to provide a video surveillance apparatus and a video surveillance method of a video surveillance apparatus capable of reducing information throughput while improving the accuracy of recognizing a dangerous situation in order to solve the above problems.

According to another aspect of the present invention, there is provided a method for monitoring a video surveillance apparatus, the method comprising: receiving an image to be monitored; Obtaining a background of the received image; Dividing the background of the acquired image into a plurality of background areas, and allocating predetermined semantic information to each of the divided background areas; Extracting a moving object from the received image; Determining a type of the extracted moving object; Comparing the type of the determined moving object with semantic information corresponding to a background area in which the moving object is located to determine a risk level; And performing a monitoring process according to the determined risk.

According to another aspect of the present invention, there is provided a video surveillance apparatus including a reception unit for receiving a video to be monitored; A background acquiring unit acquiring a background of the received image; A semantic information setting unit for dividing the background of the obtained image into a plurality of background areas and allocating predetermined semantic information for each of the divided background areas; A moving object discrimination unit for extracting a moving object from the received image and determining a type of the extracted moving object; A risk judging unit for judging a risk by comparing semantic information corresponding to the type of the determined moving object and the background area in which the moving object is located; And a monitoring processor for performing a monitoring process according to the determined risk.

According to the embodiment of the present invention, the obtained background can be divided and a predetermined meaning can be assigned to the divided area, so that it is possible to process the background area efficiently without a passive background boundary designation process.

According to another embodiment of the present invention, even if the moving object overlaps with the feature, the type of the moving object can be accurately determined by comparing the semantic information between the moving object and the background area.

According to another embodiment of the present invention, it is possible to determine the risk according to the kind of the moving object and the semantic information of the background area, and it is possible to accurately and quickly determine the risk without complicated information processing.

In addition, according to another embodiment of the present invention, it is possible to automatically perform background segmentation and moving object detection according to the operation region of the PTZ camera, thereby enabling highly efficient monitoring processing and risk determination using the PTZ camera.

1 is a block diagram for explaining a video surveillance apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a video surveillance method according to an embodiment of the present invention.
3 is a flowchart illustrating a process of assigning semantic information to background information by a video surveillance apparatus according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating a process of determining a risk by a video surveillance apparatus according to an exemplary embodiment of the present invention.
5 is a flowchart illustrating a monitoring process performed by a video surveillance apparatus according to an exemplary embodiment of the present invention.
6 to 7 are views for explaining a background area in which semantic information is classified into contexts according to an embodiment of the present invention.
8 is a diagram illustrating background regions of a divided grid structure according to an embodiment of the present invention.
9 is a diagram for explaining a modeling method of a background region according to an embodiment of the present invention.
10 is a diagram for explaining a characteristic information determination process for determining a type of a moving object according to an embodiment of the present invention.
11 is a diagram illustrating an image in which a background is divided according to an embodiment of the present invention.
12 is a view showing an object image extracted from an image according to an embodiment of the present invention.
13 is a comparative diagram for comparing object recognition performance according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Thus, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. Furthermore, all of the conditional terms and embodiments listed herein are, in principle, only intended for the purpose of enabling understanding of the concepts of the present invention, and are not to be construed as limited to such specifically recited embodiments and conditions do.

It is also to be understood that the detailed description, as well as the principles, aspects and embodiments of the invention, as well as specific embodiments thereof, are intended to cover structural and functional equivalents thereof. It is also to be understood that such equivalents include all elements contemplated to perform the same function irrespective of the currently known equivalents as well as the equivalents to be developed in the future, i.e., the structure.

Thus, for example, it should be understood that the block diagrams herein represent conceptual views of exemplary circuits embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudo code, and the like are representative of various processes that may be substantially represented on a computer-readable medium and executed by a computer or processor, whether or not the computer or processor is explicitly shown .

The functions of the various elements shown in the figures, including the functional blocks depicted in the processor or similar concept, may be provided by use of dedicated hardware as well as hardware capable of executing software in connection with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

Also, the explicit use of terms such as processor, control, or similar concepts should not be interpreted exclusively as hardware capable of running software, and may be used without limitation as a digital signal processor (DSP) (ROM), random access memory (RAM), and non-volatile memory. Other hardware may also be included.

In the claims hereof, the elements represented as means for performing the functions described in the detailed description include all types of software including, for example, a combination of circuit elements performing the function or firmware / microcode etc. , And is coupled with appropriate circuitry to execute the software to perform the function. It is to be understood that the invention defined by the appended claims is not to be construed as encompassing any means capable of providing such functionality, as the functions provided by the various listed means are combined and combined with the manner in which the claims require .

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, in which: There will be. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram for explaining a video surveillance apparatus according to an embodiment of the present invention.

1, an image surveillance apparatus 100 according to an embodiment of the present invention includes an image receiving unit 110 for receiving a monitored image, a background obtaining unit 120 for obtaining a background for the received image, A moving object discrimination unit 140 for discriminating a moving object from the received image, a type of the moving object discriminated and a background region in which the moving object is located, A risk judging unit 150 for judging a risk based on the meaning of the risk, and a monitoring processing unit 160 for performing a monitoring process according to the determined risk.

The image receiving unit 110 can receive the image to be monitored. The surveillance target image may be an image captured and transmitted by a surveillance camera installed in the surveillance target area. Surveillance cameras can be installed in various places where security surveillance or risk judgment is required, such as apartment parking, outside boundary of facility, entrance in company office, etc., and can shoot images of the area. Here, the monitoring target image may be an image transmitted from the camera through the network interface in real time, or may be the digital image data already recorded.

In particular, the monitored image may be an image taken from a PTZ (pan, tilt, zoom) camera. The PTZ camera is a camera that includes a pan mo rotator that can rotate in the horizontal direction in addition to a zoom, focus, and iris motors that drive the lens, and a tilt motor that can rotate in the vertical direction. A variety of intelligent actions such as Group, Swing, Privacy Zone Masking, Auto Flip, Motion Tracking, Auto Parking, Schedule, . Accordingly, the video surveillance apparatus 100 can receive an image photographed from such a PTZ camera through the image receiving unit 110. [

The video surveillance apparatus 100 may further include a PTZ camera connected to the video receiving unit 110 as a part of the video surveillance apparatus 100. The video surveillance apparatus 100 may be connected to the PTZ camera through various interfaces to control the operation of the PTZ camera 100 It is possible.

The image receiving unit 110 may include a network interface unit (not shown) to receive the monitored image. The network interface unit may provide an interface for connecting the image receiving unit to a wired / wireless network including the Internet network. The network interface unit may include an Ethernet terminal or the like for connection to a wired network and may be a WLAN (Wi-Fi), a WiBro (Wireless Broadband), Wimax (World Interoperability for Microwave Access), and HSDPA (High Speed Downlink Packet Access) communication standards.

In addition, the image receiving unit 110 can transmit or receive data with another user or another electronic device through the network interface unit using the connected network or another network linked to the connected network. The transmitted or received data may include digital image data including a monitoring target image, and may further include various application data for video surveillance.

The background obtaining unit 120 obtains the background of the monitored image. The background obtaining unit 120 may obtain the background by analyzing the monitoring target image received by the image receiving unit 110 and acquiring the background data of the monitored area. In addition, for example, the background obtaining unit 120 may determine the amount of change of the characteristic information for each continuous image to be monitored, and acquire the background through a background determination process according to the amount of change. An embodiment of the detailed background acquisition method will be described later.

On the other hand, the meaning setting unit 130 divides the background obtained by the background obtaining unit 120 into a plurality of background regions, and assigns predetermined semantic information to each of the divided background regions. The meaning setting unit 130 may assign or set semantic information for each background area by extracting characteristic information for each background area and assigning semantic information corresponding to the characteristic information to each background area.

Therefore, the meaning setting unit 130 can generate and output the background data to which the semantic information is allocated, based on the background data obtained by the background obtaining unit 120. [ An embodiment of the detailed semantic information setting method will be described later along with the background acquisition method of the background acquisition unit 120 described above.

The meaning setting unit 130 may output the background data to which the semantic information is allocated to at least one of the moving object determining unit 140 and the risk determining unit 150.

Meanwhile, the moving object determining unit 140 extracts moving objects of the received image received from the image receiving unit 110, and determines the type of the extracted moving object. For example, the moving object discrimination unit 140 can receive the monitoring target image from the image receiving unit 110 and can analyze the presence of the moving object by analyzing some frames of the received monitoring target image.

The moving object determination unit 140 may determine the type of the identified moving object and output the determined type to the risk determination unit 150 according to a predetermined condition.

The moving object determining unit 140 can determine the type of the currently determined moving object based on the background data to which the semantic information output from the meaning setting unit 130 is allocated. For example, the moving object determination unit 140 may use not only the characteristic information of the moving object itself, but also the relationship information between the semantic information of the background where the moving object is located and the characteristic information of the moving object, The type of recognition can be determined. For this purpose, the moving object determination unit 140 may use a relation table between the stored semantic information and the moving object.

The moving object determination unit 140 may output the moving object information to the risk determination unit 150 according to the type of the determined moving object. The moving object information may include type information of the determined moving object and position information of the moving object.

The risk judging unit 150 judges the risk based on the moving object information outputted from the moving object judging unit 140 and the background data allocated with the semantic information outputted from the meaning setting unit 130, And transmits it to the monitoring processing unit 160.

The risk level determiner 150 may compare the type of the moving object and the semantic information of the background area in which the moving object is located in order to determine the risk. For this, the risk determination unit 150 may use a risk table including the semantic information of the background area and the risk information according to the type of the moving object located in the background area.

The monitoring processing unit 160 performs necessary monitoring processing according to the result of the risk determination of the risk determination unit 150. [

For example, the surveillance processing unit 160 may generate an alarm when the risk level is equal to or higher than a threshold value, and may perform an alarm using a means perceived by a user of the video surveillance apparatus 100 or a person concerned.

In addition, the monitoring processing unit 160 may output a real-time risk level according to a risk determination result. For this, the monitoring processor 160 may further include a display unit (not shown) or a voice output unit (not shown). The display unit may generate a driving signal by converting the hazardous video signal, the data signal, the OSD signal, or the video signal and the data signal received through the external device interface into the R, G, and B signals respectively processed by the monitoring processing unit 160 And may be a PDP, an LCD, an OLED, a flexible display, a 3D display, and the like. In addition, the sound output unit may receive a risk signal, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, which has been processed in the monitoring processing unit 160, and output the sound. have.

Then, the monitoring processing unit 160 can perform alarms according to the predetermined risk situations based on the result of the risk determination. The alarm according to the situation can be changed according to the type of the preset moving object and the background area in which the moving object is located. For example, the surveillance processor 160 can perform a drowning alarm when the type information of the moving object included in the risk determination result indicates a person and the background area information in which the moving object is located indicates a lake. In addition, for example, the surveillance processor 160 may perform an intrusion alert when the type of the moving object included in the risk determination result is a person or an automobile, and the background area information in which the moving object is located indicates a security facility. For example, the surveillance processing unit 160 may perform a burglar alarm when the type of the moving object included in the risk determination result is a person or an automobile, and the background area information in which the moving object is located indicates a valuable area.

As described above, the video surveillance apparatus 100 according to the embodiment of the present invention can acquire the background from the image received from the PTZ camera or the like through the background acquisition process, The accuracy of object type discrimination and risk judgment can be improved. In addition, since the monitoring process can be performed for each type of risk, it is possible to cope with various situations.

Hereinafter, a video surveillance method of the video surveillance apparatus 100 will be described with reference to flowcharts, video screens, and the like.

2 is a flowchart illustrating a video surveillance method of the video surveillance apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 2, the video surveillance apparatus 100 according to the embodiment of the present invention first receives an image (S101).

The video surveillance apparatus 100 can receive an image through the image receiving unit 110. [ The input image may include the above-described monitored image, and the monitored image may be an image captured from a surveillance camera located in the surveillance area. The surveillance camera may include a PTZ camera capable of pan rotation and tilt rotation operations as described above.

Then, the video surveillance apparatus 100 obtains the background based on the input image, and divides the background to set the per-area semantic information (S103).

The video surveillance apparatus 100 can acquire the background of the image input through the background obtaining unit 120 and can assign the divided information of the background region by the meaning setting unit 130. [

For example, the background obtaining unit 120 may obtain the background of the input image using a background modeling algorithm. Background modeling algorithms may include, for example, a Codebook method, a mixed Gaussian method, or a kernel method. It is preferable to use a codebook method which can easily adjust the boundary value according to the setting of the user and occupy less memory. This is a method of comparing the pixel values observed from the image with the new background to the previous pixel values, and the detailed algorithm will be described later.

Then, the video surveillance apparatus 100 recognizes the moving object based on the input image (S105).

As described above, the video surveillance apparatus 100 can recognize the moving object through the moving object determining unit 140 and determine the type of the moving object. There are various algorithm methods for extracting moving objects. For example, a feature extraction method using Histogram of Gradient Orientations (HOG) can be used.

In particular, the moving object discrimination unit 140 can perform accurate moving object discrimination using the background data to which the semantic information is allocated. For example, the moving object discrimination unit 140 can improve the accuracy of the moving object hidden by the obstacle by setting the relation condition between the background data and the moving object and determining the type of the moving object accordingly. For example, a moving object behind a tree in a school lawn can be more likely to be a person. Accordingly, the moving object discrimination unit 140 can set the human detection tolerance of the object person detection algorithm located in the background allocated to the school lawn area high, so that the moving object in the area can be easily and accurately detected .

Then, the video surveillance apparatus 100 determines the risk level based on the type and position of the recognized moving object (S107).

The video surveillance apparatus 100 can determine the risk level using the risk determination unit 150 as described above. The video surveillance apparatus 100 can determine and output the risk by comparing the type of the moving object and the semantic information of the background area in which the moving object is located. For example, if the semantic information of the background area in which the moving object is located represents a lake, and the moving object is a person, the risk determination unit 150 may output a risk level higher than a predetermined threshold value. When the semantic information of the background area indicates delivery, when the moving object is not a person but another moving object, the risk level may be higher than a predetermined threshold value. The risk value and the threshold value can be set in advance for each situation, and can be used for the surveillance processing operation of the video surveillance apparatus 100, which will be described later.

Then, the video surveillance apparatus 100 performs surveillance processing in accordance with the determined risk, and outputs the processing result (S109).

The video surveillance apparatus 100 can perform monitoring processing using the monitoring processing unit 160 described above and output the processing result. As described above, the monitoring process may be, for example, generating an alarm for each situation according to a predetermined condition or performing a user-perceptible alarm.

According to the output of the surveillance processing unit 160, the device operator of the video surveillance apparatus 100 can easily determine the risk level of the current surveillance target area without any special observation.

3 is a flowchart illustrating background acquisition and semantic information allocation according to an embodiment of the present invention.

Referring to FIG. 3, the image receiving unit 110 receives a surveillance target image photographed in a surveillance target area (S201).

Then, the background obtaining unit 120 divides the received monitoring target image and extracts characteristic information (S203).

For example, the background obtaining unit 120 may divide each frame into regions having a grid structure of a predetermined size based on some frames of the received monitoring object image, and extract characteristic information about the divided regions have. The characteristic information may include, for example, at least one of color information, edge information, and histogram information for each of the partitioned lattice structure regions.

Then, the background acquiring unit 120 determines a characteristic information variation amount for each of the divided grid areas (S205), and performs background learning according to the variation amounts (S207). The background learning may use a method of classifying the areas belonging to the boundary of the characteristic information among the plurality of divided areas as the background. The background acquiring unit 120 groups the similar plurality of areas, and improves the accuracy of the background acquisition by removing background groups that have not appeared for a certain period of time in the learning time.

Then, the meaning setting unit 130 assigns semantic information corresponding to the background obtained in the background obtaining unit 120 (209).

The meaning setting unit 130 may divide and allocate predetermined semantic information based on the property information of the plurality of grouped background areas. Here, the characteristic information for the plurality of grouped background areas may include at least one of color information and edge information.

The semantic information for each background area may be preset and may be specified by the user. The meaning setting unit 130 may assign semantic information such as a tree, a rock, a lawn, or water to an area having similar colors and edges as seen on the screen according to preset conditions. Such semantic information can be used in the risk determination and the process of determining the type of the moving object.

FIG. 4 is a flowchart illustrating a process of determining a risk by a video surveillance apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 4, first, the moving object determination unit 140 receives a monitoring target image received from the image receiving unit 110 (S301).

Then, the moving object determining unit 140 detects the moving object from the received monitoring target image (S303). The moving object discrimination unit 140 can check whether there is a moving object by checking the image change during a certain frame period of the received monitoring target image.

When the moving object is detected, the moving object determining unit 140 obtains the semantic information of the background area in which the moving object is located (S305).

The moving object discrimination unit 140 may acquire semantic information of the background area in which the moving object is located based on the output of the background data to which the semantic information of the semantic setting unit 130 is allocated.

Then, the moving object determining unit 140 determines the type of the moving object according to the background data, the obtained moving information, and the characteristics of the detected moving object (S307).

The moving object discrimination unit 140 detects a foreground object by removing a part classified as background data from the image and determines the type of the foreground object based on the detected characteristic information of the foreground object, for example, tilt information to be described later .

Also, as described above, the moving object determination unit 140 can improve the accuracy of determining the type of the foreground object by comparing the degree of relationship with the semantic information. For example, in a case where a background area in which a detected foreground object is located is an area (e.g., a tree, etc.) in which a person often travels but can be occluded, the probability of determining the foreground object as a person is set to be high, The accuracy can be improved.

Meanwhile, the moving object determination unit 140 determines whether a risk determination is required according to the type and location of the moving object (S309).

For example, if the moving object discrimination unit 140 determines that the type of the moving object is a general object or an animal rather than a person or an automobile, the video surveillance apparatus 100 re- It is possible to determine again based on the subsequent image data.

However, if it is determined that the risk judgment is necessary, the video surveillance apparatus 100 performs the risk determination and the monitoring process accordingly (S311). Through this determination process, the video surveillance apparatus 100 can skip the unnecessary risk determination process and improve the calculation efficiency. In addition, the video surveillance apparatus 100 can automatically perform storage of images requiring risk determination, and perform scene recognition.

To this end, the video surveillance apparatus 100 may further include a storage unit. The storage unit may include, for example, a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a RAM , ROM (EEPROM, etc.), and the like.

5 is a flowchart illustrating a monitoring process performed by a video surveillance apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 5, the video surveillance apparatus 100 stores the received video (S401). An image to be stored may include an image that requires monitoring processing. The image requiring monitoring processing may include, for example, image data determined to require the risk determination described above.

Then, the video surveillance apparatus 100 performs scene recognition on the stored image (S403), acquires scene information on the recognized scene (S405), and based on the scene information and the semantic information of the background data described above, (S407). The scene information may include information on the moving object described above (position information of the moving object or type information of the moving object). The risk determination may be performed by the risk determination unit 150. As described above, the predetermined risk may be determined according to the semantic information of the background area in which the moving object is located and the type information of the moving object.

Thereafter, the video surveillance apparatus 100 determines whether the determined risk level is equal to or greater than a threshold value (S409). The monitoring processing unit 160 can determine whether or not an alarm is generated based on the determined risk level.

Then, the video surveillance apparatus 100 performs an alarm when the determined risk level is equal to or greater than the threshold value (S411). The video surveillance apparatus 100 may output an alarm through the surveillance processing unit 160. As described above, the surveillance processing unit 160 may output alarms in various manners, and may output a warning according to the situation using the risk level, the location information of the moving object, and the type of the moving object.

If the determined degree of risk is less than the threshold value, the video surveillance apparatus 100 may perform the analysis on the image after acquiring the next image without performing the alarm (S413).

6 to 7 are views for explaining a grouped background area in which semantic information is classified into a context according to an embodiment of the present invention.

The video surveillance apparatus 100 according to the embodiment of the present invention can acquire a background area using a support vector machine (SVM) scheme. This is a category learning and annotation classification technique, in which the color and edge feature information are classified based on the background.

Also, the video surveillance apparatus 100 according to the embodiment of the present invention can use a background pixel learning based on a YCrCb color in order to model the background and distinguish it from the foreground.

The SVM method is a method of applying a semantic dividing technique to image data, and may mean a method of dividing a scene into regions and recognizing the scene. This can be applied by extracting features from the classification items to be recognized, generating a classifier using Support Vector Machine learning, and classifying the divided regions into respective categories using the generated classifier.

Such an SVM method may be a technique applied to recognize a pattern of an image or analyze image data. It can be applied to vision algorithms and can be used particularly useful for classifying images or their feature vectors. Image data for classification can be expressed in vector format, and a process of defining a feature vector for classification may be required. Accordingly, according to one embodiment, the video surveillance apparatus 100 can perform a linear SVM to classify the categories of the possible regions that can appear in the recognized scene, and classify them. Accordingly, the video surveillance apparatus 100 according to an exemplary embodiment can learn the maximum margin of the classification vectors, and can show good performance in classifying predetermined categories and data.

Such a predetermined category can be classified as shown in Figs. 6 and 7. Fig. Each of the divided areas can be classified into a pedestrian-positionable area or an area capable of covering a pedestrian depending on a characteristic vector, and can be classified into various ways such as a forest, a lawn, a land, a water, and a building. Accordingly, the video surveillance apparatus 100 can allocate semantic information on the background region according to the category classification.

8 is a diagram illustrating background regions of a divided grid structure according to an embodiment of the present invention.

As shown in Fig. 8, each of the areas may be divided into a plurality of background areas, and a plurality of background areas may be grouped according to their characteristics and classified into the categories described above.

Then, the video surveillance apparatus 100 can use the color and edge information of each area for segmentation and category classification. For color, the color vector of HSV space is used. When the distribution of learning data is observed with respect to various color spaces, it can be classified more clearly by color information, not largely depending on the brightness value. Accordingly, the video surveillance apparatus 100 can determine whether or not the degree of expression of the pattern is in the case of the image filtered by the edge detection filter. The video surveillance apparatus 100 can obtain a vector that is twice as large as the size N by obtaining an ortho histogram after filtering for each region of N x N size. In this way, the video surveillance apparatus 100 can classify regions having similar pattern characteristics by simply modeling the patterns.

9 is a diagram for explaining a modeling method of a background region according to an embodiment of the present invention.

As the background region modeling method, the code book method described above can be used. To this end, the video surveillance apparatus 100 may compare the pixel values observed in the background with the pixel values of the previous pixel values.

Then, the video surveillance apparatus 100 can increase the background boundary when the pixel is within the learning boundary, and generate a new codebook box when the pixel is outside the learning boundary. The video surveillance apparatus 100 can determine a background boundary through the learning process and classify the pixel values belonging to the boundary as a background in the monitoring process. The video surveillance apparatus 100 estimates a predetermined value of the permitted occurrence frequency and removes a background group that has not appeared for a certain period of time in the learning time.

As shown in FIG. 9, in the YCrCb space, the brightness can be approximated to the Y component, and the remaining components can express the hue and saturation. Although the brightness of the image changes frequently in the process of modeling the background, this may not be considered in the process of classification as background. Therefore, since the Cr and Cb components are independent of brightness, the video surveillance apparatus 100 can improve performance by modeling only using these components. As a similar case, in the case of the HSV space, the image monitoring apparatus 100 can use the YCrCb because the hue value is not well classified in the bright or dark image. The video surveillance apparatus 100 may also use a hybrid cone-cylinder model to reduce errors. The cone model solves the problem of less color difference in dark images. The cylinder model can prevent the boundaries from becoming too large for too bright images.

10 is a diagram for explaining a characteristic information determination process for determining a type of a moving object according to an embodiment of the present invention.

10A is an original image, FIG. 10B is a slope image, and FIG. 10C is a schematic diagram of a directional histogram in each cell.

In this manner, the video surveillance apparatus 100 can detect the pedestrian using the Histogram of Gradient Orientations (HOG) feature extraction for the determination of the moving object type. The video surveillance apparatus 100 may calculate the slope of the image, and may be performed through a Gaussian filter, for example, a discrete differential mask.

For example, the video surveillance apparatus 100 can group 66 pixels in an input image and define the unit as a cell. And you can group 33 cells together and define them as blocks. Then, a histogram for 9 directions can be created by dividing 20 by 20 from 0 to 180, and 9-dimensional vectorization can be performed.

According to the embodiment of the present invention, the video surveillance apparatus 100 uses the difference results and foregrounds of the background modeling using the codebook to normalize the block descriptors in the L2-norm method, By normalizing the block using the L2-Hys method, the pedestrian can be detected.

FIG. 11 is a diagram illustrating an image in which a background is divided according to an embodiment of the present invention. FIG. 12 is a view showing an object image extracted from an image according to an embodiment of the present invention, and FIG. FIG. 7 is a comparison diagram for comparing object recognition performance according to an example.

The embodiment of the present invention has been used to recognize pedestrians along with text background modeling results, and has been tested in an Intel core i5 3.3GHz computer environment. The input image size is 1280 * 720 pixels and the system is implemented based on C language. The input image was photographed so that the whole body of the pedestrian could come out from the outside of the building.

First, the meaning of scene information is set to the area of land, grass, water, etc., based on color characteristics only. However, there is a problem that the area with trees is recognized as grass. Thus, by using the embodiment of the present invention, the edge characteristic is additionally utilized, and the edge characteristics revealed in the wood area and the grass area are distinguished and classified.

Through this technique, we have classified trees, sediments, water and rocks, which are the same color characteristics seen on the screen. Then, assuming that there is an object in the vicinity of the area where the obstacle is present after foreground detection, the probability of pedestrians is set high. For example, when a person passes through a region such as a tree or a rock, it is estimated to be continuously tracked even if the phenomenon occurs.

Finally, we used the color and edge information to classify the land, grass, and water terrain, and the areas with occlusion such as trees and rocks were also classified by semantic information. In order to give information to the result of the pedestrian detected as the foreground after the background learning through the background scene segmentation information, the result annotated with the brightness value was provided and the experiment was performed.

On the other hand, for the foreground, if the pixel values in the Cr and Cb channels are included between the minimum boundary and the maximum boundary at the same time, the background is classified as a background. Pixels belonging to the box indicated by the learned boundary have a value of 0, And classified them into images. Since the shadows are made up of pixels with less color difference and lower brightness than the surrounding pixels, it is possible to obtain objects by removing them.

The performance of the pedestrian recognition result is shown in Table 1 below

HOG + SVM Background modeling +
HOG Features + SVM Operation time 5 sec 180 msec

  Since the background is not modeled in the input image, it takes about 5 seconds per frame to recognize the pedestrian without specifying the ROI. However, it takes about 180ms to process one frame in the same scene if the pedestrian is recognized after modeling the background to detect only the foreground region, designating the region of interest, and so on. In addition, since the pedestrian recognition module is applied only to the detected foreground region, false positives are not displayed in the background, and the false positives are also significantly reduced.

  In the pedestrian detection system using only the background modeling, it is not possible to recognize whether or not the pedestrian is detected in the foreground part. However, our proposed system uses the background modeling to specify the area of interest and the pedestrian recognition module operates within it. . The false negation rate performance according to the embodiment of the present invention is as follows.

HOG + SVM Context information +
HOG + SVM False negativity 15.5% 8%

The SVM classifier learned by the HOG feature is robust to the shape of the pedestrian being learned but is vulnerable to the change of the posture and the recognition rate may be greatly reduced even when the pedestrian is partially obscured by some other object. Therefore, in order to overcome this, context information of the scene can be combined with the pedestrian recognition module.

In the context information, the part where the foreground is detected is set in advance as an object which can hide the pedestrian for objects having a high probability of being a pedestrian. If the foreground is detected in the defined part, the probability of pedestrians is very high. By doing so, the false negative probability is reduced. FIG. 13A shows a general recognition method, and FIG. 13B shows that the left person is recognized by the monitoring method according to the embodiment of the present invention.

According to the embodiment of the present invention, the average recognition rate can be improved to 90% ~ 95%, and the recognition rate can be 70 ~ 100% depending on the situation. In particular, by learning and using semantic information about the background that can be covered by the PTZ camera, the semantic information of the already learned background can be utilized even if the camera rotates and the other is viewed at a different magnification, There is an effect that can be imported.

The video surveillance method according to the present invention may be implemented as a program to be executed by a computer and stored in a computer-readable recording medium. Examples of the computer readable recording medium include a ROM, a RAM, a CD- A magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet).

The computer readable recording medium may be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And, functional programs, codes and code segments for implementing the above method can be easily inferred by programmers of the technical field to which the present invention belongs.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

Claims (14)

A video surveillance method for a video surveillance apparatus,
Receiving an image to be monitored;
Extracting characteristic information from the received image and obtaining a background based on the extracted characteristic information;
Dividing a background of the obtained image into a plurality of background areas, and allocating preset semantic information according to the characteristic information for each of the divided background areas;
Extracting a moving object from the received image;
Obtaining semantic information of a background area in which the moving object is located, and determining whether the risk is required by determining the type of the extracted moving object using the semantic information of the background area;
Comparing the type of the determined moving object with semantic information corresponding to a background area in which the moving object is located to determine a risk level; And
Performing a monitoring process according to the determined risk; The method comprising the steps of: The method according to claim 1,
The acquiring of the background may comprise:
Extracting some frames of the received image;
Dividing each of the plurality of frames into a plurality of regions of a predetermined size;
Extracting the characteristic information including at least one of color information, edge information, and histogram information for the divided regions for each of the plurality of frames; And
Determining a change amount of the extracted characteristic information between the extracted partial frames;
Performing background learning according to a change amount of the characteristic information to determine a background of the received image; The method comprising the steps of: 3. The method of claim 2,
A plurality of regions of the partitioned predetermined size are partitioned into a lattice structure,
Wherein the characteristic information includes at least one of color information and edge histogram information for each of the partitioned lattice structures
Video surveillance method of video surveillance apparatus. The method according to claim 1,
Wherein the assigning of the semantic information comprises:
Generating a semantic information table according to image characteristics;
Dividing a background of the obtained image into a plurality of background areas;
Extracting characteristic information about the plurality of background areas and grouping the background areas having similar characteristic information; And
And assigning semantic information to each of the grouped plurality of background areas according to the semantic information table and the extracted characteristic information
Video surveillance method of video surveillance apparatus. The method according to claim 1,
Wherein the step of determining the type of the extracted moving object comprises:
Determining a background area in which the extracted moving object is located; And
And determining the type of the extracted moving object based on semantic information corresponding to the background area
Video surveillance method of video surveillance apparatus. The method according to claim 1,
The step of determining the risk includes:
Determining whether the risk level is determined according to the type and location of the moving object; And
And comparing the semantic information corresponding to the background area in which the moving object is located only when the risk determination is required, to determine the risk level
Video surveillance method of video surveillance apparatus. The method according to claim 1,
The step of determining the risk includes:
Determining a risk according to the kind of the moving object and the semantic information of the background area in which the moving object is located using a preset risk table
Video surveillance method of video surveillance apparatus. The method according to claim 1,
The step of performing the monitoring process includes:
Determining whether the risk is above a threshold; And
And generating an alert if the risk level is equal to or greater than the threshold value
Video surveillance method of video surveillance apparatus. The method according to claim 1,
The video to be monitored includes an image photographed from a PTZ (Pan Tilt Zoom) camera,
The step of acquiring the background of the image
Obtaining an enlarged background corresponding to an operation region of the PTZ camera from an image captured by the PTZ camera
Video surveillance method of video surveillance apparatus. A video surveillance apparatus comprising:
A receiving unit for receiving a video to be monitored;
A background obtaining unit for extracting characteristic information from the received image and obtaining a background based on the extracted characteristic information;
A semantic information setting unit for dividing the background of the obtained image into a plurality of background regions and allocating predetermined semantic information according to the characteristic information to each of the divided background regions;
A moving object discrimination unit for extracting a moving object from the received image, determining a type of the extracted moving object by using semantic information about a background area of the moving object location, and determining whether a risk determination is necessary;
A risk judging unit for judging a risk by comparing semantic information corresponding to the type of the determined moving object and the background area in which the moving object is located; And
And a monitoring processing unit for performing monitoring processing according to the determined risk. 11. The method of claim 10,
The background obtaining unit extracts some frames of the received image, and divides each of the plurality of frames into a plurality of regions of a predetermined size having a lattice structure, and extracts characteristic information about the divided regions for each of the plurality of frames Determines a background of the received image based on a variation amount of the extracted characteristic information between the extracted frames,
Wherein the characteristic information includes at least one of color information and edge histogram information for each of the partitioned lattice structures
Video monitoring device. 11. The method of claim 10,
The risk determining unit determines whether or not a risk is determined according to the type and location of the moving object, and compares the semantic information corresponding to the background area in which the moving object is located only when the risk determination is required,
Video monitoring device. 11. The method of claim 10,
Further comprising a PTZ (Pan Tilt Zoom) camera unit for capturing the image to be monitored,
The background obtaining unit obtains an enlarged background corresponding to an operation region of the PTZ camera from the image photographed by the PTZ camera
Video monitoring device. A computer-readable recording medium storing a program for causing a computer to execute the method according to any one of claims 1 to 9.

Images