KR20150055269A - Survellance video division apparatus and method for comprising indexing function - Google Patents

Abstract

The present invention relates to a surveillance video segmentation device capable of segmenting and indexing a surveillance video without shot boundaries and a method thereof. Accordingly, the surveillance video segmentation device includes: a parsing unit; a segmentation unit; and an indexing unit. The parsing unit detects and updates the background of the surveillance video through modeling the background of the surveillance video recorded by a camera, detects an object in the surveillance video through comparing the background and a current frame, and extracts a feature value through extracting and coupling a color histogram for the object and the texture information of the object. The segmentation unit calculates the object similarity and feature similarity by comparing the object and the feature value of the previous frame and the object and the feature value of the current frame and segments the surveillance video based on the object similarity and feature similarity. The indexing unit indexes the frames of the segmented video and stores the index information in a storage unit.

Description

TECHNICAL FIELD [0001] The present invention relates to a surveillance image dividing apparatus and method including an index function,

The present invention relates to a surveillance image segmentation apparatus and method, and more particularly to a surveillance image segmentation apparatus and method capable of dividing a surveillance image shot continuously through a camera such as CCTV and having no shot boundary, will be.

Recently, camera-based image security systems such as CCTV have been installed and operated for the purpose of crime prevention and monitoring recording in public places. Generally, a CCTV-based image security system has two main purposes as follows. The first objective is to prevent and respond to real-time crime by monitoring real-time image information about the area of interest. Second, the primary objective is to perform primary investigation using video information stored when an accident or crime occurs Support.

It is generally known to investigate crime suspects and dragon vehicles by checking crime scenes, crime scenes and surrounding CCTV videos. However, there is a problem that it requires a lot of time and human resources input to check the recorded video with the naked eye and to find the clue of the event.

In order to solve such a problem, a method of detecting an event such as a motion in a surveillance image and indexing the search for only the detected part in a moving image search is used. The most important step for indexing a moving image is to divide a moving image into units in which a scene change occurs.

When a general video is divided based on a scene change, a method of detecting a boundary of a shot constituting a video and dividing the video into a scene change is generally used. However, CCTV-based surveillance video does not have a separate shot unit, and since the video is input continuously, it is difficult to apply general shot boundary detection method as it is.

As a method for solving such a problem, there is a technique of calculating the reliability of the detection result using the average value of the difference histogram, detecting the scene change in the supervised image, and using the normalized weight value. However, this method focuses on improving the speed and computation of existing scene detection methods, and does not propose an image segmentation method considering the nature of the surveillance image. It is difficult to define the scene change in the normal monitoring image and it is difficult to give meaning to the detected scene change.

Therefore, there is a need for an image segmentation method using a change of a moving object such as a person and a vehicle, which is a unique characteristic of a surveillance image continuously input without a specific shot boundary.

In this regard, Korean Patent Registration No. 1133924 entitled " Active Video Surveillance System and Method Using Behavior Pattern Database "

It is an object of the present invention to provide an image dividing apparatus and method considering an image background and moving objects for an image having no separate shot classification, such as an image taken by a camera such as CCTV.

According to an aspect of the present invention, there is provided a surveillance image segmentation apparatus for detecting and updating a background of a surveillance image through background modeling of a surveillance image photographed by a camera, A parser for extracting a feature value by extracting and combining color histogram and texture information for an object; A division unit for dividing the supervised image based on the object similarity and feature similarity by calculating the object similarity and the characteristic similarity by comparing the object and the characteristic value of the previous frame extracted from the parsing unit with the object and the characteristic value of the current frame, And an index unit for indexing the frames of the divided images and storing the index information in a storage unit.

According to an aspect of the present invention, there is provided a surveillance image segmentation method including: detecting and updating a background in a surveillance image through a background modeling of a surveillance image photographed by a camera by a parsing unit; Detecting an object in the supervised image through comparison of the background and the current frame by the parsing unit; Extracting feature values by extracting and combining color histogram and texture information for an object by a parsing unit; Calculating an object similarity and feature similarity by comparing the object and feature value of the previous frame extracted in the step of detecting the object and the feature value extracted in the extracting step with the object and the feature value of the current frame, Dividing the surveillance image based on the object similarity degree and feature similarity degree by the install part; And a step of indexing the frames of the divided image by the index unit and storing the index information in the storage unit.

According to the monitor image dividing apparatus and method of the present invention, it is possible to perform image segmentation based on a change of moving objects of interest, for example, on a monitor image taken by a camera such as CCTV, Can be easily detected.

Further, according to the supervisory image segmentation apparatus and method of the present invention, it is possible to perform semantic-based summarization and search for events by indexing divided video clips.

1 is a block diagram of a supervisory image segmentation apparatus according to an embodiment of the present invention.
2 is a diagram showing an example of a method of extracting features of an object from an index unit of the surveillance video apparatus of the present invention.
3 is a flowchart illustrating a method of dividing a supervised image according to an exemplary embodiment of the present invention.
4 is a flowchart illustrating a process of comparing feature values in a feature comparison module of a divided portion of the supervisory image segmentation apparatus according to the present invention.

The present invention will now be described in detail with reference to the accompanying drawings. Hereinafter, a repeated description, a known function that may obscure the gist of the present invention, and a detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

Hereinafter, a supervisory image segmentation apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 1 is a block diagram of a supervisory image segmentation apparatus 100 according to an embodiment of the present invention. The monitoring image segmentation apparatus 100 of the present invention may include a parsing unit 110, a segmenting unit 120, an index unit 130, and a storage unit 140. The description of each of these constitutions is made below.

The parsing unit 110 receives the surveillance image photographed from the camera 10 and analyzes the received surveillance image. Specifically, the parsing unit 110 detects and updates the background in the supervised image through background modeling of the surveillance image photographed from the camera 10, detects the object in the surveillance image by comparing the background and the current frame, Extracts feature values by extracting and combining color histogram and texture information for an object. The parsing unit 110 may include a background modeling module 111, an object detection module 112, and a feature extraction module 113, as shown in FIG. Also, the camera 10 may include a camera such as a CCTV.

The background modeling module 111 receives the surveillance image photographed from the camera 10. Thereafter, the background modeling module 111 detects and updates the background in the supervised image through background modeling. As mentioned above, the surveillance video is video stream data that has not undergone a separate editing process. These surveillance images are not composed of shots which can distinguish the boundary between the beginning and the end as mentioned above. That is, in order to divide an image into a supervised image and distinguish the background and the object, a process of first detecting the background in the surveillance image photographed from the camera 10 is required. To this end, the background modeling module 111 detects a continuously changing background and dynamically updates it to provide an always-up-to-date background.

The object detection module 112 detects an object using the difference image between the background image received from the background modeling module 111 and the current image frame. Through this function, the object detection module 112 can calculate detection information such as the number of objects for the current frame and the distribution of objects. The number of such objects and the distribution of the objects are used in the object analysis module 121 described below.

The feature extraction module 113 extracts the feature value of the currently input image frame to calculate the similarity of the image continuously input from the camera 10. [ Specifically, the feature extraction module 113 extracts a feature value of the object detected by the object detection module 112. Here, the feature value extraction for the object can be performed through the histogram and the texture information for the color space. An example of this is shown in FIG. That is, the feature extraction module 113 extracts the texture information 21 and the color histogram 22 from the detected objects as shown in FIG. 2, and combines them to define the feature values. Here, the color histogram and the texture histogram are not specifically defined, and in the case of the color histogram, histograms in various color models such as RGB (Red Green Blue) and HSV (Hue Saturation Value) can be used. For texture histograms, you can use one of the histograms to represent the texture, such as Local Binary Patterns (LBP), Histograms of Oriented Gradients (HoG), and Edge Histograms. The histogram extracted as the feature value is normalized so that the scale of the histogram may vary according to the size of the detected object, so that the histogram has a size of [0, 1]. The histogram 23 indicates histogram integration and normalization of the texture histogram 21 and the color histogram 22.

The partitioning unit 120 calculates object similarity and feature similarity by comparing the object and feature value detected in the previous frame with the object and feature value detected in the current frame, Function. The division unit 120 may include an object analysis module 121, a feature comparison module 122, and an image segmentation module 123.

The object analysis module 121 calculates the object similarity (Sim _OBJ ) by comparing the object detected in the previous frame with the object detected in the current frame. For this, the object analysis module 121 calculates the object similarity (Sim _OBJ ) by comparing the object information of the previous frame and the current frame detected by the object detection module 112, that is, the number and distribution of objects. The formula for this object similarity (Sim _OBJ ) is as follows.

In Equation 1, NO _i denotes the number of objects in the i-th frame, NO _{i - 1} is the number of objects in the i-1-th frame that is the previous frame. That is, exp (| NO _{i -1} - NO _i |) indicates that the difference between the number of objects in the current frame and the number of objects in the previous frame is used to calculate the degree of similarity. Dist (a, b) denotes distances between objects a and b. This distance function can be achieved through various distance functions such as, for example, Hamming distance, Manhattan distance and Euclidean distance. Here, k and l denote variables for distinguishing the i-th included object. That is, in Equation (1), O _{i, k} denotes the kth object in the i-th frame, and O _{i -1, l} denotes the l-th object in the i-1 th frame (i.e., the previous frame). That is, for each object existing in the current frame, the object similarity degree is defined as a value obtained by adding the exponential function of the difference in the number of previous objects to the sum of distances to all objects in the previous frame plus weight.

The feature comparison module 122 calculates the feature similarity (Sim _Feature ) between the feature value of the previous frame extracted by the feature extraction module 113 and the feature value of the current frame. Feature comparison module similarity function to calculate the feature similarity (Sim _Feature) in 122 may be implemented using any one of a number of functions such as cosine similarity degree of similarity, the histogram intersection. Using this similarity function, the feature similarity between the object detected in the previous frame and the feature values of the object detected in the current frame is calculated.

The image segmentation module 123 calculates the sum of feature similarities calculated by the feature comparison module 122 and the object similarity calculated by the object analysis module 121 so that the sum of the similarities is equal to or smaller than a specific threshold If the image is divided.

That is, in the present invention, a moving image is divided based on a change in a moving object (that is, a person and an automobile) that is an object of interest in a supervised image and a visual similarity between frames. This allows image segmentation to be performed, for example, when a frame change occurs, when the number of objects changes or when a change occurs in the distribution, that is, when a change occurs to the object of interest.

The index unit 130 indexes the frames of the divided images and stores the index information in the storage unit 140. That is, the index unit 130 receives the index of the frame of the divided image in the divided unit 120, and indexes the image to the actual database, that is, the storage unit 140. To this end, the index unit 130 includes a DB management module 131 and a motion picture management module 132. The DB management module 131 is responsible for storing and managing indexes of image clips to be indexed in the storage unit 140 using the division information. In addition, the moving image management module 132 performs synchronization between the index of the indexed image clip and the actually stored moving image.

Hereinafter, referring to FIG. 3, a supervisory image segmentation method according to an embodiment of the present invention will be described. 3 is a flowchart illustrating a method of dividing a supervised image according to an exemplary embodiment of the present invention. In the following description, items overlapping with those mentioned above with reference to Figs. 1 and 2 are omitted for clarity of description.

First, a step S310 of receiving a surveillance image from the camera is performed by the parsing unit.

Thereafter, a step S320 of detecting and updating the background through background modeling is performed by the parsing unit in the monitoring image received in step S310. As mentioned earlier, for image segmentation in surveillance video, a process of distinguishing between background and object is needed. For this, step S320 is performed.

Thereafter, in step S330, an object is detected using the difference between the background image received in step S320 and the current image frame by the parser (step S330). In addition, in step S330, detection information such as the number of objects for the current frame and the distribution of the object can be calculated simultaneously with the detection of the object.

Thereafter, in step S340, a feature value for the object detected in step S330 is extracted by the parsing unit. Specifically, extraction of the feature values performed in step S340 may be performed through the histogram and texture information on the color space. The description thereof has been given with reference to Figs. 1 and 2, and a detailed description thereof is omitted here. Also, the color histogram can use histograms in various color models such as RGB and HSV. In addition, texture histograms can use histograms to represent textures such as LBP, HoG, and edge histograms. It should be understood that the types of histograms herein are not limited to those mentioned and may be used in various ways that have been previously developed or will be developed in the future.

Thereafter, the similarity is calculated (S350) by comparing the object and the feature value detected in the previous frame with the object and feature value detected in the current frame by the partitioning unit. Specifically, in step S350, the object similarity is calculated by comparing the object information detected in the previous frame and the current frame, that is, the number and distribution of objects, and the feature similarity is calculated by comparing the feature values of the objects of the previous frame and the current frame. . Since the process of calculating these similarities has been mentioned above with reference to FIGS. 1 and 2, it is omitted here for the sake of clarity.

In step S360, the dividing unit divides the supervised image based on the degree of similarity calculated in step S350. That is, in step S360, the sum of the object similarity and feature similarity calculated in step S350 is used, and when the sum is less than or equal to a specific threshold value, the image is divided.

Step S370 is a step of indexing the frame of the divided image by the index unit. The information on the index made in step S370 may be stored in the storage unit in step S380.

Referring to FIG. 4, description will be made of the process of calculating feature similarity in the feature comparison module of the divided part of the supervisory image segmentation apparatus of the present invention. 4 is a flowchart illustrating a process of comparing feature values in a feature comparison module of a divided portion of the supervisory image segmentation apparatus according to the present invention. In the following description, the variable is assumed to be 1 to n.

First, a step S410 is performed to determine whether a variable k indicating the division of an object in the current frame exceeds the number (NO _i ) of objects included in the current frame. If it is determined in step S410 that the variable k exceeds the number of objects NO _i included in the current frame, the control is transferred to step S430. Otherwise, control is passed to step S420.

Then, at S430 step, initialize variables Max indicates the maximum value of the characteristic similarity and the variable l indicating the classification of objects in the i-1-th frame that is the previous frame is the number of objects contained in the preceding frame (NO _{i - 1} ) (S440) is performed. If it is determined in step S440 that the variable l exceeds the number of objects included in the previous frame (NO _{i -1} ), control is passed to step S450. Otherwise, control passes to step S480.

Then, a temporary similarity (Sim _F ) is calculated (S450) by using the feature value of the object for the current frame and the feature value of the object for the previous frame. The temporal similarity is a value calculated by calculating the similarity to the histogram of the kth object of the current frame and the lth object of the previous frame, and is a variable for finding the highest similarity value for identifying the same object. As described above, the similarity function Sim (a, b) used in the calculation of the temporal similarity Sim _F in step S450 can be implemented by using one of various similarity functions such as the cosine similarity and the histogram intersection. The variable O _{i , k} mentioned in step S450 represents the histogram of the i-th frame, that is, the k-th object in the current frame, and O _{i -1 and l} denote the histogram of the i-1 th frame, .

Thereafter, the temporary similarity (Sim _F ) is compared with the Max value to determine whether the temporary similarity (Sim _F ) is equal to or greater than the Max value (S460). As a result of the determination in step S460, if it is determined that the temporal similarity Sim _F is equal to or greater than the maximum value, control is passed to step S470, and the maximum value is set to the temporal similarity Sim _F calculated in step S450. Otherwise, control is passed to step S440.

Step S480 is a step of setting a feature similarity (Sim _Feature ). That is, the temporal similarity (Sim _F ) for all the objects calculated in step S450 is calculated by calculating the similarity of all the objects of the previous frame and the feature values for each object of the current frame, And the similarity value of all objects is defined as similarity degree of feature between two frames. When this process is completed, control is passed to step S410.

Step S420 is a step performed when the variable k representing the classification of the object in the current frame exceeds the number (NO _i ) of objects included in the current frame, and returning the calculated feature similarity (Sim _Feature ). After step S420, control is transferred to the end block.

The monitoring image segmentation apparatus and method according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Includes all types of hardware devices that are specially configured to store and execute magneto-optical media and program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The teachings of the principles of the present invention may be implemented as a combination of hardware and software. In addition, the software can be implemented as an application program that is actually implemented on the program storage unit. The application program can be uploaded to and executed by a machine that includes any suitable architecture. Advantageously, the machine may be implemented on a computer platform having hardware such as one or more central processing units (CPUs), a computer processor, a random access memory (RAM), and input / output (I / . In addition, the computer platform may include an operating system and microinstruction code. The various processes and functions described herein may be part of microcommand codes or a portion of an application program, or any combination thereof, and they may be executed by various processing devices including a CPU. In addition, various other peripheral devices such as additional data storage and printers may be connected to the computer platform.

It is to be understood that the actual connections between system components or process functional blocks may vary depending on how the principles of the present invention are programmed, as some of the constituent system components and methods illustrated in the accompanying drawings are preferably implemented in software It should be further understood. Given the teachings herein, those skilled in the relevant art (s) will be able to contemplate these and similar implementations or configurations of the principles of the invention.

100: monitoring image dividing device 110: parsing unit
120: division part 130: index part

Claims (1)

Detecting and updating a background in the surveillance image through background modeling of a surveillance image photographed from a camera, detecting an object in the surveillance image by comparing the background and a current frame, and generating a color histogram and texture information A parsing unit for extracting feature values by extracting and combining the extracted feature values;
The object similarities and feature similarities are calculated by comparing the object and feature values of the previous frame extracted by the parser with the object and feature values of the current frame, Installment; And
And an index unit for indexing frames of the divided images and storing the index information in a storage unit.

Images