KR100968024B1 - Method and system for tracing trajectory of moving objects using surveillance systems' network - Google Patents

Abstract

Disclosed are a method and a system for tracking an object movement path using a surveillance network. An object movement path tracking system using a surveillance network includes a surveillance network including a plurality of surveillance cameras for acquiring an image of an object; An object recognition unit extracting feature information of objects from an image acquired by the surveillance camera; And an index unit configured to generate an image index structure for distinguishing the objects based on the feature information of the objects, and a space-time index structure indicating a time at which an image of the object is obtained.

Spatiotemporal Indexes, Surveillance Cameras, Objects, Networks, Extraction.

Description

Method and system for tracking moving path of object using surveillance system network {METHOD AND SYSTEM FOR TRACING TRAJECTORY OF MOVING OBJECTS USING SURVEILLANCE SYSTEMS 'NETWORK}

The present invention relates to a method and a system for tracking a moving path of an object using a surveillance system network.

Increasing importance of security has led to an increase in the location of surveillance systems using surveillance cameras. However, the conventional surveillance system independently captured the area or part of each surveillance camera included in the surveillance system, and stored and analyzed images in the system.

Therefore, even if a plurality of surveillance cameras monitor a specific area, there is a lack of networking capability between the surveillance cameras. Therefore, the correlation between the images captured from each surveillance camera and the movement path of the same object depend on the manual operation of the administrator who manages the surveillance system. I'm doing it.

Therefore, by using a network connected to a plurality of surveillance cameras, information about the image captured by the surveillance cameras is shared to automatically search for a moving path of a specific person or object at a specific time in a large area or building where the surveillance system is installed. And there is a need for a way to track.

The present invention provides a method and system for tracking an object movement path using a surveillance network capable of identifying a common object in an image received from the surveillance camera by configuring a plurality of surveillance cameras in a network and tracking the movement path of the object. .

The present invention extracts a face region and a feature of an object from an image received from a surveillance camera, and stores the object in a database at a specific time and location by distinguishing the objects included in the image from each other by storing them in a database together with time-space data added with time. The present invention provides a method and system for tracking an object movement path using a monitoring network that can search and identify a movement trajectory of the object.

An object movement path tracking system using a surveillance network according to an embodiment of the present invention includes a surveillance network including a plurality of surveillance cameras for acquiring an image of an object; An object recognition unit extracting feature information of objects from an image acquired by the surveillance camera; And an index unit configured to generate an image index structure for distinguishing the objects based on the feature information of the objects, and a space-time index structure indicating a time at which an image of the object is obtained.

According to an aspect of the invention, the object recognition unit, a pre-processing unit for pre-processing the image obtained by the surveillance camera to divide by color; An area extracting unit extracting a face area on the image divided by the colors; And a feature extractor configured to extract feature information from the face area.

According to one aspect of the invention, the feature extraction unit can determine the uniqueness of the individual and whether the mask or mask, wearing a hat based on the color of the skin or geometrical relationship of the neck.

According to an aspect of the invention, the object recognition unit may further include a dimension reduction unit for reducing the dimensional information for the feature information by using a dimension reduction technique to the feature information.

According to an aspect of the present invention, the index unit may generate the space-time index structure using the time of the object as a parameter.

According to an aspect of the present invention, the index unit may generate the space-time index structure based on the TB-tree structure.

According to an aspect of the present disclosure, the image index structure may further include a search unit for searching for an image including the object and searching for a moving path of the object with the spatiotemporal index structure.

According to an aspect of the present invention, when the searcher receives a search target image from a user, the searcher may extract the feature information of the search target image by transmitting the search target image to the object recognition unit.

According to an aspect of the present invention, the search unit may search for and provide an image including an image index corresponding to the feature information of the search target image and an object corresponding thereto in the image index structure stored in the database.

Object tracking path tracking method using a surveillance network according to an embodiment of the present invention comprises the steps of obtaining an image of the object; Extracting feature information of the objects from an image of the object; And generating an image index structure for distinguishing the objects based on the feature information of the objects, and a spatiotemporal index structure indicating a time at which an image of the object is obtained.

According to the present invention, by configuring a plurality of surveillance cameras in a network, it is possible to identify a common object in the image received from the surveillance camera and to track the movement path of the object.

According to the present invention, by extracting a face region and a feature of an object from the image received from the surveillance camera and storing it in a database with time-space data added to the time to distinguish the objects included in the image from each other at a specific time and location The object can be searched and the movement trajectory of the object can also be identified.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

1 is an example illustrating an overview of an object movement path tracking system 100 using a monitoring network according to an embodiment of the present invention.

When an image is acquired from the surveillance network 101 including a plurality of surveillance cameras, the object recognition unit 102 extracts feature information of the objects from the image acquired by the surveillance camera, and the index unit 103 is configured to generate the image. An image index structure for distinguishing the objects and a spatiotemporal index structure representing the time at which the image of the object is obtained based on the feature information are generated and stored in the database 104. In this case, the searcher 105 may search for an image including the object in the image index structure and search a moving path of the object in the space-time index structure.

In this case, the surveillance network 101 configures a network by identifying a camera by assigning a serial number corresponding to a region to be monitored to the plurality of surveillance cameras, and transmits the image information obtained from the surveillance camera to the object recognition unit 102. Can transmit The configuration of the monitoring network 101 will be described in more detail with reference to FIG. 2.

In this case, the object recognizing unit 102 may correct an image acquired by the surveillance camera, distinguish an object included in the image, and extract a feature of the object. The configuration of the object recognition unit 102 will be described in more detail with reference to FIG. 3.

In this case, the image index structure generated by the index unit 103 may be configured by using an image to be searched similar to the object to be searched by the user by constructing an index by distinguishing images based on the objects distinguished by the object recognition unit 102. You can search for images to which an object belongs. In this case, the image index structure may be generated using a general tree structure.

In this case, the spatio-temporal index structure generated by the index unit 103 may construct an index structure using the time of the object as a parameter, so that the user may search for the movement path according to the time of the object. In this case, the spatiotemporal index structure may be generated based on a TB-tree structure optimized for spatiotemporal information, which is not general image information, but needs to be generated and stored. The spatiotemporal index structure will be described in more detail with reference to FIG. 4.

In this case, when the search unit 105 receives a search target image from a user, the search unit 105 transmits the search target image to the object recognition unit 102 to extract feature information of the search target image, and the image index stored in the database 104. In the structure, an image including an image index corresponding to the feature information of the search target image and an object corresponding thereto may be searched and provided to the user.

As such, the object movement path tracking system 100 using the checked monitoring network will be described in more detail with reference to FIGS. 2, 3, and 4.

2 is an example showing an overview of the monitoring network 101 according to an embodiment of the present invention.

As shown in FIG. 2, the surveillance network 101 bundles a plurality of surveillance cameras 211, 212, 213, and 214 into one network to acquire an image in a region 210, which is a specific area, and monitors a plurality of other surveillance. The cameras 221, 222, 223, and 224 may be bundled into one network to acquire an image in the second region 220, which is another region, and transmit the image to the object recognition unit 102.

At this time, the surveillance network 101 assigns a serial number 1 to the surveillance camera 211, assigns a serial number 2 to the surveillance camera 212, assigns a serial number 3 to the surveillance camera 213, Assign serial number 4 to 214, assign serial number 5 to surveillance camera 221, assign serial number 6 to surveillance camera 222, assign serial number 7 to surveillance camera 223, and monitor Serial number 8 may be assigned to camera 224.

In this case, the surveillance network 101 acquires images of the object 231 and the object 241 from the plurality of surveillance cameras 211, 212, 213, 214, 221, 222, 223, and 224, thereby providing the object 231 with the image. The route 232 and the object 241 passing through the first zone 210 and the second zone 220 to the arrival point 233 pass through the first zone 210 and the second zone 220 to the arrival point 243. It may be possible to track the moving path 242.

In this case, the object movement path tracking system 100 using the surveillance network indicates that there is an object having the characteristics of the object 231 in the image obtained from the surveillance cameras of serial numbers 1, 2, 4, 5, 6, 7, and 8. As a result, the object 231 may be searched to move to the path 232 through the area photographed by the surveillance camera of serial numbers 1, 2, 4, 5, 6, 7, and 8.

3 is an example illustrating an overview of the object recognition unit 102 according to an embodiment of the present invention.

The object recognizer 102 may include an image input unit 301, a preprocessor 302, an area extractor 303, a feature extractor 304, and a dimensional reducer 305.

In this case, the image input unit 301 sequentially receives the still image information between frames from the monitoring network 101 and waits for the next processing sequence in order, and when the preprocessing of the image previously transmitted from the preprocessing unit 302 ends, The image may be sequentially transmitted to the preprocessor 302. In this case, the still image information between the frames may be an image obtained from the surveillance camera.

In this case, the preprocessor 302 may pre-process the image transmitted from the image input unit 301 to correct the image so as to be easily processed, and then segment the colors by executing image segmentation.

At this time, the preprocessing unit 302 may improve the accuracy of segmentation and feature extraction by increasing the image quality of the image and fixing roughness, which is an important factor for readability, by removing noise and performing basic correction through a preprocessing process.

In this case, the preprocessor 302 may use a median filter having high edge retention and strong impulse noise in preprocessing in consideration of the characteristics of the surveillance image and the efficiency of segmentation.

In addition, the preprocessing unit 302 may use an α-trimmed mean filter for pretreatment, which may operate like a median filter depending on the value of α, and may perform stronger or weaker filtering, depending on the value of α. have.

In this case, the preprocessor 302 may segment the image by color to detect a part corresponding to a human skin color by executing image segmentation according to color.

In this case, the preprocessor 302 may use a method of segmenting an image using edge extraction.

In this case, the area extractor 303 may extract a face area from the image divided for each color.

In this case, the area extractor 303 extracts a component (typically an eye, a nose, a mouth, etc.) corresponding to a face from a region corresponding to a skin color in the image segmented by the preprocessor 302, thereby extracting a face region from the image. You can identify areas such as arms and legs that are similar in color and color.

In this case, the area extractor 303 may extract a face area by applying a plurality of color-based detection methods, geometric methods, and SVM methods.

In this case, the feature extractor 304 may extract a feature for each face region detected by the region extractor 303 in order to distinguish between faces included in the image.

At this time, the feature extraction unit 304 can find out the distinctiveness between the objects by identifying the uniqueness of the individual, as well as whether the mask, mask, hat, etc., as well as the uniqueness of each individual through the geometric relationship between the color of the skin and the neck. .

At this time, the feature extraction unit 304 extracts the overall features of the face using a principal component analysis (PCA) method, a linear discriminant analysis (LDA) method and a local feature analysis (LFA). Analysis) can extract the detailed features that are insufficient in the principal component analysis method.

In this case, the dimension reduction unit 305 may reduce the dimension information on the feature information by using a dimension reduction technique on the feature information extracted by the feature extractor 304.

The feature information extracted by the feature extractor 304 is usually expressed as n-dimensional data. When n becomes large, a large amount of space is required to store one still image, and the similarity between the image data is measured. High computational complexity is required.

In addition, in order to apply the index structure to the feature information data, it is necessary to apply the spatial index structure suitable for multi-dimensional. However, the spatial index structure has a problem that the efficiency decreases rapidly as the dimension increases.

Accordingly, the dimension reduction unit 305 reduces the feature information formed of the n-dimensional data to the d (0 <d << n) dimension and transmits the feature information to the index unit 103 by using a dimension reduction technique. The original feature data can be well represented.

In this case, the object recognizer 102 receives the search target image from the searcher 105 as a preprocessing unit 302, preprocesses the search target image, extracts feature information from the face region, and extracts the dimension of the extracted feature information. Can be reduced.

4 illustrates an example of a space-time index structure stored in the database 104 according to one embodiment of the present invention.

The spatiotemporal index structure is generated based on the TB-tree structure, and has a one-zone node corresponding to the one-zone 210 of the monitoring network 101 shown in FIG. 2 at a root node located at the top. There are two zone nodes 420 corresponding to 410 and two zones 220. In this case, the number of zone nodes may correspond to the number of zones included in the monitoring network 101.

In this case, leaf nodes corresponding to the image acquired by the surveillance camera included in the zone may be connected to the zone node.

For example, the object 231 is included in the image captured by the surveillance cameras of serial numbers 1, 2, 4, 5, 6, 7, 8, and the object 241 is serial numbers 1, 3, 4, 5, 7 When included in an image captured by the surveillance camera of FIG. 4, the object 231 is stored in the serial number 1 leaf node 411 connected to the first zone node 410, and the serial number 1 leaf node as shown in FIG. 4. The object 241 may be stored at 412.

At this time, the serial number 1 leaf node 411 is a serial number 2 leaf node 413, the serial number 4 leaf node 415, the serial number 5 leaf node 421, the serial number 6 leaf node (storage object 231) is stored ( 423, the serial number 7 leaf node 424 and the serial number 8 leaf node 426 may be linked to the link list, and the trajectory may be three-dimensionally displayed as shown on the right side of FIG. 4.

Also, at this time, the serial number 1 leaf node 412 is a serial number 3 leaf node 414, serial number 4 leaf node 415, serial number 5 leaf node 422, serial number 7 leaf in which the object 231 is stored. Nodes 425 may be linked to a linked list.

5 is a flowchart illustrating a method for registering object movement path information using a monitoring network according to an embodiment of the present invention.

In operation S501, the surveillance network 101 obtains an image of an object from the surveillance camera. In this case, the surveillance network 101 may transmit the acquired image to the image input unit 301 of the object recognition unit 102.

In step S502, the preprocessor 302 preprocesses the image of the object obtained in step S502 and divides the image by color.

In this case, the preprocessor 302 may pre-process the image transmitted from the image input unit 301 to correct the image so as to be easily processed, and then segment the colors by executing image segmentation.

In operation S503, the area extractor 303 extracts a face region from the image divided for each color in operation S502.

In this case, the area extractor 303 extracts the components corresponding to the face (typically eyes, nose, mouth, etc.) from the area corresponding to the skin color in the image segmented in step S502 and the face area in the image. You can identify areas such as arms or legs that are similar in color.

In operation S504, the feature extractor 304 extracts feature information from the face region extracted in operation S503.

At this time, the feature extraction unit 304 can find out the distinctiveness between the objects by identifying the uniqueness of the individual, as well as whether the mask, mask, hat, etc., as well as the uniqueness of each individual through the geometric relationship between the color of the skin and the neck. .

In operation S505, the dimension reduction unit 305 reduces the dimension information of the feature information by using a dimension reduction technique on the feature information extracted in operation S504.

In this case, the dimension reduction unit 305 may reduce the feature information formed of the n-dimensional data into the d (0 <d << n) dimension by using the dimension reduction technique and transmit it to the index unit 103.

In operation S506, the index unit 103 generates an image index structure for distinguishing the objects based on the feature information, and a space-time index structure indicating the time when the image of the object is obtained.

In this case, the spatiotemporal index structure may be generated based on a TB-tree structure.

In step S507, the index unit 103 stores the image index structure and the spatiotemporal index structure generated in step S506 in the database 104.

6 is a flowchart illustrating a method for searching for an object moving path using a monitoring network according to an embodiment of the present invention.

In operation S601, the search unit 105 receives an input of a search target image. In this case, the search target image may be an image that is similar to a predetermined number or more of the object to be searched by the user.

In step S602, the preprocessor 302 preprocesses the image of the object obtained in step S602 and divides the image by color.

In this case, the preprocessor 302 may preprocess the image transmitted from the searcher 105 to correct the image so as to easily process the image, and then segment the image by color by executing image segmentation.

In operation S603, the area extractor 303 extracts a face region from the image divided for each color in operation S602.

In this case, the area extractor 303 extracts the components corresponding to the face (typically the eyes, the nose, the mouth, etc.) from the areas corresponding to the skin color in the image segmented in step S602 and the face area in the image. You can identify areas such as arms or legs that are similar in color.

In step S604, the feature extractor 304 extracts feature information from the face region extracted in step S603.

At this time, the feature extraction unit 304 can find out the distinctiveness between the objects by identifying the uniqueness of the individual, as well as whether the mask, mask, hat, etc., as well as the uniqueness of each individual through the geometric relationship between the color of the skin and the neck. .

In step S605, the dimension reduction unit 305 reduces the dimension information for the feature information by using a dimension reduction technique on the feature information extracted in step S604.

In this case, the dimension reduction unit 305 may reduce the feature information formed of the n-dimensional data into the d (0 <d << n) dimension by using the dimension reduction technique and transmit it to the search unit 105.

In operation S606, the search unit 105 searches for an image index that matches the feature information of the search target image in the image index structure stored in the database 104.

In operation S607, the search unit 105 searches a space-time index corresponding to the image index found in operation S606 to search for a moving path of the object corresponding to the search target image.

In step S608, the searching unit 105 displays the image index found in step S606 and the moving path of the object found in step S607.

As described above, when the spatio-temporal index and the multimedia index are used in the network-based surveillance camera system according to an embodiment of the present invention, an efficient retrieval of similar images such as moving object tracking, other feature information such as masking or masking, etc. according to the user's needs is performed. This makes it possible to provide a more efficient and faster intelligent surveillance system for monitoring suspicious figures or clothing.

Embodiments according to the present invention can be implemented in the form of program instructions that can be executed by various computer means can be recorded on a computer readable medium. The computer readable medium may include program instructions, file data, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware systems specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware system described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by specific embodiments such as specific components and the like. For those skilled in the art, various modifications and variations are possible from these descriptions.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and all of the equivalents or equivalents of the claims as well as the claims to be described later belong to the scope of the present invention.

1 is an example illustrating an overview of an object movement path tracking system using a surveillance network according to an embodiment of the present invention.

2 is an example illustrating an overview of a monitoring network according to an embodiment of the present invention.

3 is an example illustrating an overview of an object recognition unit according to an embodiment of the present invention.

4 illustrates an example of a space-time index structure stored in a database according to an embodiment of the present invention.

5 is a flowchart illustrating a method for registering object movement path information using a monitoring network according to an embodiment of the present invention.

6 is a flowchart illustrating a method for searching for an object moving path using a monitoring network according to an embodiment of the present invention.

Claims (23)

A surveillance network composed of a plurality of surveillance cameras for acquiring an image of an object; The image obtained by the surveillance camera is preprocessed and divided by color, the face region is extracted from the image divided by the color, and the uniqueness of the object can be confirmed based on the geometrical relationship between the color of the skin and the neck in the face region. An object recognition unit extracting feature information; And An index unit for generating an image index structure for distinguishing the objects based on the feature information of the objects and a space-time index structure indicating a time at which the image of the object is obtained Including, The surveillance network is an object movement path tracking system using a surveillance network, characterized in that the plurality of surveillance cameras grouped by area. delete The method of claim 1, The object recognition unit preprocesses the image to remove and correct noise, and segmentation to segment the image by color. Object tracking path tracking system using a surveillance network characterized in that. delete delete The method of claim 1, The index unit generates the space-time index structure using the time of the object as a parameter. Object tracking path tracking system using a surveillance network characterized in that. delete The method of claim 1, A database storing the image index structure and the space-time index structure generated by the index unit Object tracking path tracking system using a surveillance network further comprising a. The method of claim 8, A searcher for searching for an image including the object by the image index structure and searching a moving path of the object by the space-time index structure Object tracking path tracking system using a surveillance network further comprising a. 10. The method of claim 9, The search unit extracts feature information of the search target image by transmitting the search target image to the object recognition unit when the search target image is input from a user. Object tracking path tracking system using a surveillance network characterized in that. The method of claim 10, The search unit searches for and provides an image including an image index corresponding to the feature information of the search target image and an object corresponding thereto in the image index structure stored in the database. Object tracking path tracking system using a surveillance network characterized in that. Acquiring an image of an object by each of the surveillance cameras included in the surveillance network in which the plurality of surveillance cameras are grouped by region; Preprocessing the image of the object and dividing the image by color; Extracting a face region from the image divided for each color; Extracting feature information from the face region based on the geometrical relationship between the color of the skin and the geometrical appearance of the individual; And Generating an image index structure for distinguishing the objects based on the feature information of the objects and a spatiotemporal index structure indicating a time at which the image of the object is obtained; Object tracking path tracking method using a surveillance network comprising a. delete The method of claim 12, The dividing may include preprocessing the image to remove and correct noise, and segmenting the image by color. Object tracking path tracking method using a surveillance network characterized in that. delete delete The method of claim 12, The generating of the index structure may include generating the spatiotemporal index structure using the time of the object as a parameter. Object tracking path tracking method using a surveillance network characterized in that. delete The method of claim 12, Storing the image index structure and the spatiotemporal index structure generated in the step of generating the index structure in a database. Object tracking path tracking method using a surveillance network further comprising a. The method of claim 19, Searching for an image including the object by the image index structure and searching a moving path of the object by the space-time index structure Object tracking path tracking method using a surveillance network further comprising a. The method of claim 20, The searching may include extracting feature information of the search target image when a search target image is input from a user. Object tracking path tracking method using a surveillance network characterized in that. The method of claim 21, The searching may include searching for and providing an image including an image index corresponding to the feature information of the search target image and an object corresponding thereto in the image index structure stored in the database. Object tracking path tracking method using a surveillance network characterized in that. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 12, 14, 17 and 19-22.

Images