KR20070006390A - Tracking method and system for tracking with multiple point-of view - Google Patents

Abstract

A multiple visual field tracking method and system are provided to track the position of an object and simultaneously obtain the moving trace, admission time, the state of the object, a direction of visual field and height information about the object by arranging a plurality of cameras to track the object to minimize tracking error and information loss. Images are received from a plurality of image input devices(S210). The received images are analyzed to detect the motion of an object(S220). Detection area optimization is performed on the image including the object(S230). The image of the object is tracked for the image on which detection area optimization has been executed(S240). The images of the object, tracked by the image input devices, are projective-transformed to a single space(S250). Object information analysis is performed on the images projective-transformed to the single space to generate an information analysis result(S260).

Description

TRACKING METHOD AND SYSTEM FOR TRACKING WITH MULTIPLE POINT-OF VIEW}

1 is a flow chart of a conventional tracking system.

2 is a block diagram of a conventional tracking system.

3 is a flow chart of a multi-view tracking method according to the present invention.

4 is a block diagram of a multiple field of view tracking system in accordance with the present invention.

<Description of the symbols for the main parts of the drawings>

110: wide area monitoring device 120: camera

130: area monitoring device 210: video input device

220: image input and control unit 230: subject information analyzer

240: information storage unit 240: communication interface unit

The present invention relates to a multi-view tracking method and a tracking system. More specifically, it is difficult to discriminate when an object is caused by another object by tracking a subject using one camera in a conventional security system. By placing multiple cameras and tracking the subject, it is possible to not only track the position of the subject, but also the movement trajectory, time of entry, state of the subject (movement, stop, standing, sitting, etc.), field of view, height information, etc. The present invention relates to a multi-view tracking method and tracking system capable of simultaneously acquiring a plurality of information to minimize tracking errors and minimizing information loss.

The security system may be classified into a security system through image acquisition such as a camera and a sensor-based security system. Among them, sensor-based security systems (intrusion detection, infrared ray, fire detection) are failing to satisfy various needs of users. There is an increasing demand for a video-based security system and a combination of image and sensor.

Such a video security system has the advantage of monitoring the security area in real time and storing the image of the surveillance area and replaying it when necessary. In recent years, by adding an intelligent element to a video security system using a camera, it is evolving to perform various roles as well as simple surveillance. For example, application applications are being developed such as tracking a motion through such a video security system to store an image using motion information or to determine a location of a person who enters the image to control an intelligent home appliance.

 However, such a conventional video security system tracks the movement of a subject, such as a person, by using a single camera, so when the subject is moved and overlapped or covered by another object, it is difficult to perform image-based tracking of the subject. have.

Therefore, a technology for arranging a plurality of cameras and performing image tracking using the same in a video security system has been developed.

For example, Patent Application No. 10-1999-0043796 filed on October 11, 1999 and published on May 7, 2001 by Kim Hwa Yoon et al. Patent Registration No. 10-0364582 or Aman James A. filed April 28, 2000 and registered on November 30, 2002, entitled "Multiple Moving Object Tracking / Monitoring System." Patent Application No. 10-2001-7006357, filed May 19, 2001 and published October 27, 2001, discloses an image tracking system using a number of such cameras. It is starting.

1 is a flow chart of the tracking system disclosed in the patent application No. 10-1999-0043796.

The patent application No. 10-1999-0043796 is composed of a fixed camera and a mobile camera, the image input terminal, the mobile camera is configured to be equipped with a pan / tilt driver (Pan / Tilt Driver).

As shown, the stationary camera performs continuous monitoring on a certain area (S110), and the mobile camera performs circular monitoring on different monitoring areas.

At this time, by extracting the motion information from the input information of the fixed camera (S120), when a moving object moving at a speed higher than the reference value is identified, a tracking command is issued to the mobile camera, and the fixed camera stores the site information as a video (S130).

The mobile camera 102 receiving the command acquires the continuous video sequence of tracking areas (S140), predicts the moving direction of the moving object through analysis by the motion estimation technique from the input image sequence (S150), and the driver (Pan / Tilt). The driving amount of the driver is determined (S160), and the video is stored using the same (S170).

2 is a configuration diagram of the tracking system disclosed in Patent Registration No. 10-0364582.

As shown, the tracking system disclosed in Patent Registration No. 10-0364582 includes a wide area monitoring device 110 and a plurality of area monitoring devices 130a to 130n. The wide area monitoring apparatus 110 simultaneously extracts the movements of a plurality of moving objects existing within the monitoring range by using the fixed camera 120 and registers the information, and at least one of the plurality of registered moving objects to be tracked. Selects the at least one moving object, tracks the selected at least one moving object in the first surveillance region, and outputs a moving object tracking command when the selected at least one movable object leaves the first surveillance region.

The area monitoring devices 130a to 130n move in a pan / tilt to monitor independent second monitoring areas, and when a tracking command for at least one or more moving objects selected from the wide area monitoring device 110 is input, the plurality of area monitoring devices. 130a to 130n are sequentially driven to monitor while zooming and zooming until the selected moving object leaves the second surveillance region.

The wide area monitoring device 110 and the plurality of area monitoring devices 130a to 130n simultaneously extract movements of at least one selected moving object to effectively separate / merge the moving areas of each moving object to track / monitor in real time. Perform.

In addition, the tracking system disclosed in Patent Application No. 10-2001-7006357 relates to a system for tracking the movement of multiple objects in a setting area by using a combination of a tracking camera equipped with an XY axis tracking upper camera and a frequency selective filter. . Also applied is a perspective camera for tracking Z-axis and a tracking camera with a frequency selection filter.

In this case, the traced object is marked with frequency selective reflective materials such as patches, stickers and tapes, and the tracking system radiates the selective energy to the entire tracking area so that the reflective material can reflect.

On the other hand, since the tracking camera receives light of different wavelengths and receives only reflected energy, the local computer system receives an image from the tracking camera to track the positions of the marks, and sets the maximum speed and the maximum size of the preset tracked object. The tracking system extracts the image of the desired traced object excluding the background from the unfiltered image by using the information of, and the calculated movement and position information.

Although the conventional invention of tracking using multiple cameras generally uses multiple cameras, one camera is used to monitor a large area and the other camera uses the detected information to enlarge and capture. . In this case, however, accurate tracking is impossible because it is impossible to resolve overlapping or occlusion that occurs when tracking.

For example, the tracking system disclosed in Patent Application No. 10-1999-0043796 or Patent Registration No. 10-0364582 fixes a single camera that monitors a large area and uses the information received from the camera to capture a subject with a mobile camera. Although a method of enlarging and capturing is disclosed in this case, there is a disadvantage in that overlapping or covering of a subject cannot be solved as in the case of using a conventional camera.

In addition, the tracking system disclosed in Patent Application No. 10-2001-7006357 installed a camera looking down from the ceiling to resolve overlapping or obstruction, but the tracking system disclosed in Patent Application No. 10-2001-7006357 also includes There is a disadvantage in that it is restricted, and in particular, the subject is limited due to the use of a frequency selective reflective material, which makes it difficult to apply to a general image tracking system.

Therefore, the need for a tracking device that improves the disadvantage of difficult to discriminate when the subject is caused by other objects by tracking the subject using one camera in the existing security system is increasing.

An object of the present invention is to improve the disadvantage that it is difficult to determine when the subject is interference by another object by performing tracking of the subject using one camera in the existing security system by placing a plurality of cameras and tracking the subject In addition to tracking the location of the subject, it simultaneously acquires a large number of information such as the movement trajectory, entrance time, the state of the subject (moving, stationary, standing, sitting, etc.), view direction, height information, etc. to minimize tracking errors. To provide a multi-view tracking method and tracking system that can minimize the loss.

Another object of the present invention is to provide a computer-readable recording medium recording a program for implementing the multi-view tracking method.

In order to achieve the above technical problem, the present invention comprises the steps of receiving an image from a plurality of image input devices, detecting the movement of the subject by analyzing the plurality of input images, and the image comprising the detected subject Performing detection area optimization on the image, performing image tracking of the subject on the image on which the detection area optimization is performed, performing spatial matching on the image of the image tracked subject, The method provides a multi-view tracking method comprising performing subject information analysis on the spatially matched image to generate an information analysis result.

According to an aspect of the present invention, a plurality of image input apparatuses disposed in a tracking space for inputting an image and an image input from the plurality of image input apparatuses are analyzed to detect movement of a subject and to control the plurality of image input apparatuses. An image input and control unit for generating a control command, a subject information analyzing unit generating information of a subject using an image of a subject input from the image input and control unit, and a subject information analyzing unit There is provided a multi-view tracking system including an information storage unit for storing information and a communication interface unit for transmitting information of the subject generated by the subject information analyzer using a communication network.

The present invention also provides a function of receiving images from a plurality of image input devices, analyzing a plurality of input images to detect movement of a subject, and optimizing a detection region for an image including the detected subject. A function of performing image tracking of the subject on an image on which the detection area optimization is performed, a function of performing spatial matching on the image of the image tracked subject, and performing the spatial match on the image. A computer readable recording medium having recorded thereon a program for realizing a function of performing subject information analysis to generate an information analysis result.

Hereinafter, the multi-view tracking method and tracking system of the present invention will be described in more detail with reference to the drawings.

3 is a flowchart of a multi-view tracking method according to the present invention.

As shown, first, an image is received from a plurality of image input devices (S210). For example, a plurality of cameras are arranged in the same space and images are simultaneously input from multiple cameras. Each of the plurality of cameras is preferably configured such that the field of view (FOV) of each camera overlaps each other so that tracking is easy even when the image overlaps or is blocked.

Thereafter, the motion of the subject is detected by analyzing the input image (S220).

In order to detect the movement of the subject, a background image may be first formed, and thus the movement of the subject may be detected.

For example, when images are input from a plurality of cameras, image values are accumulated for each pixel position, and then an initial background image is formed using the average of the accumulated values. When the initial background image is compared with the difference between the input images, the movement of the subject can be detected.

In addition, the process may include periodically updating the background image.

Afterwards, detection area optimization is performed (S230).

Using the detected motion region as it is is inappropriate because it includes many error sources such as noise. Therefore, it is desirable to perform tracking by optimizing the detection area using a method such as binarization, morphology algorithm, size filtering, and center area checking.

Thereafter, image tracking is performed (S240).

That is, the detected motion regions are tracked by comparing which motion regions each movement region was in the previous frame. In this comparison method, each matching value is obtained using color information matching, distance matching, and region matching, and then traced to the most appropriate region.

After that, the space matching operation is performed (S250).

In other words, the subject tracked by each camera is matched with each other in one space to perform the final tracking. Projective transformation of each camera's image into one common space is performed to match the feature points to determine the end user's location.

Then, subject information analysis is performed (S260).

After the location of the subject is completed, the movement time of each subject is checked by checking the entrance time and the time within the detection space of the subject. For example, when the subject is a human, height height of the area detected by each camera and face color information are used to measure the height of the person and analyze the direction of view. Finally, the human condition is analyzed using the location and height information.

Thereafter, the analysis information is stored and transmitted (S270).

That is, information of the subject (position, movement trajectory, entrance time, human state (movement, stop, standing, sitting), direction of view, key height information, etc.) is stored and transmitted to the outside through the communication network.

4 is a block diagram of a multiple field of view tracking system in accordance with the present invention.

As shown, the multi-view tracking system according to the present invention includes a plurality of image input apparatuses 210a to 210n, an image input and control unit 220, a subject information analyzer 230, an information storage unit 240, Communication interface 250 is included.

The plurality of image input devices 210a to 210n are, for example, cameras having a pan / tilt function, and are arranged in the detection space to input an image. In this case, each of the plurality of image input apparatuses 210a to 210n may be configured such that the FOVs of the image input apparatuses overlap each other to facilitate tracking even when the images overlap or are blocked.

The image input and controller 220 analyzes images input from the plurality of image input apparatuses 210a to 210n to detect movement and generates control commands for controlling the plurality of image input apparatuses 210a to 210n.

For this purpose, although not shown, the image input and control unit 220 generates a background image by using images input from the plurality of image input apparatuses 210a to 210n, a background image and a plurality of subsequent images. A motion detector which detects the movement of the subject by comparing the image input from the input device, a detection region optimizer which performs the detection region optimization on the image including the subject, and an image of the subject on the image where the detection region optimization is performed. And an image processing apparatus controller for generating a control command for performing image tracking, and a spatial matching unit for performing a spatial matching operation on the image of the image tracked subject. The image processing apparatus controller is configured to perform image tracking by transmitting control commands such as pan / tilt and zoom to the plurality of image input apparatuses 210a to 210n. Each detailed description except for the image processing apparatus controller may refer to the multi-view tracking method according to the present invention described with reference to FIG. 3.

The subject information analyzer 230 extracts the subject information by using the image of the subject input from the image input and the controller 220.

The information storage unit 240 stores the information of the subject generated by the subject information analyzer 230.

The communication interface 250 is an interface for transmitting the subject information generated by the subject information analyzer 230 using a communication network.

In addition, the computer-readable recording medium recording a program for realizing the function according to the present invention may refer to the description of the multi-view tracking method and the tracking system according to the present invention described with reference to FIGS. 3 and 4. Detailed description will be omitted.

Such a multi-view tracking method and a tracking system according to the present invention can be applied to, for example, a security system or a DVR using a camera to grasp and store information (entry time, location, movement trajectory, etc.) of a user in real time.

Furthermore, it can be applied to intelligent home appliance control using user information in home automation system. In other words, intelligent home appliances can be controlled by acquiring user information according to the location or access time of the user and in connection with the home automation system. The linkage with the home automation system is meaningful in that it can analyze the state of a subject, that is, a person, and control the home appliance automatically, as compared to the home appliance control using a conventional sensor network or a mobile communication network. .

Although the present invention has been described in detail, this is for illustrative purposes only, and the protection scope of the present invention is not limited thereto, and the protection scope of the present invention is defined through the description of the claims.

As described above, according to the present invention, by tracking a subject using one camera in the existing security system, it is possible to arrange a plurality of cameras by improving the disadvantage that it is difficult to discriminate when the subject is caused by another object. By tracking the subject, not only tracking the position of the subject, but also simultaneously obtaining a large number of information such as movement trajectory, entrance time, the state of the subject (movement, stop, standing state, sitting state, etc.), view direction, height information, etc. Minimize information loss and minimize information loss.

Claims (30)

Receiving an image from a plurality of image input devices; Detecting a movement of a subject by analyzing the plurality of input images; Performing detection area optimization on an image including the detected subject; Performing image tracking of the subject with respect to the image on which the detection region optimization is performed; Performing a spatial matching operation on the image of the image tracked subject; Generating information analysis results by performing subject information analysis on the spatially matched image Multi-view tracking method comprising a. The method of claim 1, And storing the result of analyzing the information. The method of claim 1, And transmitting the result of analyzing the information through a communication network. The method of claim 1, The plurality of image input apparatuses, the field of view (FOV) of each image input apparatus is configured to overlap each other. The method of claim 1, Detecting the movement of the subject by analyzing the plurality of input images, Accumulating image values for respective pixel positions with respect to images input from the plurality of image input apparatuses, and forming a background image using the average of the values; Detecting a movement of a subject by comparing a difference between the background image and a later input image It comprises a multiple view tracking method. The method of claim 5, Detecting the movement of the subject by analyzing the plurality of input images, And periodically updating the background image by comparing the background image with a later input image. The method of claim 1, In the performing of the detection area optimization on the image including the detected subject, Optimizing the detection area using methods such as binarization, morphology algorithms, size filtering, or center area checking. The method of claim 1, Performing image tracking of the subject with respect to the image on which the detection area optimization is performed, And performing image tracking of the subject with respect to the selected region after obtaining each matching value using color information matching, distance matching, or region matching. The method of claim 1, Performing a spatial matching operation on the image of the image tracked subject, And matching the subjects tracked by each of the plurality of image input devices with each other in one space. The method of claim 9, Matching the subjects tracked by each of the plurality of image input devices with each other in one space, And matching the subjects together in one space using Projective Transformation. The method of claim 1, Generating an information analysis result by performing subject information analysis on the spatially matched image, And generating the information analysis result including the entry time and the movement trajectory of the subject. The method of claim 1, Generating an information analysis result by performing subject information analysis on the spatially matched image, Generating the information analysis result including the height of the person and the viewing direction based on the height of the analysis region and the face color information detected from the plurality of image input apparatuses when the subject is a human; Field of view tracking method. A plurality of image input devices disposed in the tracking space and inputting images; An image input and controller configured to analyze images input from the plurality of image input apparatuses, detect movement of a subject, and generate control commands for controlling the plurality of image input apparatuses; A subject information analyzer configured to generate subject information by using an image of the subject input by the image input and controller; An information storage unit for storing information of the subject generated by the subject information analyzer; And a communication interface configured to transmit the information of the subject generated by the subject information analyzer using a communication network. The method of claim 13, The plurality of image input devices, multiple view tracking system is configured so that the FOV of each image input device overlap each other. The method of claim 13, Wherein each of the plurality of image input devices has a pan / tilt function. The method of claim 13, The video input and control unit, A background image generator for generating a background image using images input from the plurality of image input apparatuses; A motion detector for detecting a movement of a subject by comparing the background image with images input from the plurality of image input apparatuses; A detection area optimizer configured to perform detection area optimization on an image including the subject; An image processing device controller configured to generate a control command for performing image tracking of the subject with respect to the image on which the detection region optimization is performed; Spatial matching unit for performing a spatial matching operation on the image of the image tracked subject It comprises a multiple view tracking system. The method of claim 13, The subject information analyzer, And generating information on the subject including a moving time of the subject and a movement trajectory for each position. The method of claim 13, The subject information analyzer, And generating the information of the subject including the height of the person and the viewing direction based on the height of the analysis region and the face color information detected by the plurality of image input apparatuses when the subject is a human. Receiving a video from a plurality of video input devices; A function of detecting a movement of a subject by analyzing the plurality of input images; A function of performing detection area optimization on an image including the detected subject; Performing image tracking of the subject with respect to the image on which the detection region optimization is performed; Performing a spatial matching operation on the image of the image tracked subject; A function of generating information analysis result by performing subject information analysis on the spatially matched image A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 19, And a function of storing the information analysis result. The method of claim 19, And a function of transmitting the result of the information analysis through a communication network. The method of claim 19, The plurality of video input devices, A computer-readable recording medium in which the FOVs of the respective image input devices are configured to overlap each other. The method of claim 19, The function of detecting the movement of the subject by analyzing the plurality of input images, Accumulating image values for respective pixel positions with respect to images inputted from the plurality of image input apparatuses, and forming a background image using the average of the values; A function of detecting a movement of a subject by comparing a difference between the background image and a later input image Computer-readable recording medium comprising a. The method of claim 23, wherein The function of detecting the movement of the subject by analyzing the plurality of input images, And periodically updating the background image by comparing the background image with a later input image. The method of claim 19, A function of performing detection area optimization on an image including the detected subject may include: A computer-readable recording medium comprising the function of optimizing a detection area using a method such as binarization, morphology algorithm, size filtering, or center area checking. The method of claim 19, The function of performing image tracking of the subject with respect to the image on which the detection region optimization is performed, And a function of performing image tracking of the subject with respect to a selected area after obtaining each matching value using color information matching, distance matching, or region matching. The method of claim 19, The function of performing a spatial matching operation on the image of the image tracked subject may include: And a function of matching the subjects tracked by each of the plurality of image input devices with each other in one space. The method of claim 27, A function of matching the subjects tracked by each of the plurality of image input devices with each other on one space is provided. And a function of matching the subjects together in one space using a projection transformation. The method of claim 19, The function of generating information analysis result by performing subject information analysis on the spatially matched image, And generating a result of the information analysis including the entry time and the movement trajectory of the subject. The method of claim 19, The function of generating information analysis result by performing subject information analysis on the spatially matched image, And generating a result of analyzing the information including the height of the person and the viewing direction based on the height of the analysis region and the face color information detected by the plurality of image input apparatuses when the subject is a human. Readable media.

Images