KR100744668B1 - Multiple person surveillance system using networked cameras - Google Patents

Abstract

Conventional CCTV surveillance system has been limited to place and distance, there are many restrictions on the transfer of information between multiple cameras, most of them have been limited to monitoring monitoring. The present invention proposes a person tracking surveillance system combining CCTV surveillance system and person tracking technology. The proposed system tracks people through cameras in a wide area by connecting a number of fixed non-overlapping cameras to the network and passing the information tracked from each camera to the client connected to the server and other cameras. While being tracked, a person may have six states: emerge, move, overlap, wait, reappear, and leave. The proposed system was tested for various indoor videos, and the average tracking rate of 91.2% and the average state rate of 96% were obtained.

Surveillance system, camera, network, server, people, tracking

Description

Multi-person tracking system using network camera {MULTIPLE PERSON SURVEILLANCE SYSTEM USING NETWORKED CAMERAS}

1 is a schematic configuration diagram of a multi-person tracking system using a network camera according to an embodiment of the present invention.

2 is a flow diagram of information transfer between a server and a client of FIG.

3 is a protocol used for conveying information in FIG. 2;

4 is a diagram illustrating an internal configuration of the network camera of FIG. 1.

5A to 5C are diagrams for explaining a hierarchical person model employed in the multiple person tracking system of FIG.

6 is a view showing a process of changing the state information of the tracking target employed in a multi-person tracking system using a network camera according to an embodiment of the present invention.

7A and 7B are diagrams illustrating an experimental environment and a mobile FOV-Line of a multi-person tracking system using a network camera according to an embodiment of the present invention.

8 is a graph showing the average ROC for human tracking in three cameras used in the experiments of FIGS. 7A and 7B.

Explanation of symbols on the main parts of the drawings

10: client

12, 14, 16: network camera

20: server

The present invention relates to a surveillance system using a network camera, and more particularly to a multi-person tracking system that can track a large number of people in a large area using a network camera.

Surveillance systems that detect the occurrence of a particular action or event are the most important of all research-based research. The reason is that as society develops, the importance of the safety of individuals and facilities in public places as well as in private spaces is highly recognized. In particular, as the modern society becomes more information, unmanned, automated, and computerized, warnings about the safety and security of the property and the safety of its own workplace and workplaces continue to emerge. As a result, efforts are being made to protect and manage the property and safety of themselves and the workplace, and the importance and scope of security is expanding to major facilities, government offices, schools, businesses, and homes. Necessity is demanded.

Surveillance system is suitable for video surveillance system using CCTV cameras or PC cameras, network surveillance system that can be easily monitored from mobile phones, PDAs, PCs, etc. by transmitting images to computers quickly and in real time using network cameras. Infrared surveillance system, etc. In particular, video surveillance systems using CCTVs or PC cameras, unlike other surveillance systems as a countermeasure after intrusion or destruction, can actively identify intruders, detect intrusions or offenses in advance, and record them. It is the most effective means of monitoring and providing important information.

Video surveillance systems can use one or multiple cameras. Using only one camera is inexpensive, but it is not easy to simultaneously monitor several hazardous areas. As a result, video surveillance systems use two or more surveillance cameras to broaden their limited field of view, allowing them to monitor risks scattered across multiple areas and protect their targets. Surveillance systems that track people using multiple cameras can be divided into overlapping and non-overlapping arrangements, depending on whether each camera shares a range of tracking areas or not. . The overlapping field of view of the camera is still limited in its visual range, so a non-overlapping arrangement, which allows for a wider field of view, can be tracked in a wider range of directions, paths or patterns of movement. . Omar Javed et. al 'the IEEE International Conference Computer Vision, pp. In the article "Tracking in Multiple Cameras wit Disjoint Views," published in 952-957, 2003, the subjects were tracked using 10 minutes of repetitive movement of the object to track the person between three nonoverlapping batch cameras. We calculated by learning the time, distance and direction of movement to and from the camera. Dimitrios Makris et. In his article "Bridging the Gaps between Cameras," al, published in the IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2004, Washington DC, USA, June 2004, six cameras are networked to connect people's mouths outdoors. The link structure of the entrance and exit was automatically formed. In order to calculate the probability of which camera the tracking target is in when the target is not visible, the entrance and exit statistics between the six cameras are set to calculate the entry and exit statistics of each person. Obtained.

However, since the image acquired by the camera is constantly changed by the passage of time, light and lighting, the information of the tracking target is easily distorted or lost. In addition, when using multiple network cameras, the amount of noise and light coming into each camera is different so that there is a significant difference in the amount of distortion, so it is easy to fail tracking. As such, there is a limitation that a conventional surveillance system using a network camera cannot be properly used for tracking and monitoring a person.

Accordingly, the present invention is to provide a multi-person tracking system that can form a network between a plurality of non-overlapping cameras, define the information transfer form between the cameras, and track the person multiple by synchronizing and sharing the prescribed information.

The main object of the present invention is a multi-person tracking that can be monitored by a plurality of cameras in order to sequentially monitor the movement of a large number of people in a large area, such as banks, libraries, airports, major government offices, outside the building, etc. To provide a system.

According to a preferred aspect of the present invention in order to achieve the above object, it is connected to each other via a network, extracting feature information of the person model to be tracked from the acquired image, and compares the extracted feature information with the pre-stored person list feature information A plurality of cameras to sequentially track the person on the individual surveillance area; And a server receiving and storing feature information from each of the plurality of cameras, providing the stored person list feature information to each of the plurality of cameras, and synchronizing the plurality of cameras.

Preferably, the feature information classifies the human body of the tracking object into a head, upper body and lower body, and includes hierarchical information including at least one of a location, color, a moving direction, a center point, and a boundary region constituting the human body. Human model information.

The hierarchical person model information includes information on the foreground image separated from the light-corrected input image, and the information on the foreground image is obtained by calculating a color similarity between each pixel and an adjacent pixel to determine the color of the color. Contains blob information that is a set.

The controller of the camera transmits the new blob as new human model information to the server when the minimum distance between the new blob indicated in the t-th frame and the person model obtained in the previous frame is less than a predetermined threshold, and the server The person model information stored in the person list is updated by statistical information (mean and standard deviation) of recalculating the center and uncertainty of the new blob.

The camera and the server have status information of appearance, movement, overlapping, waiting, reappearing, and exiting with respect to the human model.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following examples are provided to fully understand the present invention for those skilled in the art.

1 is a schematic configuration diagram of a multi-person tracking system using a network camera according to an embodiment of the present invention. 1 shows a process of transferring information and tracking a person between a server and a network camera.

Referring to FIG. 1, characteristic information of a person recognized by one camera is transmitted to another camera on a network through the server 20. The server 20 receives and processes information of a camera coming from a client of each of the cameras 12, 14, and 16. In addition, the server 20 confirms the synchronization between each camera so that the information matched to the plurality of cameras (12, 14, 16) connected to the tracked person (P1, P2) information.

For example, person 1 P1 extracted from the first camera 12 and modeled as a person model is stored in the person list 22 of the server 20. After that, when the person 1 exits the first camera 12 and reenters the second camera 14, the server 20 checks whether the recognized person's information is registered in the person list 22. If the recognized person is registered in the person list 22, it is determined that the same person (P1) and continues to track. If the recognized person is not registered in the person list 22, the second camera 14 determines that the newly recognized person is a new person (P2 or P3) and the person list 22 of the server 20. Register at

In this way, each camera can continue to check and track whether a person entering the camera is a registered person, a new person coming into the surveillance frame's video frame, or a person leaving the video frame coming back. have.

As such, the present invention proposes a method of transmitting and sharing tracking information of a person over a large area through a network connection between a plurality of non-overlapping cameras 12, 14, 16 and a server computer 20. To this end, the server 20 forms a network to transmit and recognize information between several cameras 12, 14, and 16 connected to the server in real time, and each client has information and movement of the tracking object obtained from the connected camera. Recognizes and passes the characteristic information of the recognized tracking target to the server 20. The information of the tracking object recognized by one camera is transmitted to another camera connected to the server 20, and the camera which has received the information can track the previously tracked person by using the received information.

On the other hand, it is very important to model the feature information of the recognized tracking target in order to deliver the tracking information and receive. Therefore, in the present invention, information to be tracked is transmitted using a hierarchical human model composed of three human body parts (head, upper body, and lower body) in order to transmit information of tracking objects and to track a plurality of people. Tracking people begins by initializing the person model when the person being tracked first appears on the camera. The hierarchical human model has information such as the recognized human position, color, direction of movement, center point, and boundary areas that make up the human body. The information of the person model generated in this way is registered in the person list 22 which stores the information of the recognized person of the server 20 for the information transfer between the cameras 12, 14, 16 and the server 20. Since the information of the person model stored in the person list 22 is changed by the movement of the person every frame, the person model is updated every frame to minimize the loss of the information change of the tracking object so that accurate tracking can be performed. The hierarchical person model will be described in detail below.

FIG. 2 is a flowchart of information transfer between a server and a client of FIG. 1.

2, when the server 20 is driven, the server 20 creates a server socket and waits for a connection of the client 10. Each client 10 sends a connection request to the server for connection with the server 20. The server 20 assigns a camera number for synchronization between corresponding cameras in the order in which each client 10 is connected, generates a connection object 20a for distributing and processing sockets and streams of the client 10, and the client Permit entry to (10). In addition, the server 20 synchronizes the data stream by creating a thread for each connected client 10 in order to always check and control the state of each client 10. By using threads, multiple clients can connect to one server independently and reduce CPU usage. Threads are stored in ThreadLists and used to exchange information with connected clients. When the server 20 transmits a protocol for allowing the client 10 to transmit the information to the client 10, the client 10 reads media data for tracking a person and starts image processing. If the client 10 is unexpectedly released, the server 20 detects a change state of the client 10 and sends a notification message indicating that the connection is released. The server 10 stops the thread and each client 10 communicates with the server 20. Communication is connected through each connection object 20a, and the generated connections receive information of the connected clients 10 and store them in the connection list. That is, each client 10 transmits a message to be transmitted using the message protocol to the server 20 using the connection object 20a, and the server 20 transmits the request of the client 10 through the protocol. The information about the person tracked by each of the cameras 12, 14, and 16 is updated every frame, and the updated information is continuously transmitted to the server 20 through the connection object 20a.

When the connection object 20a sends a message received from the client 10 to the server 20, the server 20 sends a message and a protocol received from the connection object 20a to a thread of the server 20, and the server 20 Will use the created ThreadList to send data to all clients stored in the ThreadList. In the transmission method, a thread passes a message sent from the server 20 to the connection object 20a, and the connection object 20a delivers the message to the client 10. In this way, the server 20 and the client 10 may transmit a message along with the message protocol or check which client receives the message by using the connected thread and the connection object 20a. The protocol used for the transfer of information between the server 20 and the client 10 is summarized in a table in FIG. 3.

As described above, the network-based person tracking system proposed in the present invention efficiently transfers information of a tracking target recognized from a plurality of cameras, and uses JTP (Java Media Framework) -based RTP (network media) for network design. Real-time Transfer Protocol technology is used. JMF is a Java-based multiprogramming technology developed by Sun Corporation. It is a multimedia control API developed for capturing, storing, transmitting, and streaming video and audio. JMF is suitable for the development of multimedia system such as video conferencing and video medical system because it efficiently processes the multimedia stream data in the network environment and guarantees platform independence. Currently, JMF supports video technology such as MPEG, AVI, MOV and H. It also provides video conferencing codec standards such as .261 and H.263 and audio codecs such as G.721 and G.723.

RTP, which is supported to perform real-time media transmission or video stream reference in JMF on the Internet or intranet, is an API that supports transmission of video data over a network.

Since RTP is designed for the transmission of media streams, it is possible to solve some problems such as TCP / IP based HTTP, processing overhead of FTP, network transmission delay, lack of multimedia functions, etc. Suitable for transmitting multimedia data. In addition, RTP can be used for both single (uni) cast and multi (multi) cast because various media types can coexist with each other at any position in the stream and use an identifier to distinguish each data. This RTP function is used to distinguish transmitted data, determine packet order, and synchronize multiple media.

In this system, voice data transmission is not considered for faster media stream processing between the server 20 and the client 10.

In addition, the present system uses the JMF image transmission function to obtain image data from each camera 12 (14, 16) connected to the client 10. In order to acquire and process image data sequentially, synchronization must be performed between a plurality of cameras, so that camera numbers are assigned in the order of cameras connected to the server. For example, to create an RTP session for media stream transmission, the SyncVideo object reads the camera address and creates a MediaLocator object. Clients take the connected Camera's DataSource class from the SyncVideo object and pass the DataFormat class's DataFormat class information from the DataSource class to the server along with the camera address, port number, IP, and device information. The server checks whether the information in the DataFormat class is justified. If the information is not in the DataFormat class, the server blocks the client's access and sends an error message. If the information is legitimate, the server grants the client access. In addition, the server distributes sockets and streams of each client by creating connection objects in the order in which the clients are connected to control the overhead incurred when many clients are connected at once. When all the clients are connected, the server synchronizes each client using the same TimeBase object and receives and processes the information in the order from the client. Clients connected to the server update the feature information of the person tracked by the camera connected to it every frame and continuously transmit the updated information to other clients through the server.

4 is an internal configuration diagram of one network camera of FIG. 1.

Referring to FIG. 4, the imaging device 31 is implemented as a light receiving device such as a CCD or a CMOS and converts a light source of a subject transmitted through a lens into an electrical signal. The noise removing unit 32 removes noise of the electrical signal converted through the imaging device 31. The digital signal processing unit (DSP) 33 processes the signal from which the noise is removed by the noise removing unit 32 into image data. The control unit 37 calculates the output signal of the digital signal processing unit 33 and controls the system using the information of the memory 38 that stores the information of the entire system. The decoder and selector 34 inputs an image signal from the digital signal processor 33 and an external image signal from the camera 12 under the control of the controller 37, and selects and outputs a desired image signal. At this time, the control unit 37 recognizes the information and the movement of the tracking target, and extracts the recognized tracking feature information by hierarchical person modeling. The controller 37 compares the hierarchical person model extracted with the person list information received from the server, determines whether the extracted person model is a previously registered tracking object or a new tracking object, and then determines the server. To pass on. In order to efficiently transmit information, the video signal compressor 35 compresses the video signal output from the decoder and the selector 34 under the control of the controller 37. The transmission / reception unit 36 transmits a control signal from the outside, for example, a control signal of a server, to the camera 12 or the control unit 37 of the camera through a general telephone line or a communication line (not shown), and the image signal compression unit. The video signal compressed by 35 is transmitted to the outside, for example, a server via a normal telephone line or a communication line. The power supply 39 converts a voltage input through an adapter (not shown) into voltages of various levels used in the camera 12 and supplies them.

5A through 5C are diagrams for describing a hierarchical person model employed in the multi-person tracking system of FIG. 1. In this system, the blob is used by hierarchical human model designed by matching the three body parts using location information and color information. 5A to 5C illustrate an input image (a), a foreground image (b), and a result image (c) for hierarchical person modeling, respectively.

First, since the image acquired from the camera is constantly changed by the passage of time, light and lighting, the information of the tracking target is easily distorted or lost. In addition, when multiple cameras are used, the amount of noise and light coming into each camera is different so that the distortion is severe. Since it is very difficult to track people in different lighting environments, the adaptive lighting modeling method is used to make the environment uniform by mapping each camera's information into one uniform space.

Adaptive lighting modeling is a method of modeling predictable lighting using one original image and removing the predicted lighting from the image. Since the illumination of the current image may predict the position of the illumination in the current image from the position of the illumination generated in the previous frame, the illumination and the shadow to appear in the next frame may be predicted in the current frame, and the predicted illumination may be removed. As the initial stage of adaptive lighting modeling, the image that was least affected by light was defined as the original image. Since the current image can be predicted from the previous frame, the illumination existing in the previous frame can be predicted in the position where the existing image can exist. Therefore, from the second frame, the current image is corrected by the value of the illumination calculated in the previous frame.

The standard deviation and mean value of the background were calculated to separate the background image from the light-calibrated image using the original image. Each frame is obtained by using a Mahalanobis distance, which is a second discrimination function, for the difference between the background and the current image for a predetermined time t in each Y, Cb, Cr color space of the YCbCr space. Each time a new frame is entered, the background and foreground are separated and the separated background is updated using the background model (mean and standard deviation).

5A to 5C, the foreground image (b) separated from the light-corrected input image (a) for hierarchical human model and model-based tracking is a connected element algorithm for calculating brightness similarity between each pixel and adjacent pixels. To separate into a blob, a set of colors separated by multiple colors. Each blob contains information such as color, size, center point, location, and boundaries that make up the body. Since a person can be defined as a subset of blobs corresponding to human parts, the blobs in the frame must be assigned to the corresponding human body to track a large number of people. The separated blobs were divided into three human body parts, head, torso and leg, using location and color information according to their relative position in the human model (P1), as shown in the resulting image (c). Each blob is assigned to one part of the three body parts.

As such, this system uses a hierarchical person model including body color information to track a person. To track a person using a designed person model, the information of the person model is stored to track a number of people. On the other hand, because all human motion is relatively small from frame to frame, large changes in color-based models cause tracking to fail easily. Therefore, in order to solve the sensitivity of the tracking of the color model described above, the system updates the human model by comparing the blobs of the frame with the reference model. The information of the human model is represented by statistical information (mean and standard deviation) calculated up to a predetermined time t-1. If the minimum distance between the new blob and the human model shown in the t frame is less than the predetermined threshold, the person is almost accurately tracked by adding the blobs to the human model and recalculating their center and uncertainty to update the human model.

FIG. 6 is a diagram illustrating a process of changing state information of a tracking target employed in a multi-person tracking system using a network camera according to an exemplary embodiment of the present invention.

Referring to FIG. 6, the person tracking system of the present invention tracks the movement of a person tracked using a plurality of cameras to the end through information transfer of a tracking target between the camera and the server. Therefore, the human model has the following six levels of status information from the moment it enters the camera until it disappears completely. The status information of the tracking target is Entering, Moving, Occluding, Hiding, Re-entering, and Exiting.

Entering: When a new blob is created and a new human model is created on the existing person list from the generated blob.

Moving: When the generated human model is consistently matched within a certain range from the previous frame of the camera, and the center point of the human model has continuous movement at regular intervals.

Occluding: Some people's models disappear for a while and then reappear while two or more people's models recognized within a camera are moving.

Hiding: The tracked subject is in the hide-zone between the cameras for a period of time, out of camera range.

Re-entering: Re-appearance of tracked objects in standby.

Exiting: A person model has been removed from the camera for more than a certain time but has been removed from the person list.

7A and 7B are diagrams illustrating an experimental environment and a mobile FOV-Line of a multi-person tracking system using a network camera according to an exemplary embodiment of the present invention.

The human tracking system of the present invention was implemented using JAVA (JMF) on a Pentium-IV 1.8GHz CPU and window2000 XP with a 512 MB memory specification. Experiments were conducted with six scenarios using images (200 × 180) from three connected UNIMO CCN-541 security cameras, with frames ranging from 9 to 15 frames per second. The proposed human tracking system is capable of transmitting image data information and mutual information of each camera by using fixed non-overlapping cameras and computer network technology.

The three non-overlapping batch cameras have a hide-zone where the cameras do not overlap due to the nature of the layout structure where the views of each camera do not overlap. FIG. 7A illustrates a non-overlap zone (h1, h2, h3, h4) existing between a field of view (FOV) and each camera (Camera1, Camera2, Camera3) in an experimental environment. It shows FOV-Line in which the tracking subject (Person1 (P1), Person2 (P2), Person3 (P3)) moves in the experiment space.

Referring to FIG. 7B, the three tracking targets P1, P2, and P3 each follow the FOV-Line as shown in states (1), (2), (3), (4), (5), and (6). Follow the scenario of entering, moving, occluding, leaving, re-entering, and exiting the camera sequentially. The tracked objects may appear in each of the different cameras individually, move within the camera range, and some people may temporarily disappear by overlapping when multiple people enter the camera.

In addition, if a person tracked out of one camera is out of range, it can re-appear or exit completely from another connected camera or the previous camera. During the scenario, the tracking target will always follow one of the six levels of status information in FIG. 7B. For each camera, the person entering the camera is a new person, is a person registered in the existing person list, or previously exited from the video frame. Keep track of whether a person is coming back or if it has been completely removed.

For example, person 1 (Person1) first appears in the first camera Camera1 at frame 19 (creating a new person (Entering)), and person 1 who deviates from the first camera at frame 72 is connected to the second camera (Camera2). It reappears (Re-entering) and shows the result of the movement of Person 2 and Person 3 on the third camera Camera 3 (Moving). In the case of 88 frames, the person 2 who has escaped from the third camera reappears in the second camera and overlaps with the person 1 (Occluding). In frame 116, person 1 who has exited the second camera reappears in the third camera and person 2 shows the result of exiting from the second camera (Exiting).

FIG. 8 is a graph showing an average receiver operating characteristic (ROC) result for human tracking in three cameras used in the experiments of FIGS. 7A and 7B.

In the system of the present invention, the tracking accuracy for the tracked person was measured using the ROC graph. Tracking accuracy counted the number of models tracked by people in each tracking step and calculated the percentage of the total number of people actually displayed.

Referring to FIG. 8, it can be seen that the ups and downs of tracking accuracy are severe in an average 1 to 40 frame interval. This means that in the initial human model design process, the human tracking rate dropped momentarily because the model of the incoming person was not organized. However, the declined curve immediately rises to show a stable graph, indicating that the human model tracked by the three cameras is stable. The tracking accuracy shown in the ROC graph showed an average tracking rate of 91.2% for an average of 481 frames from when a person first enters and is recognized by the camera.

  Status information measured by the system Exit 18 Reappear 2 66 Waiting 278 5954 clash 47 movement 3565 12 appear 16 78 31 appear movement clash Waiting Reappear Exit Real state information

Table 1 is a result of tracking the six-step state information of the person model recognized in the system implemented according to an embodiment of the present invention. The horizontal axis of Table 1 is the actual six-stage status information of the person shown in each camera in six scenarios, and the vertical axis is the six-stage status information of the human model measured in the system. In the results of Table 1, the state tracking accuracy for the correctly tracked person in the proposed system is 96.01%. The state accuracy of the correctly tracked human model was 88.8% in the new entering state, 89.1% in the moving state of the human model in the camera, and 79.6% in the occluding state between the human models. 68% in the re-entering state of the human model, 100% state accuracy in the waiting state where the human model has fallen out of the camera range and entered the non-overlapping area and the fully exiting state of the human model. Showed.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and it is common in the art that various substitutions, modifications, and changes can be made without departing from the technical spirit of the present invention. It will be evident to those who have knowledge of.

As described above, the present invention has proposed a network-based person tracking system for efficiently delivering and receiving information of a tracking target recognized from a plurality of cameras. In this system, the JMF API and RTP are used to efficiently transmit and process a large number of media data to JAVA, enabling information to be tracked through the network connection between cameras. According to the present invention, a hierarchical person model is used to track a person in a camera and deliver the person's information to another camera, and the feature information of the modeled person is transferred to a server in real time, and the server receives By delivering the information to each connected client, each client could use the delivered feature to detect, recognize, and track the tracking target. The person tracking system according to the present invention can be easily applied to the implementation of a virtual space for tracking a plurality of people and obtaining a track of the movement of the tracked people in various environments such as indoors as well as outdoors.

In addition, this system, which can actively track people away from the existing monitoring methods, can track the movements of a large number of people and large and complex spaces that were difficult to do in the existing surveillance system. It can be widely used for security system.

Claims (5)

A plurality of features connected to each other through a network, extracting feature information of a person model to be tracked from the acquired images, comparing the extracted feature information with previously stored person list feature information, and sequentially tracking a person on an individual surveillance area; camera; And A server configured to receive and store the feature information from the plurality of cameras, provide the stored person list feature information to the plurality of cameras, and synchronize the plurality of cameras. Including, The characteristic information classifies the human body of the tracking object into a head, upper body and lower body, and includes hierarchical human model information including at least one of a location, a color, a moving direction, a center point, and boundary regions constituting the human body. ego, The hierarchical person model information includes information on the foreground image separated from the light-corrected input image, and the information on the foreground image is obtained by calculating a color similarity between each pixel and an adjacent pixel to determine the color of the color. Multi-person tracking system that includes blob information that is a set. delete delete The method of claim 1, The controller of the camera transmits the new blob as new human model information to the server when the minimum distance between the new blob indicated in the t-th frame and the person model obtained in the previous frame is less than a predetermined threshold, and the server A multi-person tracking system for updating a person model information stored in a person list by statistical information re-calculated the center and uncertainty of the new blob. The method of claim 1, The camera and the server has a multi-person tracking system having the status information of appearance, movement, overlap, waiting, re-appearance and exit for the human model.

Images