KR20210065520A - System for Object Tracking Using Multiple Cameras in Edge Computing - Google Patents

Abstract

An objective of the present invention is to efficiently support an object tracking application using multiple cameras in an edge computing environment. According to one embodiment of the present invention, an object tracking system using multiple cameras in an edge computing environment comprises: multiple IoT cameras including one or more virtual containers (VCs) and a hypervisor performing a camera function and frame preprocessing; and an edge server including a processing device for video processing, multiple camera queues in which frames received from the multiple IoT cameras stand by, and one or more virtual container masters (VCMs), and performing a tracking job for each tracking application through the virtual container masters. The virtual containers use a target template provided from the virtual container masters to provide only frames including a tracking target among frames provided from the hypervisor for the edge server, and perform a hand-over operation of monitoring the movement of the target and handing over a tracking job to an adjacent IoT camera.

Description

{System for Object Tracking Using Multiple Cameras in Edge Computing}

This application relates to an object tracking system using a plurality of cameras in an edge computing environment.

Edge Computing is a technology that utilizes computing resources as close to the client as possible without relying entirely on the cloud. Edge computing provides a variety of video analytics services such as object detection, identification, and tracking in real time based on data from network cameras at the edge. Among the various types of video analytics applications, object tracking applications require real-time analysis of real-time video streams. To do this, the edge server must track the target in real-time in the video frame fed by the edge IoT camera.

Conventional approaches for this can be broadly classified into two types: a server driving method and a camera driving method.

In a server-driven approach, the edge server simultaneously receives video streams from all relevant cameras and processes the video frames to suit application needs. However, as the number of cameras involved increases, it is fed a large amount of useless video frames, i.e. video frames that do not contain the target object, which creates a bottleneck for edge servers to handle and process raw video frames in time. . Using more powerful GPU hardware can mitigate this to some extent, but it can continue to hinder the scalability of edge servers, which in turn can result in lower quality of service (QoS).

On the other hand, the camera-driven approach allows each camera to analyze the video stream and discard useless frames, allowing the edge server to process a much smaller number of frames. However, the conventional camera driving method does not consider tracking using a plurality of cameras. This results in a large amount of energy consumption as all potentially relevant cameras for tracking must be running simultaneously. This can be critical to performance if the camera is running on battery power.

Accordingly, there is a need in the art for a method for efficiently supporting an object tracking application using a plurality of cameras in an edge computing environment.

In order to solve the above problems, an embodiment of the present invention provides an object tracking system using a plurality of cameras in an edge computing environment.

The object tracking system using a plurality of cameras in the edge computing environment includes: a plurality of IoT cameras configured including a hypervisor and one or more virtual containers (VC) in charge of camera functions and frame preprocessing; and a processing device for video processing, a plurality of IoT camera queues waiting for frames received from each of the plurality of IoT cameras, and one or more virtual container masters (VCMs), wherein the virtual container master and an edge server to perform a tracking operation for each tracking application through the virtual container, wherein the virtual container uses a target template provided from the virtual container master and only frames including a tracking target among frames supplied from the hypervisor are the edge It is provided to a server, monitors the movement of the target, and performs a handover operation of transferring a tracking operation to an adjacent IoT camera.

Incidentally, the means for solving the above problems do not enumerate all the features of the present invention. Various features of the present invention and its advantages and effects may be understood in more detail with reference to the following specific embodiments.

According to an embodiment of the present invention, it is possible to efficiently support an object tracking application using a plurality of cameras in an edge computing environment.

1 is a diagram illustrating the overall structure of an object tracking system using a plurality of cameras in an edge computing environment according to an embodiment of the present invention.
FIG. 2 is a graph showing the relationship between the speed of an object and the time it reappears in the field of view of an adjacent camera.
3 is a diagram illustrating a handover procedure between cameras according to an embodiment of the present invention.
4 is a diagram illustrating a trajectory of an object observed through a plurality of cameras.
5 is a diagram illustrating an average waiting time according to the number of applications running on an edge server.
6 is a diagram illustrating an application level goodput according to the number of applications running on an edge server.
7 is a diagram comparing useful and useless frames according to the number of running applications.
8 is a diagram illustrating an application-level good put and additional bandwidth usage according to an average moving speed of a target.
9 is a diagram comparing standby times in cameras.
10 is a diagram comparing energy consumption of all IoT cameras.

Hereinafter, preferred embodiments will be described in detail so that those of ordinary skill in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing the preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is 'connected' with another part, it is not only 'directly connected' but also 'indirectly connected' with another element interposed therebetween. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated.

First, things to be considered for efficient object tracking using a plurality of cameras will be described before describing an embodiment of the present invention.

In an object tracking system using a plurality of cameras, cameras are generally arranged according to the physical structure of a surveillance area. That is, depending on the speed and direction of the object's movement, an object that appears on one camera will reappear in the field of view of another camera relative to the previous camera, depending on its physical location. In other words, it is more efficient to utilize the spatial and temporal relationships between cameras installed in space, and only the relevant cameras can be activated instead of all cameras.

First, in the case of a stationary camera, the direction of the target object toward the inside of the camera field of view can be analyzed in advance. By analyzing the movement of each object throughout the video cluster, it is possible to identify objects moving from one camera to another adjacent camera. Surveillance cameras are usually installed at important points, such as exits and the only passage to other points, so from a certain point along a certain way, based on the direction of travel, it is possible to estimate the next point at which an object will appear. This means that all cameras do not have to constantly monitor and transmit video frames. Depending on the physical constraints created by the path, each camera may prioritize some set of cameras out of all cameras in the building. Therefore, by considering a spatially correlated set of cameras to predict the appearance of a target object, it is possible to reduce the search space of the camera.

As described above, after estimating a specific camera in which an object is detected after the object deviates from the field of view of the previous camera based on the spatial location, it is possible to predict the time when the object will appear in the specific camera. It depends on the target's movement speed. By comparing the current position and the previous position of the object for a given time, it is possible to evaluate how far the target object has moved from the camera during the given time. Assuming that the object moves at the same speed after it leaves the current camera's field of view, the time the object will appear in the next camera's field of view can be predicted with a margin of error.

In other words, spatial locality and temporal locality are key clues for supporting object tracking applications. In order to implement an efficient tracking application when multiple cameras are used, the direction and velocity of the target object are estimated from the camera currently hosting the target object. and efficient technique to pass control to the next camera.

On the other hand, since each tracking application has its own set of parameters including the target template, the resolution of the video frame, the frame rate, etc., each tracking task is executed in an independent environment created in a virtual container. In this case, camera-to-camera handover performance depends on moving that container from one camera to another.

To ensure the quality requirements of the tracking application (i.e., application-level goodput), the container must be light enough so that movement between cameras does not affect the real-time tracking process. This requires coordinating handovers between cameras and being able to boot the destination camera's container in a timely manner.

1 is a diagram illustrating the overall structure of an object tracking system using a plurality of cameras in an edge computing environment according to an embodiment of the present invention.

Referring to FIG. 1 , an object tracking system using a plurality of cameras in an edge computing environment according to an embodiment of the present invention may include an edge server 110 and a plurality of fixedly installed IoT cameras 120 .

The edge server 110 may be configured to include a processing unit (ie, GPU Units) for video processing, a plurality of IoT camera queues, and a virtual container master (VCM), and a plurality of video analysis applications (eg, For example, an object tracking application). In this case, the edge server 110 may perform a task for each application through the virtual container master.

As described above, since spatial locality is important in finding and tracking a target in an object tracking application, a frame received from each IoT camera may be configured to wait in a separate queue.

The virtual container master may poll the frames of IoT camera queues and validate and post-process the frames for application use. In addition, when the virtual container master receives the target template including the tracking target from the user, the virtual container master selects an arbitrary IoT camera and starts the virtual container using the target template to search for the target.

Meanwhile, each of the plurality of IoT cameras 120 may be configured to include a hypervisor and one or more virtual containers (VCs).

Here, the hypervisor may be in charge of general functions of a physical camera and frame preprocessing (eg, change of size or resolution, etc.).

The virtual container may manage the unique requirements for each tracking application and provide appropriate video frames to the edge server 110 accordingly. Each virtual container may analyze and re-identify frames supplied from the hypervisor for a target template, and provide only useful frames to the edge server 110 . In addition, the virtual container can monitor the movement of the target and perform a handover operation of passing the tracking task to an adjacent IoT camera.

A plurality of IoT cameras 120 are connected to the edge server 110 and other cameras through a network and may be equipped with a computing engine such as, for example, a Raspberry Pi to support a certain level of video processing. .

Additionally, each IoT camera 120 may accommodate multiple tracking tasks, where each task may be dedicated to one tracking application.

More specifically, in order to allocate a plurality of tracking tasks to the IoT camera 120 , it may be configured to virtualize a real camera using a Docker container designed to be lightweight compared to an existing virtual machine.

Since tracking applications require similar video analysis functions including object detection, re-identification, tracking, etc., a software library such as OpenCV, for example, can be used.

Once a Docker container for tracking is created, it can later be configured to reuse previously downloaded libraries by other containers. This speeds up the creation of the next container, and the executable portion of that common library can be copied to the memory space of each container for isolation. In addition, there is a need to reduce duplicates and unaltered portions in each tracking container while maintaining isolation using lightweight containerization techniques as described below.

To maintain software libraries commonly used for video analysis, the hypervisor may include a Common Library Execution Manager. The Common Library Execution Manager may have a shared storage called Common Library Storage (CLS).

A virtual container created for any tracing application can mount the CLS location for access via a command-line interface (CLI) and perform package imports in a programming language. When executing a command, the Virtual Camera Driver in the virtual container sends the command to the Common Library Execution Manager instead of directly executing the command.

Similarly, if multiple virtual containers requesting the same library are running with the same video stream, the common library execution manager can execute the command only once and copy the results to each virtual container.

During startup, the IoT camera 120 may send a join message to the edge server 110 to join the edge network. Thereafter, the edge server 110 may respond to the IoT camera 120 with a list of common libraries recently used by other already connected IoT cameras 120 . If there is no previously connected IoT camera 120 , the edge server 110 may provide a basic video analysis library set to the IoT camera 120 .

Meanwhile, when a new virtual container is created in the IoT camera 120 according to the request of the edge server 110 , the virtual container detects an object matching the provided target template and starts tracking the detected object.

First, it is possible to determine when and in which direction the object moves by analyzing the spatiotemporal relationship between the IoT cameras 120 . To this end, the IoT camera 120 includes, for each object, a timestamp when the object disappears from the camera's field of view, the next timestamp when the object reappears in an adjacent camera, the average movement speed of the object until it disappears, and the direction of movement of the object. trajectory data can be recorded.

Accordingly, the IoT camera 120 may estimate from which adjacent camera the object reappears from the recorded trajectory data. The direction of movement of the target will be displayed as one of the following values: North (N), Northeast (NE), East (E), Southeast (SE), South (S), Southwest (SW), West (W), and Northwest (NW). can

When an object disappears from the field of view of one camera, Equation 1 for estimating the time to reappear in an adjacent camera may be derived by applying a linear regression model. Here, s _xy is the speed of the object and represents the distance moved per unit time, and α, β, and ε are preset variables. However, without prior knowledge of the coordinates and movement direction of the object in the field of view of each camera, it is impossible to guarantee the camera to which the object will move. Therefore, each IoT camera 120 records the outbound area of each object movement. That is, each IoT camera 120 may predict the reproduction of an object in a neighboring camera based on the above two pieces of information.

[Equation 1]

exp = α + βs _xy + ε

The obtained model is stored in a camera map of each IoT camera 120 and can be used as a basis for tracking spatiotemporal objects and determining handover. FIG. 2 is a graph showing the relationship between the speed of an object and the time it reappears in the field of view of an adjacent camera.

Algorithm 1 below shows a tracking operation including identifying a target in a virtual container and estimating its moving speed and direction.

First, the IoT camera may detect an object in a current frame by extracting a feature object by executing, for example, neural network-based object detection. Among all the feature classes, the IoT camera can remove irrelevant feature classes except for the feature class that matches the class to be tracked. The detection & re-identification process of a virtual container can compare all potential candidates of the detected object class to the target template. At time t, each candidate coordinate k of camera i and the target template can be transformed into vectors to calculate similarities, e.g. Euclidean distance. In addition, the candidate list most similar to the target template can be output in order. Such a process can be expressed in lines 1 to 11 of Algorithm 1.

If there is an object in the frame, the object's coordinates can be entered into the virtual container's Tracker for further tracking. The tracker can implement a correlation-based tracking algorithm that tracks the coordinates of a frame to the next frame. To this end, the tracker may first initialize a new coordinate position of the newly found object. After that, you can check if the object is still in the next frame. An object may be considered missing if the coordinates of a previously registered object are no longer correlated with the next predefined number of frames for a predetermined time. By operating in this way, the tracker can identify objects that are new to the primary camera's field of view. Such a process can be expressed in lines 12 to 33 of Algorithm 1. In addition, by performing re-identification of the predefined number of frames as described above, the processing waiting time can be shortened.

After the target is tracked in this way, the following three things must be monitored in real time in order to determine an appropriate handover to the adjacent camera.

First, we estimate the time it will take for the object to appear again in an adjacent camera. As expressed in lines 23 to 24 of Algorithm 1, the moving speed of the object can be estimated by calculating the latest coordinate position by the elapsed time. Based on this, the expected arrival time (exp) of the object in the adjacent camera can be estimated according to Equation 1 described above.

Next, predicting when the subject will currently leave the camera's field of view is important to stop the currently running task. Similar to known methods of position prediction, for example dead reckoning, the tracker of a virtual container multiplies the number of frames (f) by the current movement speed as expressed in line 25 of Algorithm 1 to determine the future position of the object. can be predicted Here, f should be appropriately selected in consideration of the moving speed of the object.

Finally, we need to estimate the direction the object is facing to determine which camera should be ready. As expressed in line 26 of Algorithm 1, the direction of movement of the object can be estimated by setting the north to 0 degrees and calculating the angle between the current coordinate and the previous coordinate.

3 is a diagram illustrating a handover procedure between cameras according to an embodiment of the present invention, for example, a handover procedure from camera i to camera j.

In addition, the following algorithm 2 shows a process of handling a handover request in the IoT camera.

Referring to FIG. 3 , first, if the tracker of the virtual container confirms that the target is out of view, it may request a handover manager of the hypervisor to send a handover request message to an adjacent IoT camera.

(1) At time t _trigger , the virtual container of camera i can detect the disappearance of the object from the frame (line 27 of Algorithm 1).

(2) Camera i's virtual container sends a handover message to the hypervisor's handover manager with a handover message containing a) the message type, b) the target's expected arrival time (exp), and c) the target's template captured by camera i. You can ask it to be sent to j (line 30 of Algorithm 1).

Meanwhile, when the handover message is received from camera j, the remaining handover procedure may be performed as described below.

(1) At time t _{trigger+transmission} , camera j can process the message. If t _exp is _{shorter than t cur} , a new VC(app1, task2) must be loaded to track the target (lines 3 and 4 of Algorithm 2).

(2) exp cannot always predict the exact arrival time of an object. When the object is out of view of camera i, the new exp value is updated in camera j. Camera j can adjust _{t exp} by averaging the _{previous t exp} values to reserve the container. (Lines 6-8 of Algorithm 2)

(3) At time t _adj , camera j may launch a virtual container with its type and parameters to support the application. When the virtual container is launched on camera j, an Ack message to unload the virtual container from camera i may be returned.

(4) However, if the object does not reach camera j at the given offset time, camera j considers the object to have disappeared from view and sends a disappear message to camera i and the virtual container master of the edge server. Accordingly, it is possible to allow the application user to decide whether to restart the application.

(5) According to the above-described process, camera j starts tracking and the handover procedure from camera i to camera j is completed.

In order to evaluate the performance of the embodiment of the present invention as described above, an experiment as described below was conducted.

Three IoT cameras were installed in the hallway of the building and a 60-second video was recorded at a resolution of 480x640 pixels while the target moved along these cameras. 4 is a diagram illustrating a trajectory of an object observed through a plurality of cameras, and specifically, a trajectory of a target object moving along cameras 1, 2, and 3 is shown. In addition, a second video was recorded using two cameras (cameras 1 and 3) to evaluate the system in terms of temporal locality. Here, the target object to be tracked is a specific person.

It is assumed that the edge server is equipped with a somewhat limited amount of computing resources compared to a remote data center or cloud. In this experiment, the edge server is equipped with two Intel® Xeon E5-2630v4 CPUs (20 cores total, 2.2GHz each), an Nvidia GeForce 1080Ti GPU, and 128GB of memory space (RAM). Recently developed smart cameras such as the Amazon Deeplens or Nvidia Jetson TX are equipped with two cores, 8G memory and built-in accelerators. Similar to the IoT camera setup, a barebones machine with an Intel(R) Pentium(R) 2020M CPU with a 2.4GHz clock speed, 4GB memory footprint and Intel Movidius neural computing stick was used to accelerate the detection phase.

For synchronization, all IoT cameras are initialized at the start of every experiment to calibrate the time with an edge server. Implement a sample tracking application and its operations in Python, including video processing operations, using OpenCV. For the object detection model, we use a Single Shot Multibox Detector (SSD) with MobileNet. MOSSE trackers are powerful because they are resistant to changes in frame size, lighting, and scale, and because they process frames at high rates, even on devices with low computing power (about 80 frames per second in IoT cameras).

An evaluation index, which will be described later, may be used to verify the performance.

1) Application level goodput (%)

In order for the tracking application to be able to provide users with the correct tracking results in a timely manner, tracking actions in the edge cloud must be processed within the specified event occurrence time. Therefore, application-level goodput can be defined as the number of frames processed within a given time after an actual event occurs. In this experiment, the expiration times were set to 0.5 sec and 1 sec, respectively. Measures how long a frame is held in the edge server's queue. Frames are not deleted from the queue even if they contain outdated timestamps.

2) Total bandwidth usage (%)

One of the main drivers supporting application requirements is how much useless video frames sent from IoT cameras can be reduced. To quantify the number of frames sent from the camera to the edge server, we measure the number of bytes sent to the server for each experiment. All frames are uncompressed and each frame is about 900kB in size.

3) Processing delay (sec)

In an embodiment of the present invention, the application task of the IoT camera is containerized. To quantify the processing latency of an IoT camera, each step of the process is measured in seconds.

4) Energy consumption (%)

In terms of energy consumption, we measure the amount of work required to run application tasks on each IoT camera. Assuming that all IoT cameras are equipped with the same hardware, we measure the power consumption of Joule (J) according to the number of applications.

In addition, the above-described embodiment of the present invention is compared with the prior art described below.

1) Prior art 1 (bl1: streaming all frames)

Every video frame from each IoT camera is blindly sent to an edge server. At the edge server, every frame transmitted from each camera queue goes through a re-identification step to find the target object in each video stream in a round-robin fashion. When an object is found, frames from one camera queue continue to be processed, while video frames from other queues are ignored for further processing as they have nothing to track. Prior art 1 corresponds to a cloud-based video processing method in which the entire video analysis operation is performed in an edge server.

2) Prior art 2 (bl2: object detection in camera, tracking in edge server)

Each IoT camera independently inspects the video stream and discards useless frames based on the information of the object to be detected. Before running the application, the edge server sends a pre-trained inference model to each IoT camera. IoT cameras use this model to try to detect specific objects in a given frame. If the object is included, the frame is sent to the server, otherwise the frame is dropped. As multiple applications run concurrently and request to detect different types of objects, the edge server updates its inference model to cover all other types of objects. For each application, the edge IoT camera runs a container that processes video frames and decides to discard or transmit each frame. However, handover between cameras is not supported when the target is moving.

For performance evaluation, this experiment compares the application-level goodput of the edge server while changing the number of concurrently running applications. First, we analyze the latency, how long each useful frame, i.e. the frame actually used for tracking, remains in the queue before being delivered to the application.

FIG. 5 is a diagram showing the average waiting time according to the number of applications running on the edge server, and shows the waiting time of a frame useful in the embodiment of the present invention and the prior arts 1 and 2 described above. Here, it is assumed that each application of the edge server receives frames from two cameras at the same transmission rate.

According to the prior art 1, all IoT cameras supply frames to the edge server without filtering, thereby increasing the amount of frames waiting in the edge server for image processing. If there is one application, the average latency of frame delivery is about 690 ms, and as the number of applications increases, the latency increases significantly to about 5700 ms.

Further, according to prior art 2, each IoT camera autonomously starts and stops frame transmission in order to reduce the total number of frames received by the edge server. As a result, the queue length of the edge server is shortened, resulting in a waiting time of about 350 ms when there is only one application. In addition, edge servers are not congested and latency does not change as the number of applications increases.

On the other hand, according to the embodiment of the present invention, when there is one application, the average waiting time is about 313 ms, which is slightly smaller than that of the prior art 2. This is because, according to the embodiment of the present invention, handover between cameras is supported, so that a specific object can be continuously tracked by predicting in advance when each IoT camera will handover a task to the next IoT camera. Moreover, even when the number of applications served by the edge server increases, switching through multiple queues does not incur significant overhead, thus allowing frames to be delivered to applications on time.

6 is a diagram illustrating an application level goodput according to the number of applications running on an edge server.

As can be seen from FIG. 6 , according to an embodiment of the present invention, it can be seen that all frames are processed before t1, and about 89.3% of frames are processed within t0.5.

On the other hand, according to prior arts 1 and 2, when one application is running on the edge server, the application-level goodput is about 86.2% (t1) and 50.8% (t0.5), 81.1% (t1), and 42.5%, respectively. (t0.5). Also, as the number of concurrently running applications increases, it can be seen that the application level goodput drops significantly due to the large amount of frames that must be processed before useful frames can be processed.

8 is a diagram illustrating an application level good put and additional bandwidth usage according to an average moving speed of a target.

In embodiments of the present invention, to evaluate whether temporal locality is handled well, it is possible to play different sets of videos in which the subject moves at different speeds and compare the additional bandwidth usage with the application level goodput. In Equation 1, α and β are set to −0.15 and 10, respectively. The x-axis represents the average moving speed of the target, the left y-axis shows the application-level goodput, and the right y-axis shows the additional bandwidth usage.

It can be seen from FIG. 8 that all frames reach the t1 (1 second) threshold, but 83% to 84.1% of the frames are processed within the t0.5 (0.5 second) threshold at various target moving speeds. Also, experiments show that the virtual container scheduled at _{t exp is loaded slightly earlier as the target speed increases.} This increases the number of additional frames sent to the edge server application by 25.1%.

7 is a diagram comparing useful and useless frames according to the number of running applications.

To see how many stale video frames are filtered out, you can compare the total number of frames sent to the edge server (i.e. stale frames + useful frames) based on the number of concurrently running applications.

As can be seen from FIG. 7 , in the prior art 1, the useless frame increases from 80% to 82% of the total frame. This is because all frames are blindly sent to the edge server.

In contrast, in the embodiment of the present invention and prior art 2, it can be seen that the bandwidth usage is much less because the camera transmits only the frame including the target object to the edge server.

9 is a diagram comparing latency in cameras, where p(d&r) represents the delay generated when detecting and re-identifying an object, p(d) represents the delay generated in the target correlation, and sc is Represents the delay when streaming raw video frames without any other processing. From this, it can be seen that the overhead of creating virtual containers using Docker does not significantly affect the overall performance.

When a handover message is received from a nearby IoT camera, the time taken to create and initialize a virtual container in the IoT camera is measured. Table 1 shows the virtual container creation and initialization times.

step time (ms) produce 2814.574 reset 1.253

That is, creating a trace job instance from scratch takes about 2.814 seconds. In addition, the initialization time is the time it takes for the virtual container to process a frame or start transmitting, and the initialization time takes about 0.00125 seconds after a task is created. In other words, once the virtual container is created, it can be seen that the initialization time does not take much.

10 is a diagram comparing energy consumption of all IoT cameras.

Depending on the number of running applications, we measured the energy consumption of the camera when a part of the application's work is running on the IoT camera. We added the energy consumption of 3 IoT cameras, where the energy consumption of each IoT camera in idle state is about 30J.

According to the embodiment of the present invention, the smallest amount of energy is consumed in all cases, and in particular, it is possible to save up to 62.13% of energy consumption compared to the prior art 2 . In addition, prior art 1 also consumes more energy than the embodiment of the present invention.

In the case of prior art 1, 570J is consumed regardless of the number of running applications. This is because, in prior art 1, the IoT camera does exactly the same thing as streaming every video frame to the edge server. In this way, transmitting both useful and useless video frames over a wireless network consumes a lot of energy, which is inefficient in terms of network overhead and energy consumption.

In the case of prior art 2, energy consumption consumes 954J when there is one running application, and consumes 1323J when there are three applications. In the prior art 2, since there is no logic for an IoT camera to coordinate a tracking operation with another camera, each IoT camera must always execute an application-specific operation. Typically, each application-specific task can be run in an isolated environment using containers, and the number of containers depends on the number of applications. Therefore, the overall energy consumption of an IoT camera is proportional to the number of running applications.

In contrast, according to the embodiment of the present invention, when there is one running application, 393J is consumed, and when there are three applications, 501J is consumed. In the present invention, the IoT camera can arrange an application-specific task to an appropriate one through spatiotemporal adjustment, and even if the number of applications increases, each IoT camera does not necessarily execute all tasks simultaneously. This means that only those cameras that are presumed suitable to supply useful frames for a particular application need to maintain their virtual containers.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

Claims (6)

A plurality of IoT cameras configured to include a hypervisor and one or more virtual containers (VC) in charge of camera functions and frame pre-processing; and
It is configured to include a processing device for video processing, a plurality of IoT camera queues in which frames received from each of the plurality of IoT cameras wait, and one or more virtual container masters (VCMs), through the virtual container master Includes an edge server to perform tracking tasks for each tracking application,
The virtual container uses a target template provided from the virtual container master to provide only a frame including a tracking target among frames supplied from the hypervisor to the edge server, monitors the motion of the target, and performs a tracking operation with a neighboring IoT camera An object tracking system using a plurality of cameras, characterized in that performing a handover operation to be passed to.
The method of claim 1,
The hypervisor is
a Common Library Execution Manager that maintains a software library for video analysis;
The common library execution manager is an object tracking system using a plurality of cameras, characterized in that provided with a common library storage (CLS; Common Library Storage) as a shared storage.
3. The method of claim 2,
The object tracking system using a plurality of cameras, characterized in that the virtual camera driver included in the virtual container sends the command to the common library execution manager when the command is executed.
4. The method of claim 3,
The common library execution manager executes a command only once when a plurality of virtual containers requesting the same library are executed with the same video stream and copies the result to each virtual container.
The method of claim 1,
The virtual container is
The object tracking system using a plurality of cameras, characterized in that it further comprises a tracker for performing re-identification according to a correlation-based tracking algorithm for a predefined number of next frames when there is the target in the frame.
The method of claim 1,
The virtual container determines handover to a neighboring IoT camera in consideration of an expected time for the target to appear in the field of view of the neighboring IoT camera, an expected time when the target will depart from the field of view of the current IoT camera, and the moving direction of the target. An object tracking system using multiple cameras.

Images