KR20200011237A - Stream reasoning surveilance system and Stream reasoning surveilance method using the same - Google Patents

Abstract

The present invention relates to a stream reasoning surveillance system capable of automatically tracking the separation or movement of a specific person in a place where many persons come and go and a stream reasoning surveillance method using the same. To this end, the stream reasoning surveillance system comprises: a common device interface layer collecting data; an image analysis module analyzing a collected image; and a trajectory inference module inferring the trajectory of a moving object. The stream reasoning surveillance system provides the stream reasoning surveillance method which comprises: a step of collecting images captured by a plurality of cameras by the common device interface layer; a step of merging the collected images by the image analysis module; a step of determining the same group for the moving object in the merged image in the trajectory inference module; and a step of determining whether the moving object satisfying separation conditions is present in the group. Therefore, the present invention can provide the stream reasoning surveillance system which can automatically recognize a specific situation at a site where many persons come and go to provide a notification to a surveillant, thereby efficiently monitoring the situation with less manpower.

Description

Stream reasoning surveilance system and Stream reasoning surveilance method using the same}

The present invention relates to a stream listening monitoring system and a stream listening monitoring method using the same. Ning surveillance method.

CCTVs are installed everywhere to protect citizens from violent crimes and accidents. These CCTVs are increasing by an average of 15 ~ 20% each year, but there is a shortage of manpower to operate and monitor CCTVs.

Therefore, in reality, the surveillance system through manpower has its limitations, and the necessity for research on intelligent CCTV that can automatically grasp the situation is increasing.

Today's intelligent CCTV focuses on object recognition and tracking through image analysis.

According to Korean Patent No. 10-1732115 shown in FIG. 1, displaying an image photographed by a main surveillance camera in a main screen region, and displaying images photographed by a plurality of sub surveillance cameras in a plurality of sub screen regions, respectively; Extracting an object feature by recognizing an object included in an image displayed in the main screen area; when one of the objects recognized in the main screen area and extracted from the main feature is selected by the user, the main screen area and the plurality of subscreen areas Calculating similarity with the object selected by the user for each recognized object; Displaying the similarity calculated for each object.

However, the above patent is only a function for calculating the similarity with respect to one object selected by the user does not automatically recognize the relationship with various people. Understanding these relationships can help you infer the trajectories of different people more accurately.

In order to immediately recognize the situation occurring in the field where many people are moving, it is essential to collect and analyze various video information to understand the relationship with people. This requires smart city middleware and stream listening technologies.

The present invention has been made to achieve the above object, and to provide a stream listening monitoring system that can automatically inform the monitor by monitoring the situation in the field where many people come and go.

In addition, it is to provide a stream listening monitoring system that can infer the trajectory of people to determine the abnormality of the movement path of a specific person.

In order to achieve the above object, the present invention provides a stream listening monitoring system comprising a common device interface layer for collecting data, an image analysis module for analyzing the collected image, and a trajectory inference module for inferring the trajectory of the moving object. The stream listening monitoring system may include: collecting images captured by a plurality of cameras by the common device interface layer; Merging the collected images by the image analysis module; Determining the same group for the moving object in the merged image by the trajectory inference module; It provides a stream listening monitoring method for performing a step of determining whether there is a moving object that satisfies the departure condition in the group.

The image analysis module extracts an rgb value for each pixel of the moving object from an image collected by one camera, and extracts an rgb value for each pixel of the moving object from an image collected by another camera, thereby extracting an rgb value for each rgb. The step of calculating the similarity may be performed.

The image analysis module may determine the color of the moving object, and the trajectory inference module may determine the same object in a plurality of different images using the color.

The image analysis module extracts trajectory data for a moving object, and the trajectory inference module determines the same object based on the autocorrelation coefficient, and generates the entire trajectory for the same object using the trajectory data. It is desirable to.

The trajectory inference module preferentially infers a trajectory for the moving object by analyzing an image collected by a camera photographing an image of an adjacent region with respect to one moving object, and photographing the specific camera when there is no image collected by the specific camera. It is preferable to infer the trajectory of the moving object by analyzing the image collected by the camera photographing the region closest to the region.

The locus inference module may determine whether the locus is the same group based on the locus similarity for the locus of each moving object and the distance between moving objects for each time zone.

The image analysis module merges some of the images collected from the plurality of cameras into camera groups, and the trajectory tracking module infers a trajectory for each camera group.

The trajectory inference module infers a trajectory by analyzing an image of a predetermined time collected from the camera, and infers a trajectory by updating an image every predetermined time.

According to the present invention, it is possible to provide a stream listening monitoring system capable of efficiently monitoring the situation with a small number of personnel by automatically recognizing a specific situation at a site where many people come and go.

In addition, it automatically infers the trajectories of people, enabling immediate response in the event of a lost child.

1 is an exemplary view for explaining an object tracking method using a surveillance camera according to the prior art;
2 is a block diagram showing the configuration of a smart city middleware system used in the stream listening monitoring system according to the present invention;
3 is a block diagram showing a procedure of a stream listening monitoring system according to the present invention;
4 is an exemplary view showing an example of a camera group used in FIG. 3;
5 is an explanatory diagram showing a data format of a stream listening monitoring system according to the present invention;
6 is an exemplary view for explaining the same object determination method according to the present invention;
7 is an exemplary view for explaining a method for processing the same object according to the present invention;
8 is an exemplary view for explaining a method for determining a group according to the present invention;
9 is an exemplary view for explaining a processing method when data reception is not performed.
10 is an exemplary view for explaining a method for determining automatic lost occurrence according to the present invention.

The configuration and operation of the embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2, the stream listening monitoring system according to the present invention may be implemented using a smart city server 100, and the smart city server 100 may include a common device interface layer 110 and an image analysis module ( 230, the trajectory inference module 250 is configured.

The smart city middleware collects data through the common device interface layer 110 from a plurality of cameras photographing each region.

3 shows an overview stream listening monitoring method according to the present invention. When data including an image captured by each camera is collected, the data is adapted to a format determined by the common device interface layer 110. In addition, the data are divided into a plurality of camera groups predetermined by the image analysis module 230, and each camera group infers the trajectory of the moving object by analyzing the image at a predetermined time interval.

In detail, each camera photographs a predetermined region, and the image analysis module 230 divides the region to tie up the regions photographed by the plurality of cameras to form a camera group.

Referring to FIG. 4, a total of nine cameras photograph each region. In other words, the first camera is located at the upper left, the second camera is located at the center, and the fifth camera is located at the center of each camera. At this time, the image analysis module 230 merges the images transmitted from the cameras 1, 2, 4, 5 to create a camera group A. In addition, by merging the images sent from the cameras 2, 3, 5, 6 to the camera group B, and merges the images sent from the cameras 4, 5, 7, and 8 to the camera group C, 5, Merge images from cameras 6, 8, and 9 to create camera group D.

Meanwhile, the trajectory inference module 250 packetizes and divides 1 minute of image data among the image data merged into the camera group, and infers the trajectory of the moving object using the image data packet for 1 minute of packet time. In addition, the image data packet is updated every three seconds to generate a new image data packet.

Referring to FIG. 5, when the moving object A is detected, the image analysis module generates a data format including an ID of the moving object. The data format may include a CCTV ID, a clothes color, a key, a hair color, a bag carrying case, wearing glasses, a wearing time, an entry time, a trajectory data, etc. photographing the moving object.

The trajectory inference module 250 first determines whether the moving object of the image photographed by one camera and the moving object of the image photographed by the other camera are the same moving object.

Specifically, the image analysis module extracts the color values r, g, and b for each pixel of the moving object 1 photographed by the camera 1, and also extracts the color values for each pixel of the moving object 2 photographed by the camera 2 which is another camera. Rgb histogram is generated by extracting r, g, b values. The x-axis (horizontal axis) of the rgb histogram represents the brightness, and the y-axis (vertical axis) represents the number of pixels for the corresponding brightness.

Then, in the rgb histogram by the moving object 1, depending on the number of pixels

X (1,1), X (1,2), X (1,3),... , A sequence of X (1,255) is generated,

In the histogram of rgb by moving object 2,

X (2,1), X (2,2), X (2,3),... , A sequence of X (2,255) is generated.

Then, the same object similarity T of the moving object 1 and the moving object 2 is calculated as follows.

X_bar means the average of the sequence X, and the similarity of the same object may be calculated for each of r, g, and b.

The trajectory inference module 250 determines that an object having the same object similarity of 0.7 or more is the same object, and when there are a plurality of objects having the same object similarity of 0.7 or more, the object having the highest object similarity is determined to be the same object.

The data format generated by the image data includes trajectory data, and the coordinate data of the moving object is stored for each time. If it is determined that the same object is determined by the trajectory inference module, the image analysis module adds ID information of the same object to the data format and stores the same as shown in FIG. 7. In this way, the same object can be quickly determined even if the image is taken by another camera.

In addition, the trajectory inference module merges the trajectory data using the trajectory data to generate the entire trajectory for the same object in the camera group. At this time, the coordinates of each moving object at the same time are generated.

When the entire trajectory for the same object is generated, the trajectory inference module calculates the trajectory similarity between the trajectories. The trajectory similarity P is calculated as follows.

The trajectory coordinates of the moving object 1 are (x11, y11), (x12, y12), (x13, y13),... , (x1n, y1n)

The trajectory coordinates of the moving object 2 are (x21, y21), (x22, y22), (x23, y23),... , for (x2n, y2n)

In addition, the trajectory inference module 250 measures the distance between moving bodies at a predetermined time interval to determine whether the relative distance is within a predetermined distance. For example, the relative distances of the mobile 1 and the mobile 2 may be measured for 5 minutes at intervals of 5 seconds, and may be stored as the close distance if the relative distance between the mobile 1 and the mobile 2 is within 10 m, and the deviation distance if the outside of the mobile is 10 m. .

The trajectory inference module 250 calculates the number of times the relative distances of the mobile body 1 and the mobile body 2 are stored as the proximal distances for 5 minutes to determine whether the proximal distance is 70% or more of the total measurement times.

In conclusion, the trajectory inference module 250 determines that the trajectory similarities Px and Py of the mobile 1 and the mobile 2 are both 0.7 or more, and the proximity distance is 70% or more, and the mobile 1 and the mobile 2 are the same group. On the other hand, if the above conditions are not satisfied, the mobile 1 and the mobile 2 are not determined to be the same group.

The trajectory inference module 250 infers a trajectory for the moving object by first analyzing an image collected by a camera photographing an image of an adjacent area when one moving object disappears from one camera image. In this case, when data is not received due to a failure or other cause of a specific camera as shown in FIG. 9, the trajectory inference module expands the analysis area and collects the data captured by the camera closest to the area photographed by the camera where no data is received. Analyze the trajectory of the moving object by analyzing the captured images.

Next, referring to FIG. 10, a lost alarm method using the stream listening monitoring system according to the present invention will be described.

The trajectory inference module 250 determines that the moving object 1 or the moving object 2 has left the group when the departure condition is satisfied in the moving object belonging to the same group. The detachment condition may be configured to be achieved when the trajectory similarities of the moving body 1 and the moving body 2 are both less than 0.7 and the proximity distance is less than 70%.

When there are three or more moving objects belonging to the same group, the departure condition is satisfied between the moving object 1 and the moving object 2. As a result of the analysis, it is determined that the one with fewer moving objects belonging to the same group has left the group.

 For example, if the departure condition is satisfied between the mobile body 3 and the mobile body 1 and the departure condition is not satisfied between the mobile body 3 and the mobile body 2, the mobile body 1 is determined to have left the group.

In the trajectory inference module, when the departure condition is satisfied, the moving object ID and the camera number photographing the area are stored in a separate storage. Alarms can also be triggered if necessary. The data stored in this way can be displayed again and checked if necessary.

Although described above with reference to embodiments of the present invention, those skilled in the art will be variously modified and modified within the scope of the present invention without departing from the spirit and scope of the present invention described in the claims below. It will be understood that it can be changed.

100: smart city middleware 110: common device interface layer
130: common application interface layer 200: context inference computing layer
230: image analysis module 250: trajectory inference module
300: cloud computing layer

Claims (5)

A stream device monitoring system comprising a common device interface layer for collecting data, an image analysis module for analyzing a collected image, and a trajectory inference module for inferring a trajectory of a moving object, the stream listening monitoring system comprising:
Collecting images photographed by a plurality of cameras by the common device interface layer;
Merging the collected images by the image analysis module;
Determining the same group for the moving object in the merged image by the trajectory inference module;
And determining whether there is a moving object that satisfies the departure condition in the group.
The method of claim 1,
The image analysis module extracts an rgb value for each pixel of the moving object from an image collected by one camera, and extracts an rgb value for each pixel of the moving object from an image collected from another camera, thereby extracting the same object for each rgb. And calculating a similarity level.
The method of claim 2,
The image analysis module extracts the trajectory data for the moving object,
And the trajectory inference module determines the same object based on the autocorrelation coefficient, and generates the entire trajectory for the same object using the trajectory data.
The method of claim 3,
The trajectory inference module preferentially analyzes an image collected by a camera photographing an image of an adjacent region with respect to one moving object, and infers a trajectory for the moving object, and when no image is collected from the specific camera, the specific camera captures the image. Stream deduction monitoring method characterized in that to infer the trajectory to the moving object by analyzing the image collected by the camera photographing the region closest to the.
The method of claim 3,
And the trajectory inference module determines whether the trajectory inference module is the same group based on the trajectory similarity for the trajectory of each moving object and the distance between the moving objects for each time zone.

Images