WO2020029518A1

WO2020029518A1 - Monitoring video processing method, device and computer readable medium

Info

Publication number: WO2020029518A1
Application number: PCT/CN2018/123511
Authority: WO
Inventors: 王翼
Original assignee: 平安科技（深圳）有限公司
Priority date: 2018-08-10
Filing date: 2018-12-25
Publication date: 2020-02-13
Also published as: CN109118510A

Abstract

Disclosed in the embodiments of the present application are a monitoring video processing method, a monitoring video processing device and a computer readable medium. The method comprises: acquiring video image frames of a monitoring video, and using an inter-frame differential algorithm to determine whether a moving object is present in a current image frame among the video image frames; if a moving object is present in the current image frame, extracting the contour of the moving object in the current image frame; then tracking a moving trajectory of the moving object according to the contour of the moving object in combination with the Camshift algorithm; finally, saving video image frames in which the moving object is present as well as the moving trajectory of the moving object. By means of the embodiments of the present application, it is possible to effectively detect a video segment in which a moving object is present in a monitoring video and track the moving object, and only save a monitoring video segment in which the moving object is present during the process of saving the monitoring video, thereby saving storage space.

Description

Monitoring video processing method, device and computer-readable medium

This application claims the priority of a Chinese patent application filed on August 10, 2018 with the Chinese Patent Office, application number 2018109082310, and application name "A Surveillance Video Processing Method, Apparatus, and Computer-readable Media", all of which passed Citations are incorporated in this application.

Technical field

The present application relates to the technical field of surveillance video processing, and in particular, to a surveillance video processing method, device, and computer-readable medium.

Background technique

For surveillance video, what is really useful in surveillance video is the surveillance picture that has changed and there are moving objects, but it has no value for the surveillance picture that remains unchanged for a long time. Therefore, in the application of intelligent surveillance video processing systems, the detection and trajectory tracking of moving object images are the core technologies in video surveillance systems. However, the existing surveillance video processing is to monitor and save all the pictures captured by the camera, and then perform corresponding processing on the saved surveillance video, such as compressing and blurring the saved surveillance video to save storage.

In the existing surveillance video processing methods, there is no behavior trajectory tracking for moving objects. If the surveillance picture remains stationary for a long time, these stationary pictures are not the pictures we want to monitor, but the monitoring equipment still keeps these pictures Saved to the server, causing unnecessary waste of storage space.

Summary of the invention

An embodiment of the present application provides a monitoring video processing method, which can effectively detect and track moving object video segments in which a moving object exists in the monitoring video, and only save the monitoring video in which the moving object exists during the process of saving the monitoring video. Fragments, saving storage space.

In a first aspect, an embodiment of the present application provides a monitoring video processing method, which includes:

Acquiring a video image frame of a monitoring video, and determining whether a moving object exists in a current image frame in the video image frame by using an inter-frame difference algorithm;

If there is a moving object in the current image frame, extracting the outline of the moving object in the current image frame;

Track the moving trajectory of the moving object in combination with the Camshift algorithm according to the outline of the moving object;

A video image frame where the moving object exists and a moving track of the moving object are saved.

If it exists, extract the outline of the moving object in the current image frame;

Track the moving trajectory of the moving object according to the contour of the moving object in combination with the Camshift algorithm;

In a second aspect, an embodiment of the present application provides a monitoring video processing apparatus, where the monitoring video processing apparatus includes a unit for executing the method of the first aspect.

According to a third aspect, an embodiment of the present application provides another monitoring video processing apparatus, including a processor, an input device, an output device, and a memory, and the processor, the input device, the output device, and the memory are connected to each other, where the memory A computer program for storing a monitoring video processing device to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer storage medium stores a computer program, where the computer program includes program instructions, and the program instructions cause the processing when executed by a processor. The processor performs the method of the first aspect.

In the embodiment of the present application, a video image frame of a monitoring video is acquired, and an inter-frame difference algorithm is used to determine whether a moving object exists in the current image frame in the video image frame; if it exists, extract the moving object in the current image frame. Outline. Then, the moving trajectory of the moving object is tracked according to the contour of the moving object in combination with the Camshift algorithm. Finally, the video image frame where the moving object exists and the moving track of the moving object are saved. Through the embodiments of the present application, it is possible to effectively detect and track a moving object video segment in a monitoring video, and only save the monitoring video segment having a moving object in the process of saving the monitoring video, thereby saving storage space. .

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic flowchart of a monitoring video processing method according to an embodiment of the present application; FIG.

2 is a schematic flowchart of another monitoring video processing method according to an embodiment of the present application;

FIG. 3 is a schematic flowchart of another monitoring video processing method according to an embodiment of the present application; FIG.

4 is a schematic block diagram of a monitoring video processing apparatus according to an embodiment of the present application;

FIG. 5 is a schematic block diagram of a monitoring video processing apparatus according to another embodiment of the present application.

detailed description

In the following, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

It should be understood that when used in this specification and the appended claims, the terms "including" and "comprising" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude one or The presence or addition of a number of other features, wholes, steps, operations, elements, components, and / or sets thereof.

For surveillance video, what is really useful in surveillance video is the surveillance picture that has changed and there are moving objects, but it has no value for the surveillance picture that remains unchanged for a long time. Therefore, in the application of intelligent surveillance video processing systems, the detection and trajectory tracking of moving object images are the core technologies in video surveillance systems.

In the existing video surveillance, all the pictures captured by the camera are monitored, and then all the pictures monitored are packaged and compressed and uploaded to the server. However, the current video surveillance methods have the following disadvantages: in the existing video surveillance methods, there is no behavior trajectory tracking of moving objects. If the monitoring picture remains stationary for a long time, these stationary pictures are not what we want to monitor. Screens, but the monitoring device will also save these screens to the server, causing unnecessary waste of storage space.

Referring to FIG. 1, FIG. 1 is a schematic flowchart of a monitoring video processing method according to an embodiment of the present application. As shown in the figure, the method may include:

101: Obtain a video image frame of a monitoring video, and use an inter-frame difference algorithm to determine whether there is a moving object in the current image frame in the video image frame. If a moving object exists in the current image frame, extract the moving object in the current image frame. Outline.

In the embodiment of the present application, an inter-frame difference algorithm is mainly used to detect a moving object in a surveillance video. Specifically, after acquiring the video image frame of the surveillance video, an inter-frame difference algorithm is used to determine whether there is a moving object in the current image frame in the video image frame. If there is no moving object in the current image frame, the current image frame is not detected. For any operation, if there is a moving object in the current image frame, extract the contour of the moving object in the current image frame.

The above-mentioned surveillance video may be a surveillance video that has been temporarily stored in the memory, or may be a surveillance video that the camera is monitoring in real time. When the above-mentioned surveillance video is a surveillance video temporarily stored in the memory, the entire surveillance video is detected for a moving object, and when a moving object is detected, the moving object is tracked. When the surveillance video is a surveillance video that the camera is monitoring in real time, the real-time detection method is used to detect the moving object in the surveillance video, and when the moving object is detected, the moving object is tracked.

The above-mentioned inter-frame difference algorithm is a method for determining a change region in an image by using inter-frame differences that are continuous or separated by a certain number of frames, so as to detect a moving object. Usually, the inter-frame difference algorithm performs a difference operation on two consecutive frames or multiple frames of images, then binarizes and filters the difference images, detects possible motion areas, and detects moving objects in the images.

As an optional implementation manner, a two-frame difference algorithm may be used to detect whether a moving object exists in the foregoing image frame. First, after obtaining the image frame of the monitoring video, the image is pre-processed. Wherein, preprocessing the image frame includes: performing grayscale processing on the image frame. Specifically, two consecutive image frames are extracted from the foregoing image frames, and the current image frame is defined as the kth frame. When extracting the image frames, the kth frame and the k-1th image frame are selected. Then, the two extracted color images are converted into grayscale images. Specifically, formula (1) is used to replace the RGB values in the images, and then the grayscale images of the two selected image frames are obtained.

Y = 0.299 * R + 0.587 * G + 0.114 * B (1);

After obtaining the grayscale images of two image frames, find the absolute value of the grayscale difference between the two frames, then the grayscale difference of the stationary object on the difference image will be small, and the outline of the moving object, especially the moving object Due to the presence of grayscale changes, the presence of moving objects in the monitoring screen can be determined based on the difference image, and the position, contour, and moving path of the moving objects can be roughly calculated. Record the previous image frame as F _k-1 and the current image frame as F _k-1 . The gray values of the corresponding pixel points of the two image frames are recorded as F _k-1 (x, y) and F _k (x, y). , And then subtract (2) the gray values of the corresponding pixel points of the two image frames and take the absolute value to obtain the differential image D _k (x, y) as follows:

D _k (x, y) = | F _k-1 (x, y) -F _k (x, y) | (2);

The binarization threshold T is set, and the pixels are binarized one by one according to the formula (3) to obtain a binarized image R _k . Among them, the point with a gray value of 255 is the foreground (moving object) point, and the point with a gray value of 0 is the background point; the connectivity analysis of the image R _k can finally obtain an image R _k containing a complete moving object .

If all the pixels (x, y) in the above difference image are smaller than the binarization threshold, it is determined that the current image frame has not changed from the previous graphic frame, that is, there is no moving object in the current image frame, so The current image frame is regarded as an invalid image frame, and the current image frame is not saved. If there are pixels in the difference image that are larger than the binarization threshold, it is determined that the current image frame has changed from the previous graphics frame, that is, there is a moving object in the current image frame, so the current image frame is regarded as valid An image frame, and obtain position and contour information of the moving object according to the image R _{k of} the complete moving object.

As another optional implementation manner, a three-frame difference algorithm may be used to detect whether a moving object exists in the foregoing image frame. First, the current image frame is defined as the k-th frame, and three consecutive image frames are selected as k-2, k-1, and k frames, and then preprocessed. The processing process is similar to the two-difference algorithm. Not to go into details.

After obtaining the grayscale images of three consecutive image frames, the absolute values of the grayscale differences of the k-2nd and k-1th image frames and the grayscales of the k-1th and kth image frames are obtained, respectively. The absolute value of the difference is used to obtain two difference images, and then the difference image is binarized to obtain two binarized images. Finally, the two binarized images are ANDed to obtain the final binarized image. The specific operation process is similar to the two-frame difference algorithm, and is not described in detail.

As an optional implementation manner, after obtaining a binarized image through a difference algorithm, the obtained binarized image is usually not always the contour of the moving target due to the interference of noise and slight changes in the background. Do some processing on the image to get the complete area of the moving object. In the embodiment of the present application, a basic morphological algorithm is used to denoise the binary image, and finally a clear binary image of a moving object is obtained. Specifically, there are four basic methods of morphology: corrosion, swelling, open operation, and operation. The above four operations are common knowledge, so they will not be described again.

102: Track the moving trajectory of the moving object according to the contour of the moving object in combination with the Camshift algorithm.

In the implementation of this application, when it is determined that a moving object exists in the surveillance video frame, and the outline of the moving object is extracted, the movement trajectory of the moving object is tracked by using the Camshift algorithm and the outline information of the moving object. Camshift is based on the color information of moving objects in the video image. Mean shift is performed on each frame of the input image, and the target center of the previous frame and the search window size (kernel function bandwidth) are used as the next frame. The center of the algorithm and the initial value of the search window size. Iterating this way, you can track the target. Because the position and size of the search window are set to the position and size of the current center of the moving target before each search, and the moving target is usually near this area, the search time is shortened.

The specific steps for tracking the trajectory of a moving object by using the Camshift algorithm are as follows:

1) Set the entire monitoring screen as the search area;

2) Initialize the size and position of the search window;

3) Calculate the color probability distribution (back projection) in the search window;

4) Run the Meanshift algorithm to get the new size and position of the search window;

5) In the next frame of video image, initialize the position and size of the search window with the values obtained in step (4), skip to step (3) and continue running.

The traditional Camshift algorithm needs to manually select the tracked moving object to track the moving object. In the implementation of this application, the detection result of the inter-frame difference algorithm is used to track a moving object.

In the embodiment of the present application, the tracking the moving trajectory of the moving object according to the contour of the moving object in combination with the Camshift algorithm specifically includes:

1021: Initialize the size and position of the search window in the current image frame according to the contour of the moving object.

In the embodiment of the present application, after detecting the presence of a moving object in the current image frame according to the inter-frame difference algorithm, and obtaining the contour of the moving object according to the binary image of the detection result, the contour of the moving object is obtained according to the contour of the moving object. Circumscribed rectangle and position. Then, the size and position of the search window in the current image frame are initialized according to the circumscribed rectangle and position.

1022: Calculate a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window.

Since the RGB color space is more sensitive to changes in light brightness, in order to reduce the effect of this change on the tracking effect, the image is first converted from RGB space to HSI space. Then make a histogram of the H components, which represents the probability of the occurrence of different H component values or the number of pixels, that is, you can find the probability of the H component size h or the number of pixels, and you get the color probability. Lookup table. By replacing the value of each pixel in the image with the probability of its color appearing, the color probability distribution map is obtained. This process is called back projection. The color probability distribution map is a grayscale image.

In the embodiment of the present application, after the size and position of the search window in the current image frame is initialized, the color probability distribution of moving objects in the search window needs to be calculated in order to obtain the probability distribution map of the moving objects in the search window.

Specifically, a color histogram in the target area is calculated. Usually, the input image is converted to the HSI color space (or a color space similar to HIS), the target area is the initial set search window range, and the hue H component is separated for the hue histogram calculation of the area. In this way, the color histogram of the moving object is obtained, the probability distribution map I (x, y) is normalized, and it is used as a lookup table. Each pixel on the H-channel image is corresponding to its pixel value. Probability instead, get a probabilistic projection map.

As an optional implementation manner, after obtaining the color probability distribution image of the moving object in the search window, since not all pixels of the moving object are included in the search window, noise and interference exist in the color probability distribution image. Therefore, after the color probability distribution map in the search window is obtained, it needs to be denoised. In the implementation of the application, the median filtering downsampling can be used to remove noise from the above probability distribution image.

1023: Calculate the position and size of the centroid of the moving object in the current image frame according to the probability distribution map and the meanshift algorithm.

According to the size and position of the search window, the position of the center of mass of the search window is calculated. Specifically, the zero-order moment and the first-order moment of the pixels in the search window are calculated to find the position of the centroid of the search window. Let (x, y) be the position of a pixel in the search window, and I (x, y) be the pixel value at that pixel (x, y) in the probability distribution map.

Among them, the zero-order moment is calculated by using formula (4);

M ₀₀ = Σ _x Σ _y I (x, y) (4);

First-order moments are calculated using equations (5) and (6);

M ₁₀ = Σ _x Σ _y xI (x, y) (5);

M ₀₁ = Σ _x Σ _y yI (x, y) (6);

The centroid (x _c , y _c ) of the search window is:

x _c = M ₁₀ / M ₀₀ (7);

y _c = M ₀₁ / M ₀₀ (8);

Then adjust the size of the search window;

width:

Length: 1.2s.

Next, move the center of the search window to the center of mass. If the moving distance is greater than the set threshold, recalculate the center of mass of the adjusted window and perform a new round of window position and size adjustment. Until the moving distance between the center of the window and the center of mass is less than the threshold, or the number of iterations reaches a certain maximum, the convergence condition is considered to be satisfied. At this time, the center position and size of the window are the center of mass and position of the moving object in the current frame.

1024: Initialize the size and position of the search window in the next image frame according to the centroid position and size of the moving object in the current image frame, use the next image frame as the current image frame, and trigger the calculation of the search window. The step of obtaining the probability distribution map of the search window by color probability distribution of the moving object.

In the embodiment of the present application, after the position and size of the centroid of the moving object in the current image frame are obtained, the position and size of the centroid of the moving object in the current image frame are saved. Then, the next image frame is used as the current image frame and the step of calculating the color probability distribution of the moving objects in the search window to obtain the probability distribution map of the search window is triggered. Specifically, in the next image frame, using the position and size of the centroid of the moving object in the previous image frame as input to initialize the size and position of the search window in the next image frame, and repeating the above calculation of the centroid in the current image frame The position and size steps, so as to obtain the position and size of the centroid of the moving object in the next image frame, so as to achieve the purpose of tracking the moving object.

103: Save the video image frame in which the moving object exists and the moving track of the moving object.

In the embodiment of the present application, an inter-frame difference algorithm is used to detect whether there is a moving object in the video image frame. When a moving object is detected in the current image frame, the detected moving object is tracked by using the Camshift algorithm. When the moving object disappears from the video image frame, it is determined that the tracking ends. During the process of tracking the moving object or after determining that the tracking is finished, the video image frames in which the moving object exists are stored in the memory or uploaded to the server for storage, and for the video image frames in which there is no moving object, the video image frames are not processed. save.

It can be seen that in the embodiment of the present application, the video image frame of the surveillance video is obtained, and an inter-frame difference algorithm is used to determine whether there is a moving object in the current image frame in the video image frame; if it exists, the movement in the current image frame is extracted The outline of the object. Then, the moving trajectory of the moving object is tracked according to the contour of the moving object in combination with the Camshift algorithm. Finally, the video image frame where the moving object exists and the moving track of the moving object are saved. Through the embodiments of the present application, it is possible to effectively detect and track a moving object video segment in a monitoring video, and only save the monitoring video segment having a moving object in the process of saving the monitoring video, thereby saving storage space. .

Referring to FIG. 2, FIG. 2 is a schematic flowchart of another monitoring video processing method provided by an embodiment of the present application. As shown in the figure, the method may include:

201: Obtain the current image frame and the previous image frame of the surveillance video.

Wherein, the current image frame is defined as the k-th frame, and the previous image frame is defined as the k-1 frame.

202: Calculate a difference image D _k (x, y) between the current image frame and the previous image frame.

Before the difference image D _k (x, y) of the current image frame and the previous frame is obtained, the current image frame and the previous image frame need to be denoised and grayed.

203: Determine whether there is a moving object. If there is a moving object, step 204 is performed. If there is no moving object, return to step 202 to obtain the next image frame.

Specifically, it is determined whether there is an area where the difference image D _k (x, y)> T exists in the difference image. If yes, it is determined that there is a moving object in the k-th image frame, and step 204 is continued. If not, the k-th is determined. If there is no moving object in the frame image frame, the next image frame is continuously acquired, where T is a binarization threshold.

204: Binarize the difference image D _k (x, y) to obtain a binarized image R _k (x, y).

205: Extract the outline and position information of the moving object based on the binarized image.

After obtaining the binarized image R _k (x, y), it is necessary to perform moving object segmentation on the moving region. The obtained binarized image is usually not always the contour of the moving target due to the interference of noise and slight changes in the background. Therefore, some processing is required on the binarized image to obtain the complete area of the moving object. In the embodiment of the present application, a basic morphological algorithm is used to denoise the binary image, and finally a clear binary image of a moving object is obtained. Specifically, there are four basic methods of morphology: corrosion, swelling, open operation, and operation. The above four operations are common knowledge, so they will not be described again.

In the embodiment of the present application, after the above-mentioned binarized image is processed, the contour and position information of the moving object is extracted according to the binarized image after processing.

206: Initialize the size and position of the search window according to the outline and position information of the moving object.

207: Convert the current image frame into a HIS image.

208: Calculate a histogram of the hue H of the moving object in the search window.

209: Back-project the histogram to obtain a probability distribution map I (x, y) of the moving object.

210: Calculate the zero-order moment and the first-order moment of the pixels in the window according to the probability distribution map, the position of the search window, and the size of the search window to obtain the centroid position of the search window.

211: Adjust the center position and size of the search window according to the centroid position.

212: Determine whether to converge according to the meanshift algorithm. If it converges, use the adjusted center position and size of the search window as the position and size of the search window in the next image frame to initialize. If it does not converge, return to step 210.

213: Determine whether the tracking of the moving object is ended, and if the tracking is ended, save the image frame of the moving object and the moving track of the moving object.

It can be seen that in the embodiment of the present application, the video image frame of the monitoring video is obtained, and an inter-frame difference algorithm is used to determine whether there is a moving object in the current image frame in the video image frame; if it exists, the movement in the current image frame is extracted. The outline of the object. Then, the moving trajectory of the moving object is tracked according to the contour of the moving object in combination with the Camshift algorithm. Finally, the video image frame where the moving object exists and the moving track of the moving object are saved. Through the embodiments of the present application, it is possible to effectively detect and track a moving object video segment in a monitoring video, and only save the monitoring video segment having a moving object in the process of saving the monitoring video, thereby saving storage space. .

Referring to FIG. 3, FIG. 3 is a schematic flowchart of another monitoring video processing method according to an embodiment of the present application. As shown in the figure, the method may include:

301: Obtain the current image frame and the first two image frames of the surveillance video.

The current image frame is defined as the k-th image frame, and the first two image frames are the k-1th image frame and the k-2th image frame, respectively.

302: Calculate the difference image Dk1 (x, y) of the current image frame and the previous image frame, and the difference image Dk2 (x, y) of the previous two image frames.

Before obtaining a differential image, the current image frame and the previous image frame need to be denoised and grayed out.

303: Perform an AND operation on the two difference images to obtain a final difference map Dk (x, y).

304: Determine whether there is a moving object. If there is a moving object, step 204 is performed. If there is no moving object, return to step 202 to obtain the next image frame.

Specifically, it is determined whether there is an area of the difference image Dk (x, y)> T in the difference image. If yes, it is determined that there is a moving object in the k-th image frame, and step 204 is continued, and if not, the k-th frame is determined. If there is no moving object in the image frame, the next image frame is obtained, where T is the binarization threshold.

305: Binarize the difference image Dk (x, y) to obtain a binarized image Rk (x, y).

306: Extract contour and position information of the moving object according to the binarized image.

After obtaining the binarized image Rk (x, y), it is necessary to perform moving object segmentation on the moving region. The obtained binarized image is usually not always the contour of the moving target due to the interference of noise and slight changes in the background. Therefore, some processing is required on the binarized image to obtain the complete area of the moving object. In the embodiment of the present application, a basic morphological algorithm is used to denoise the binary image, and finally a clear binary image of a moving object is obtained. Specifically, there are four basic methods of morphology: corrosion, swelling, open operation, and operation. The above four operations are common knowledge, so they will not be described again.

307: Initialize the size and position of the search window according to the outline and position information of the moving object.

308: Convert the current image frame into a HIS image.

309: Calculate a histogram of the hue H of the moving object in the search window.

310: Back-project the histogram to obtain a probability distribution map I (x, y) of the moving object.

311: Calculate the zero-order moment and the first-order moment of the pixels in the window according to the probability distribution map, the position of the search window, and the size of the search window to obtain the centroid position of the search window.

312: Adjust the center position and size of the search window according to the centroid position.

313: Determine whether to converge according to the meanshift algorithm. If it converges, use the center position and size of the adjusted search window as the position and size of the search window in the next image frame to initialize. If it does not converge, return to step 311.

314: Determine whether the tracking of the moving object is ended, and if the tracking is ended, save the image frame of the moving object and the moving track of the moving object.

An embodiment of the present application further provides a monitoring video processing apparatus, where the monitoring video processing apparatus is configured to execute any one of the foregoing methods. Specifically, referring to FIG. 4, FIG. 4 is a schematic block diagram of a monitoring video processing apparatus according to an embodiment of the present application. The monitoring video processing apparatus of this embodiment includes: an obtaining unit 401, a judging unit 402, an extracting unit 403, a tracking unit 404, and a saving unit 405.

The acquiring unit 401 is configured to acquire a video image frame of a surveillance video.

The determining unit 402 is configured to determine whether a moving object exists in a current image frame in the video image frame by using an inter-frame difference algorithm.

The extraction unit 403 is configured to extract a contour of the moving object in the current image frame when the determining unit 402 determines that a moving object exists.

The tracking unit 404 is configured to track a moving trajectory of the moving object according to a contour of the moving object in combination with a Camshift algorithm.

The storage unit 405 is configured to store a video image frame in which the moving object exists and a movement track of the moving object.

As an optional implementation manner, the monitoring video processing device further includes:

The processing unit 406 is configured to pre-process the video image frame, and the pre-processing includes graying processing.

As an optional implementation manner, the foregoing determining unit includes:

The first calculation unit 407 is configured to perform a difference operation between the current image frame and a previous image frame of the current image frame to obtain a difference image.

A determining unit 408 is configured to determine that a moving object exists in an image frame when an area in which a gray value of a pixel is greater than a binarization threshold exists in the difference image.

As an optional implementation manner, the foregoing determining unit includes:

The first calculation unit 407 is configured to perform a difference operation between the current picture frame and the first two image frames of the current picture frame to obtain a difference image.

As an optional implementation manner, the above-mentioned extraction unit 403 includes:

A binarizing unit 409 is configured to binarize the difference image to obtain a binarized image according to the binarization threshold.

The extraction unit is used for 403, and extracts the contour and position of the moving object according to the binary image.

As an optional implementation manner, the processing unit 406 is further configured to perform noise processing on the binary image using a basic morphological algorithm.

As an optional implementation manner, the tracking unit 404 includes:

The initial unit 410 is configured to initialize the size and position of the search window in the current image frame according to the contour of the moving object.

The second calculation unit 411 is configured to calculate a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window.

The second calculation unit 411 is further configured to calculate, according to the probability distribution map and the meanshift algorithm, the position and size of the centroid of the moving object in the current image frame.

The initialization unit 410 is further configured to initialize the size and position of a search window in the next image frame according to the centroid position and size of the moving object in the current image frame, use the next image frame as the current image frame, and trigger The step of calculating a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window.

As an optional implementation manner, the second calculation unit is configured to convert the current image frame from RGB space to HSI space; and calculate a histogram of the current image frame according to the H component in the HIS space of the current image frame Figure; calculating the probability distribution map of the search window according to the histogram.

As an optional implementation manner, the processing unit 406 is further configured to perform desalination processing on the probability distribution map of the search window by using a median filtering method.

Referring to FIG. 5, FIG. 5 is a schematic block diagram of a monitoring video processing apparatus according to another embodiment of the present application. The monitoring video processing apparatus in this embodiment as shown in the figure may include: one or more processors 501; one or more input devices 502, one or more output devices 503, and a memory 504. The processor 501, the input device 502, the output device 503, and the memory 504 are connected through a bus 505. The memory 502 is configured to store a computer program, and the computer program includes program instructions, and the processor 501 is configured to execute the program instructions stored in the memory 502. Wherein, the processor 501 is configured to call the above program instructions for execution: obtaining a video image frame of a surveillance video, and using an inter-frame difference algorithm to determine whether a moving object exists in the current image frame in the video image frame; For moving objects, extract the contours of the moving objects in the current image frame; track the moving trajectories of the moving objects based on the contours of the moving objects in combination with the Camshift algorithm; save the video image frames of the moving objects and the Move track.

It should be understood that, in the embodiment of the present application, the processor 501 may be a central processing unit (CPU), and the processor may also be another general-purpose processor or a digital signal processor (DSP). Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The input device 502 may include a touch panel, a fingerprint sensor (for collecting fingerprint information and orientation information of a user), a microphone, and the like, and the output device 503 may include a display (LCD, etc.), a speaker, and the like.

The memory 504 may include a read-only memory and a random access memory, and provide instructions and data to the processor 501. A portion of the storage 504 may also include non-volatile random access memory. For example, the memory 504 may also store information of a device type.

In specific implementation, the processor 501, input device 502, and output device 503 described in the embodiments of the present application may execute the first, second, and third embodiments of a monitoring video processing method provided by the embodiments of the present application. The implementation manner described in the embodiment may also be implemented as the implementation manner of the monitoring video processing apparatus described in the embodiment of the present application, and details are not described herein again.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The program instructions are implemented by a processor to implement: For video image frames, use an inter-frame difference algorithm to determine whether there are moving objects in the current image frame in the video image frame; if there are moving objects in the current image frame, extract the outline of the moving object in the current image frame; The contour of the moving object is combined with the Camshift algorithm to track the moving trajectory of the moving object; a video image frame in which the moving object is stored and the moving trajectory of the moving object are stored.

The computer-readable storage medium may be an internal storage unit of the monitoring video processing apparatus in any one of the foregoing embodiments, such as a hard disk or a memory of the monitoring video processing apparatus. The computer-readable storage medium may also be an external storage device of the surveillance video processing device, such as a plug-in hard disk, a Smart Media Card (SMC), and a secure digital (Secure Digital, SD) card, flash card, etc. Further, the computer-readable storage medium may further include both an internal storage unit and an external storage device of the monitoring video processing apparatus. The computer-readable storage medium is used to store the computer program and other programs and data required by the monitoring video processing device. The computer-readable storage medium described above may also be used to temporarily store data that has been or will be output.

Those of ordinary skill in the art may realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the hardware and software, Interchangeability. In the above description, the composition and steps of each example have been described generally in terms of functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working processes of the above-mentioned end devices and units can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed monitoring video processing apparatus and method may be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or may be combined. Integration into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, which may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions in the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

When the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application is essentially a part that contributes to the existing technology, or all or part of the technical solution may be embodied in the form of a software product, which is stored in a storage medium Included are several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above method in each embodiment of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .

The above is only a specific implementation of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, and these modifications or replacements should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A monitoring video processing method, comprising:

Acquiring a video image frame of a monitoring video, and determining whether a moving object exists in a current image frame in the video image frame by using an inter-frame difference algorithm;

If there is a moving object in the current image frame, extracting the outline of the moving object in the current image frame;

Track the moving trajectory of the moving object according to the contour of the moving object in combination with the Camshift algorithm;

A video image frame where the moving object exists and a moving track of the moving object are saved.
The method according to claim 1, wherein before the judging whether a moving object exists in a current image frame in the video image frame by using the inter-frame difference algorithm, the method further comprises:

Pre-processing the video image frame, the pre-processing includes graying processing.
The method according to claim 1, wherein the determining whether a moving object exists in a current image frame in the video image frame by using an inter-frame difference algorithm comprises:

Performing a difference operation between the current image frame and a previous image frame of the current image frame;

Judging whether there is an area in the difference image where the gray value of the pixel is greater than the binarization threshold;

If there is an area in the difference image where the gray value of the familiar point is greater than the binarization threshold, it is determined that a moving object exists in the image frame.
The method according to claim 1, wherein the determining whether a moving object exists in a current image frame in the video image frame by using an inter-frame difference algorithm comprises:

Performing a differential operation on the current image frame and the first two image frames of the current image frame to obtain a differential image;

Judging whether there is an area in the difference image where the gray value of the pixel is greater than the binarization threshold;

If there is an area in the difference image where the gray value of the familiar point is greater than the binarization threshold, it is determined that a moving object exists in the image frame.
The method according to any one of claims 3 or 4, wherein the extracting a contour of the moving object in the current image frame comprises:

Binarizing the difference image according to the binarization threshold to obtain a binarized image;

An outline and a position of the moving object are extracted according to the binarized image.
The method according to claim 5, wherein after the binarizing the differential image according to the binarization threshold to obtain a binarized image, the method extracts the image based on the binarized image. Before describing the outline and position of the moving object, the method further includes:

Noise processing is performed on the binarized image using a basic morphological algorithm.
The method according to claim 1, wherein tracking the moving trajectory of the moving object according to the contour of the moving object in combination with the Camshift algorithm comprises:

Initialize the size and position of the search window in the current image frame according to the outline of the moving object;

Calculating a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window;

Obtain the position and size of the centroid of the moving object in the current image frame according to the probability distribution map and the meanshift algorithm;

Initialize the size and position of the search window in the next image frame according to the position and size of the centroid of the moving object in the current image frame, use the next image frame as the current image frame and trigger the calculation of the search window The step of obtaining the probability distribution map of the search window by color probability distribution of the moving object.
The method according to claim 6, wherein the calculating a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window comprises:

Converting the current image frame from RGB space to HSI space;

Calculate a histogram of the current image frame according to the H component in the HIS space of the current image frame;

Calculate a probability distribution map of the search window according to the histogram.
The method according to claim 6, characterized in that, after the color probability distribution of the moving object in the search window is calculated to obtain the probability distribution map of the search window, the method according to the probability distribution map and meanshift Before the algorithm calculates the position and size of the centroid of the moving object in the current image frame, the method further includes:

The median filtering method is used to perform desalination processing on the probability distribution map of the search window.
A monitoring video processing device, comprising:

An obtaining unit for obtaining a video image frame of a surveillance video;

A judging unit, configured to judge whether a moving object exists in a current image frame in the video image frame by using an inter-frame difference algorithm;

An extracting unit, configured to extract a contour of the moving object in the current image frame if the determining unit determines that there is a moving object;

A tracking unit, configured to track a moving trajectory of the moving object according to a contour of the moving object in combination with a Camshift algorithm;

The saving unit is configured to save a video image frame in which the moving object exists and a moving track of the moving object.
The monitoring video processing device according to claim 10, wherein the monitoring video processing device further comprises:

A processing unit, configured to perform preprocessing on the video image frame, where the preprocessing includes graying processing.
The monitoring video processing device according to claim 10, wherein the determining unit comprises:

A first calculation unit, configured to perform a difference operation between the current image frame and a previous image frame of the current image frame to obtain a differential image;

A determining unit, configured to determine that a moving object exists in an image frame when an area where a gray value of a pixel is greater than a binarization threshold exists in the difference image.
The monitoring video processing device according to claim 10, wherein the determining unit comprises:

A first calculation unit, configured to perform a difference operation between the current image frame and the first two image frames of the current image frame to obtain a difference image;

A determining unit, configured to determine that a moving object exists in an image frame when an area where a gray value of a pixel is greater than a binarization threshold exists in the difference image.
The monitoring video processing device according to any one of claims 12 or 13, wherein the extraction unit comprises:

A binarization unit, configured to binarize the difference image to obtain a binarized image according to the binarization threshold;

The extraction unit is configured to extract a contour and a position of the moving object according to the binary image.
The monitoring video processing device according to claim 14, wherein the processing unit is further configured to perform noise processing on the binary image using a basic morphological algorithm.
The monitoring video processing device according to claim 10, wherein the tracking unit comprises:

An initial unit, configured to initialize the size and position of a search window in the current image frame according to the outline of the moving object;

A second calculation unit, configured to calculate a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window;

The second calculation unit is further configured to calculate, according to the probability distribution map and the meanshift algorithm, the position and size of the centroid of the moving object in the current image frame;

The initialization unit is further configured to initialize the size and position of a search window in the next image frame according to the position and size of the centroid of the moving object in the current image frame, and use the next image frame as the current image frame. And a step of calculating a color probability distribution of a moving object in the search window to obtain a probability distribution map of the search window is triggered.
The monitoring video processing device according to claim 16, wherein the second calculation unit is configured to convert the current image frame from RGB space to HSI space; and according to the H component in the HIS space of the current image frame A histogram of the current image frame is calculated; a probability distribution map of the search window is calculated according to the histogram.
The monitoring video processing device according to claim 16, wherein the processing unit is further configured to perform desalination processing on the probability distribution map of the search window by using a median filtering method.
A monitoring video processing device is characterized in that it comprises a processor, an input device, an output device, and a memory. The processor, the input device, the output device, and the memory are connected to each other. The memory is used to store a computer program. The computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1-7.
A computer-readable storage medium, characterized in that the computer storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions, when executed by a processor, cause the processor to execute the program according to claim 1 -7 Any of the methods.