CN117690085A - Video AI analysis system, method and storage medium - Google Patents

Abstract

The present invention relates to the technical field of video surveillance processing, and specifically relates to a video AI analysis system, method and storage medium. This invention obtains the degree of change of each pixel point based on the movement change of each pixel point between multiple monitoring image frames, classifies the pixel points based on the change trend of the degree of change, and obtains the regional category of each pixel point. ;According to the grayscale deviation of each pixel and the grayscale deviation between the area categories, the grayscale impact index of each pixel in each monitoring image frame is obtained, and further combined with the distribution of change degree to obtain each pixel The enhancement degree of the point in each surveillance image frame; based on the enhancement degree of each pixel point in each surveillance image, the enhanced surveillance video is obtained for abnormal analysis. The present invention accurately enhances the video by combining the motion changes and grayscale conditions of the pixels to obtain higher-quality videos, thereby making the results of abnormal analysis of video monitoring more accurate.

Description

A video AI analysis system, method and storage medium

Technical field

The invention relates to the technical field of video surveillance processing, and specifically to a video AI analysis system, method and storage medium.

Background technique

With the development of the Internet, the field of artificial intelligence has emerged rapidly and has been applied in many fields due to its low cost and intelligence. During the transportation of minerals in mineral mining mines, in order to ensure the safety of transportation, the videos collected by on-site monitoring will be analyzed and monitored online to facilitate the detection and early warning of abnormalities during mineral transportation.

Due to the dark environment of mine transportation, existing video surveillance image processing technology generally does not consider the degree of enhancement required for different areas in the mine transportation environment when enhancing the image as a whole, resulting in poor enhancement effects in some areas and even details. The content is lost, so that when the video image is subsequently analyzed, due to the poor enhancement effect of the image frame and the low video quality, it is impossible to accurately analyze the abnormality of video surveillance.

Contents of the invention

In order to solve the technical problem in the prior art that the image frame enhancement effect is poor and the video surveillance abnormality analysis cannot be accurately performed, the purpose of the present invention is to provide a video AI analysis system, method and storage medium. The technical solutions adopted are as follows: :

The present invention provides a video AI analysis method, which method includes:

Obtain more than two surveillance image frames based on mineral transportation surveillance video;

According to the movement change of each pixel point between each two adjacent monitoring image frames, the movement change value of each pixel point between each two adjacent monitoring image frames is obtained; according to all the movement changes corresponding to each pixel point Move the degree of change and overall distribution of the change value to obtain the degree of change of each pixel; classify the pixels according to the distribution trend of the degree of change between all pixels, and obtain the regional category of the pixel;

In each monitoring image frame, based on the deviation degree of the gray value of each pixel point and the gray level difference between the area category where the corresponding pixel point is located and other area categories, the value of each pixel point in the corresponding monitoring image frame is obtained. According to the distribution of the degree of change of the area category where each pixel is located, and the grayscale impact index of the corresponding pixel in each monitoring image frame, the influence index of each pixel in each monitoring image is obtained. The degree of enhancement in the frame;

For each monitoring image frame in the mineral transportation monitoring video, the image is enhanced according to the enhancement degree of the pixel point to obtain the enhanced monitoring video; anomaly analysis is performed through the enhanced monitoring video.

Further, the method for obtaining the degree of change includes:

Each pixel is used as a target pixel in turn, and the average of all movement change values corresponding to the target pixel is used as the overall change index of the target pixel;

Among all the movement change values corresponding to the target pixel point, the difference between the maximum value and the minimum value of the movement change value is taken as the change range of the target pixel point; the sum of the change range of the target pixel point and the preset adjustment parameter is taken as The change floating value of the target pixel point; calculate the product of the change floating value of the target pixel point and the maximum value of the corresponding movement change value of the target pixel point to obtain the change degree index of the target pixel point; the preset adjustment parameter is a positive number;

According to the change degree index and the overall change index of the target pixel, the change degree of the target pixel is obtained; the change degree index and the overall change index are both positively correlated with the change degree.

Further, the method for obtaining the area category includes:

Sort the pixels in order from small to large to obtain a pixel change sequence; use the pixels in the pixel change sequence as points to be segmented, and calculate the degree of change between the point to be segmented and the next adjacent pixel. The difference, after normalization processing, the numerical difference index of the point to be divided is obtained;

Calculate the average of the degree of change of all pixels before one pixel after the point to be divided as the first trend indicator; calculate the average of the degree of change of a preset trend number of pixels after the point to be divided as the second trend indicator; The difference between the first trend indicator and the second trend indicator is used as the trend difference indicator of the point to be divided;

Normalize the product of the numerical difference index and the trend difference index of the point to be divided to obtain the segmentation degree of the point to be divided;

In the pixel change sequence, the pixels are traversed in order. When the division degree of the current pixel is less than the division degree of the adjacent previous pixel, the traversal stops and the current pixel is used as the dividing point, and all pixels before the dividing point are Points are regarded as a regional category and the pixels of the regional category are segmented and removed from the pixel change sequence, and a new pixel change sequence is obtained for iterative segmentation to obtain the regional category; the iteration stops when the segmentation point cannot be obtained, and all remaining pixels are treated as a Regional Category.

Further, the method for obtaining the grayscale influence index includes:

For any surveillance image frame, each pixel in the surveillance image frame is used as a reference pixel in turn; the grayscale difference between the pixel value of the reference pixel and the average grayscale value of the surveillance image frame is calculated, and the reference pixel is obtained. The grayscale deviation in the monitoring image frame;

Calculate the average grayscale value of the pixels corresponding to each regional category in the surveillance image frame, and obtain the regional grayscale value of each regional category in the surveillance image frame; calculate the difference between the regional category where the reference pixel point is located and each other regional category The difference in regional gray value is used to obtain the regional difference of the reference pixel; the average of all regional differences of the reference pixel is used as the regional deviation of the reference pixel in the monitoring image frame;

After negative correlation mapping and normalization are performed on the product of the grayscale deviation of the reference pixel in the monitoring image frame and the regional deviation, the grayscale influence index of the reference pixel in the monitoring image frame is obtained.

Further, the method for obtaining the enhancement degree includes:

The average value of all changes in the area category where the reference pixel is located is used as the change impact index of the reference pixel;

According to the grayscale impact index and change impact index of the reference pixel point in the surveillance image frame, the enhanced influence degree of the reference pixel point in the surveillance image frame is obtained; both the grayscale impact index and the change impact index are positively correlated with the enhanced impact degree. ;

The cumulative value of the enhanced influence of all pixels in the monitoring image frame is used as the sum of the influence of the monitoring image frame; the ratio of the enhanced influence of the reference pixel in the monitoring image frame to the sum of the influence is calculated to obtain the reference pixel The enhancement degree of the point in this surveillance image frame.

Further, the method for obtaining enhanced surveillance video includes:

Each monitoring image frame is regarded as an image frame to be enhanced in turn. For any pixel in the image frame to be enhanced, the product of the enhancement degree and the gray value of the pixel is calculated to obtain the adjustment value of the pixel; The sum of the gray value of the point and the adjustment value is rounded down to obtain the new gray value of the pixel;

The final enhanced image frame is obtained based on the new grayscale values of all pixels in the image frame to be enhanced; all the final enhanced image frames are arranged in time sequence to obtain the enhanced surveillance video.

Further, the abnormal analysis through enhanced surveillance video includes:

Take the enhanced surveillance video as input and input it into the trained neural network to output anomaly detection results.

Further, the method for obtaining the movement change value includes:

Two adjacent surveillance image frames are used as the input of the optical flow method, and the output is the moving distance of each pixel point between the corresponding two adjacent surveillance image frames; the moving distance is used as the moving distance of each pixel point between the corresponding two adjacent adjacent surveillance image frames. Monitor the movement changes between image frames.

The invention also provides a video AI analysis system, including:

The image frame acquisition module is used to acquire more than two monitoring image frames based on the mineral transportation monitoring video;

The area category acquisition module is used to obtain the movement change value of each pixel point between each two adjacent monitoring image frames based on the movement change of each pixel point between each two adjacent monitoring image frames; according to The degree of change and overall distribution of all movement change values corresponding to each pixel point are obtained to obtain the degree of change of each pixel point; the pixel points are classified according to the distribution trend of the degree of change between all pixel points, and the area of the pixel point is obtained category;

The pixel enhancement acquisition module is used to obtain in each monitoring image frame based on the degree of deviation of the gray value of each pixel and the gray difference between the area category where the corresponding pixel is located and other area categories. The grayscale impact index of each pixel in the corresponding monitoring image frame; according to the distribution of the degree of change of the area category where each pixel is located, and the grayscale impact index of the corresponding pixel in each monitoring image frame, we obtain The enhancement degree of each pixel in each surveillance image frame;

The video enhancement analysis module is used to perform image enhancement on each monitoring image frame in the mineral transportation monitoring video based on the enhancement degree of the pixels to obtain the enhanced monitoring video; and conduct abnormal analysis through the enhanced monitoring video.

The present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the program or instructions are executed by the processor, a video AI analysis method in any of the above implementation modes can be implemented. steps in.

The invention has the following beneficial effects:

This invention obtains the degree of change of each pixel by analyzing each monitoring image frame and combining the movement changes of each pixel between monitoring image frames, reflecting the overall movement of each pixel between all monitoring image frames. Changes, classify pixels based on the change trend of movement changes, obtain the regional category of each pixel, monitor the movement of the image frame through the pixels, classify the pixels, and perform different enhancement levels for different areas . Further combine the grayscale deviation of each pixel and the grayscale deviation between the area categories to obtain the grayscale impact index of each pixel in each monitoring image frame, taking into account the blur caused by the small grayscale difference. In this case, the area with smaller contrast needs stronger enhancement. Combining the distribution of change degree and grayscale influence index, we can get the enhancement degree of each pixel in each monitoring image frame. According to the different pixels in each monitoring image, The degree of grayscale difference and motion changes provide a more accurate degree of enhancement for each pixel in each monitoring image. Finally, based on the enhancement degree of each pixel in each surveillance image, the enhanced surveillance video is obtained for abnormal analysis. The present invention accurately enhances the video by combining the motion changes and grayscale conditions of the pixels to obtain higher-quality surveillance videos, thereby making the results of abnormal analysis of the surveillance videos more accurate.

Description of the drawings

In order to more clearly explain the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description The drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of a video AI analysis method provided by an embodiment of the present invention;

Figure 2 is a structural diagram of a video AI analysis system provided by an embodiment of the present invention.

Detailed ways

In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended inventive purpose, the following is a detailed implementation of a video AI analysis system, method and storage medium proposed according to the present invention in conjunction with the drawings and preferred embodiments. The method, structure, characteristics and functions are described in detail below. In the following description, different terms "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Additionally, the specific features, structures, or characteristics of one or more embodiments may be combined in any suitable combination.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which the invention belongs.

The specific solutions of the video AI analysis system, method and storage medium provided by the present invention will be described in detail below with reference to the accompanying drawings.

An example of a video AI analysis method:

Please refer to Figure 1, which shows a flow chart of a video AI analysis method provided by an embodiment of the present invention. The method includes the following steps:

S1: Obtain more than two surveillance image frames based on the mineral transportation surveillance video.

During the mineral mining process, due to abnormal situations that may occur when the mined minerals are transported, such as ore falling or excessive environmental dust, etc., monitoring and analysis is required. The main method is to arrange the minerals next to the transportation line during transportation. The camera collects video, and then transmits the video to the online monitoring system for analysis to detect and provide early warning for abnormal situations. However, when the clarity and brightness of the collected video are poor, it will cause certain interference to the abnormality detection accuracy of the monitoring system, so the collected video needs to be enhanced. Acquire more than two surveillance image frames based on mineral transportation surveillance video.

In one embodiment of the present invention, in order to ensure the integrity of the video analysis, the duration of the analyzed mineral transportation monitoring video is set to 30 seconds. In order to ensure the accuracy of subsequent video analysis, the duration of the mineral transportation monitoring video is set to 30 seconds. Each image frame is enhanced, and each image frame in the collected mineral transportation surveillance video is grayscaled to obtain more than two surveillance image frames. The acquisition frame rate in the mineral transportation monitoring video is set to 30 frames per second. It should be noted that the acquisition of image frames in the video and the image grayscale method are technical means well known to those skilled in the art, and the specific numerical values can be implemented according to the specific It can be adjusted according to the implementation situation and is not restricted here.

At this point, the surveillance image frames in the mineral transportation surveillance video are obtained for enhanced analysis.

S2: According to the movement change of each pixel point between each two adjacent surveillance image frames, obtain the movement change value of each pixel point between each two adjacent surveillance image frames; according to the corresponding movement of each pixel point The change degree and overall distribution of all movement change values are obtained to obtain the change degree of each pixel point; the pixel points are classified according to the distribution change trend of the change degree between all pixel points, and the regional category of the pixel point is obtained.

Since there are moving objects in the continuous image frames monitored during the mineral transportation process, the pixels also have movement changes. The pixels can be preliminarily classified according to the degree of movement, and the moving parts may be more complex than the stationary parts. Blurred, so different areas can be further enhanced to different degrees based on gray value conditions.

First, according to the movement change of each pixel point between each two adjacent monitoring image frames, the movement change value of each pixel point between each two adjacent monitoring image frames is obtained. Preferably, the two adjacent monitoring image frames are Each monitoring image frame is used as the input of the optical flow method. The output is the moving distance of each pixel point between the corresponding two adjacent monitoring image frames. The moving distance is used as the moving distance of each pixel point between the corresponding two adjacent monitoring image frames. The movement change value between. The optical flow method is a motion estimation method based on grayscale changes. It estimates the movement distance of pixels by calculating the grayscale differences and position changes between consecutive frames. It should be noted that the optical flow method is a technical method well known to those skilled in the art and will not be described in detail here. In other embodiments of the present invention, the block matching method can also be used to estimate the moving distance of the pixels, which is not limited here.

The movement change value can reflect the change of the pixel. Each pixel has a movement change value between all two adjacent monitoring image frames. According to the change of the pixel in the continuous monitoring image frames, the overall The degree of change is obtained, that is, the degree of change of each pixel is obtained based on the degree of change and the overall distribution of all movement change values corresponding to each pixel.

Preferably, each pixel is used as a target pixel in turn, and the average of all movement change values corresponding to the target pixel is used as the overall change index of the target pixel, and the overall change index reflects the overall position change of the target pixel. Among all the movement change values corresponding to the target pixel point, the difference between the maximum value and the minimum value of the movement change value is regarded as the change range of the target pixel point. The change range reflects the degree of fluctuation of the position change of the pixel point. Since the pixel point There may be uniform changes, that is, the transport vehicle moves at a uniform speed. The sum of the change range of the target pixel point and the preset adjustment parameters is used as the change floating value of the target pixel point. The value reflecting the movement change is obtained by adjusting the preset adjustment parameters. The floating value of the change in the degree of discreteness. In the embodiment of the present invention, the preset adjustment parameter is a positive number and is set to 0.001. The specific value can be adjusted by the implementer according to the specific implementation situation.

Furthermore, the product of the change floating value of the target pixel point and the maximum value of the corresponding movement change value of the target pixel point is calculated to obtain the change degree index of the target pixel point, and the change degree of the pixel value movement distance is amplified by the maximum value to obtain the reflected change A more obvious indicator of the degree of change. According to the change degree index and the overall change index of the target pixel, the change degree of the target pixel is obtained. The change degree index and the overall change index are both positively correlated with the change degree.

In the embodiment of the present invention, the expression of the degree of change of a pixel is:

In the formula, Q _i represents the degree of change of the i-th pixel, Expressed as the maximum value of the movement change value corresponding to the i-th pixel,/> Expressed as the minimum value of the movement change value corresponding to the i-th pixel,/> Expressed as the i-th pixel corresponding to the j-th movement change value, V _i represents the total number of all movement change values corresponding to the i-th pixel, α represents the preset adjustment parameter, and || represents the absolute value extraction function.

in, Expressed as the change range of the i-th pixel,/> Expressed as the changing floating value of the i-th pixel, /> Expressed as the change degree index of the i-th pixel,/> Expressed as the overall change index of the i-th pixel. The change degree index and the overall change index are reflected in the form of the product and are positively related to the degree of change. When the change degree index and the overall change index are larger, the degree of change is greater, indicating that the position change of the pixel point is both in degree and value. The larger the value, the pixel corresponds to the complex part of the motion, and the position change of the pixel is reflected through the degree of change. In other embodiments of the present invention, other basic mathematical operations may be used to reflect that both the degree of change index and the overall change index are positively correlated with the degree of change, such as addition, etc., which are not limited here.

Therefore, the pixels can be further divided into corresponding regional categories according to the degree of change of each pixel. When the degree of change between pixels is relatively similar, it means that the pixels are more likely to correspond to the same part of the area. The difference in degree of change is also large, that is, the pixels are classified according to the distribution change trend of the degree of change between all pixels, and the regional category of the pixel is obtained.

Preferably, all the degree of change are sorted into pixel points from small to large to obtain a sequence of pixel changes, and segmentation is performed based on changes in the degree of change. The pixel points in the pixel change sequence are used as points to be segmented in turn, and the difference in degree of change between the point to be segmented and the adjacent pixel point is calculated. After normalization processing, the numerical difference index of the point to be segmented is obtained. When the pixel point The numerical difference between the degree of change is large, indicating that the pixels no longer belong to the same category of pixels.

Furthermore, taking into account the change trend of the degree of change between local pixels before and after the pixel point, the possibility of the pixel being a segmentation point is determined based on the overall changes, and the degree of change of all pixels before the pixel after the point to be segmented is calculated. The average value, as the first trend indicator, calculates the average change degree of the preset trend number of pixels after the point to be divided. As the second trend indicator, the difference between the first trend indicator and the second trend indicator is used as the point to be divided. The trend difference index reflects the difference in the degree of change of the pixels left and right of the point to be divided. In the embodiment of the present invention, the preset trend number is 20, and the specific numerical value can be adjusted by the implementer according to the specific implementation situation. The preset trend number is to analyze the trend of the local change degree of each point to be segmented. If the pixels after the point to be segmented are less than the preset trend number, all pixels after the point to be segmented can be analyzed.

The product of the numerical difference index and the trend difference index of the point to be divided is normalized to obtain the division degree of the point to be divided. In the embodiment of the present invention, the expression of the division degree is:

In the formula, P _l represents the segmentation degree of the l-th pixel in the pixel change sequence, Q _l represents the degree of change of the l-th pixel in the pixel change sequence, Q _l+1 represents the l+th pixel in the pixel change sequence The degree of change of 1 pixel, m represents the total number of all pixels before the l+1 pixel in the pixel change sequence, Q _{l+1, u} represents the number before the l+1 pixel in the pixel change sequence The segmentation degree of the u-th pixel, r represents the total number of pixels with a preset trend number after the l-th pixel in the pixel change sequence, Q _{l, v} represents the number of pixels after the l-th pixel in the pixel change sequence. The division degree of v pixels, || is expressed as an absolute value extraction function, exp() is expressed as an exponential function with natural constants as the base, and norm() is expressed as a normalization function. It should be noted that normalization is With technical means well known to those skilled in the art, the selection of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

Among them, exp(|Q _l -Q _l+1 |) represents the numerical difference index of the l-th pixel in the pixel change sequence, Expressed as the first trend indicator of the l-th pixel in the pixel change sequence,/> Expressed as the second trend indicator of the l-th pixel in the pixel change sequence,/> It is expressed as the trend difference index of the l-th pixel in the pixel change sequence. When the numerical difference index and the trend difference index are larger, it means that the change degree of the corresponding pixel points in the left and right distribution of the pixel points in the sequence is relatively different, then The greater the segmentation degree, the more likely the corresponding pixel point is the segmentation point of the category, and the categories are further divided according to the segmentation degree.

In the pixel change sequence, the pixels are traversed in the order of arrangement. Since the degree of change of the pixels is sorted from small to large, and the degree of change of pixels belonging to the same area category is not very different, the segmentation degree is in an increasing distribution, and when When the segmentation degree of the current pixel is smaller than the segmentation degree of the adjacent previous pixel, it means that the possibility of segmentation reaches the maximum, and the difference in change degree before and after reaches the maximum, that is, the previous pixels are of the same category, and the subsequent pixels are For another category, stop traversing and use the current pixel as the segmentation point, treat all pixels before the segmentation point as a region category, segment and remove the pixels of the region category from the pixel change sequence, and obtain a new pixel change sequence. Iterative segmentation is performed to obtain the region category, ensuring that each time a segmentation point is to be analyzed, only the previous segmentation points belonging to one category are analyzed. The iteration stops when the segmentation point cannot be obtained, and all remaining pixels are regarded as a region category.

Complete the regional classification of all pixels. Each regional category represents pixels belonging to different areas in the video. For example, when a transport truck is transporting minerals, the transport vehicle and the ore on the vehicle are constantly moving, and the mine cave Other locations in the image, such as walls, lights, etc., do not move, while dust in the air moves slowly. Therefore, pixels in the area category can represent different areas such as background, dust, transport vehicles, and ores.

S3: In each surveillance image frame, based on the degree of deviation of the gray value of each pixel and the gray difference between the area category where the corresponding pixel is located and other area categories, the corresponding monitoring image of each pixel is obtained. The grayscale influence index in the image frame; according to the distribution of the change degree of the area category where each pixel point is located, and the grayscale influence index of the corresponding pixel point in each monitoring image frame, the grayscale influence index of each pixel point in each monitoring image frame is obtained. Monitor the degree of enhancement in image frames.

After obtaining different regional categories, different pixels can be analyzed based on the motion possibility and grayscale difference of different regional categories to obtain the different enhancement degrees required for each pixel in each monitoring image frame, so that each Frame images can be better enhanced, resulting in better enhanced videos.

First, analyze the grayscale difference. Due to the dim characteristics of the mine transportation and collection environment, the grayscale value difference may not be obvious. Therefore, through the grayscale difference of the area where the pixel is located and the grayscale deviation of the pixel itself Degree, comprehensively analyze the grayscale influence of each pixel in different surveillance image frames, that is, in each surveillance image frame, according to the degree of deviation of the grayscale value of each pixel and the category of the area where the corresponding pixel is located and other area categories, to obtain the grayscale impact index of each pixel in the corresponding monitoring image frame.

Preferably, for any monitoring image frame, each pixel in the monitoring image frame is used as a reference pixel in turn, the same analysis is performed on each pixel in each monitoring image frame, and the pixels of the reference pixel are calculated. The grayscale difference between the value and the average grayscale value of the monitored image frame is used to obtain the grayscale deviation of the reference pixel in the monitored image frame. When the grayscale deviation of the reference pixel is smaller, it means that the grayscale of the reference pixel is The contrast is less obvious.

Further, the average gray value corresponding to each area category in the monitoring image frame is calculated as the regional gray level value of each area category under the monitoring image frame, and the difference between the area category where the reference pixel point is located and each other area category is calculated. The difference in regional gray value between regions is used to obtain the regional difference of the reference pixel. Through the gray difference between the regional categories, the overall gray contrast of the area where the reference pixel is located is reflected, and the average of all regional differences of the reference pixel is The value is used as the regional deviation of the reference pixel. When the regional deviation is smaller, it means that the grayscale contrast in the area where the reference pixel is located is less obvious.

After negative correlation mapping and normalization are performed on the product of the grayscale deviation of the reference pixel point and the regional deviation, the grayscale influence index of the reference pixel point in the monitoring image frame is obtained. In the embodiment of the present invention, the expression of the grayscale influence index is:

In the formula, B _ja represents the grayscale influence index of the a-th pixel in the j-th surveillance image frame, H _ja represents the grayscale value of the a-th pixel in the j-th surveillance image frame, Expressed as the average gray value in the j-th surveillance image frame,/> It is expressed as the regional gray value of the area category where the a-th pixel is located in the j-th surveillance image frame, and s is expressed as the regional gray value of other area categories except the area category where the a-th pixel is located in the j-th surveillance image frame. Quantity,/> It is expressed as the regional gray value of the c-th other area category in the j-th surveillance image frame, || is expressed as the absolute value extraction function, and exp() is expressed as the exponential function with the natural constant as the base.

in, Expressed as the grayscale deviation of the a-th pixel in the j-th surveillance image frame, Expressed as the regional difference between the area category of the a-th pixel in the j-th surveillance image frame and the corresponding c-th other area category, /> Expressed as the regional deviation of the a-th pixel in the j-th surveillance image frame. When the difference between the grayscale deviation and the regional deviation is smaller, it means that the grayscale difference between the pixel point and other areas is small, its grayscale contrast is small, and the regional representation is blurred, then the grayscale impact index The larger the value, the greater the degree of enhancement required for the pixel.

Then, combined with the distribution of degree of change between area categories, the enhancement degree that needs to be enhanced is obtained for each pixel. The greater the degree of change, the more likely the part represented by the pixel is a moving area. For moving areas The part is the area where the situation needs to be analyzed more, so the greater the degree of enhancement is needed. Therefore, according to the distribution of the degree of change of the area category where each pixel is located, and the grayscale of the corresponding pixel in each monitoring image frame Impact index to obtain the enhancement degree of each pixel in each monitoring image frame.

Preferably, the average value of all degree of change in the region category where the reference pixel point is located is used as the change impact index of the reference pixel point, and the average value reflects the distribution of the overall degree of change in the region category. According to the gray scale influence index and change influence index of the reference pixel point in the monitoring image frame, the enhanced influence degree of the reference pixel point in the monitoring image frame is obtained. The enhanced influence degree representing the pixel point is obtained through the analysis of gray scale and change degree. The degree of enhancement, the grayscale impact index and the change impact index are all positively correlated with the enhancement impact degree. In this embodiment of the present invention, the expression for enhancing influence is:

In the formula, W _ja represents the enhanced influence degree of the a-th pixel in the j-th surveillance image frame, and B _ja represents the grayscale influence index of the a-th pixel in the j-th surveillance image frame. Expressed as the change impact index of the a-th pixel in the j-th monitoring image frame.

It is reflected in the form of product that both the grayscale impact index and the change impact index are positively correlated with the enhanced influence degree. In other embodiments of the present invention, other basic mathematical operations can also be used to reflect that the grayscale impact index and the change impact index are both positively correlated with the enhanced impact degree. Positive correlation, such as addition or power operation, etc., is not limited here.

Further, the cumulative value of the enhanced influence of all pixels in the monitoring image frame is used as the influence sum of the monitoring image frame, and the ratio of the enhanced influence of the reference pixel in the monitoring image frame to the influence sum is calculated, The enhancement degree of the reference pixel point in the monitoring image frame is obtained, and the optimal enhancement situation of the pixel point under each monitoring image frame is obtained through the proportion of the pixel point in the corresponding monitoring image frame. In the embodiment of the present invention, The expression of enhancement degree is:

In the formula, R _ja represents the enhancement degree of the a-th pixel in the j-th surveillance image frame, W _ja represents the enhancement influence degree of the a-th pixel in the j-th surveillance image frame, and z represents the j-th pixel. Monitor the total number of pixels in the image frame.

in, Expressed as the influence and value of the jth surveillance image frame, when the proportion of pixels is larger, it means that the grayscale change of the pixels in the surveillance image frame is not obvious, and the more likely it is to correspond to the moving area, so in this surveillance A stronger degree of enhancement is needed in the image frame.

At this point, the analysis of the grayscale and motion changes of each pixel in each monitoring image frame is completed, and the enhancement degree of each pixel in each monitoring image frame is obtained.

S4: For each monitoring image frame in the mineral transportation monitoring video, perform image enhancement based on the enhancement degree of the pixels to obtain an enhanced monitoring video; perform anomaly analysis through the enhanced monitoring video.

Each monitored image frame is enhanced by the enhancement degree. In one embodiment of the present invention, each monitored image frame is sequentially regarded as an image frame to be enhanced. For any pixel in the image frame to be enhanced, the pixel is calculated. The product of the enhancement degree and the gray value is used to obtain the adjustment value of the pixel. The sum of the gray value and the adjustment value of the pixel is rounded down to obtain a new gray value of the pixel. In the implementation of the present invention In the example, the calculation expression of the new gray value of the pixel is:

In the formula, H' _ja represents the new gray value of the a-th pixel in the j-th surveillance image frame, H _ja represents the gray-scale value of the a-th pixel in the j-th surveillance image frame, R _ja Expressed as the enhancement degree of the a-th pixel in the j-th surveillance image frame, Represented as a rounding down function.

Among them _{, Hja} Large, while improving the grayscale contrast in different areas, it also improves the overall brightness of the image.

Further, the final enhanced image frame is obtained based on the new grayscale values of all pixels in the image frame to be enhanced, and all the final enhanced image frames are arranged in chronological order, that is, the order of each frame during acquisition, to obtain the enhanced surveillance video. Further analysis can be performed based on the enhanced surveillance video, and more accurate analysis results can be obtained, that is, abnormality analysis can be performed through the enhanced surveillance video.

Preferably, the enhanced surveillance video is used as input, inputted into the trained neural network, and anomaly detection results are output. In the embodiment of the present invention, the neural network can choose the convolutional neural network CNN in deep learning, and is trained through the sample set of normal transportation videos and abnormal transportation videos prepared in advance. The training process mainly includes forward propagation and back propagation. Two steps, forward propagation is to pass the input frame through the CNN network to obtain the prediction result, and back propagation is to update the network parameters based on the error between the prediction result and the real result. The trained CNN model is obtained through multiple iterations, and can output abnormality detection results based on the characteristics in the transportation video. The abnormality detection results can include abnormality types, such as people falling or transport vehicle failures, and abnormal locations, such as where the abnormality occurs. Positioning, etc., abnormal classification, that is, normal or abnormal level classification, etc. It should be noted that the role of neural networks is mainly used to classify, analyze and process videos. CNN neural networks are technical means well known to those skilled in the art. There are many convolutional neural network structures to achieve this task, such as LSTM, etc. Specifically, The neural network structure and training process will not be described in detail here.

In summary, the present invention obtains the change degree of each pixel point by analyzing each monitoring image frame and combining the movement changes of each pixel point between monitoring image frames, reflecting the change degree of each pixel point between all monitoring image frames. The overall movement changes, classify the pixels based on the change trend of movement changes, obtain the regional category of each pixel, monitor the movement of the image frame through the pixels, classify the pixels, so that different areas can be targeted Different levels of enhancement. Further combine the grayscale deviation of each pixel and the grayscale deviation between the areas where it is located to obtain the grayscale impact index of each pixel in each monitoring image frame, taking into account the blur caused by small grayscale differences. , the area with smaller contrast needs stronger enhancement. Combining the distribution of change degree and grayscale influence index, the enhancement degree of each pixel in each monitoring image frame is obtained. For the different grayscale of pixels in each monitoring image, degree of difference and motion changes to obtain a more accurate degree of enhancement for each pixel in each monitoring image. Finally, based on the enhancement degree of each pixel in each surveillance image, the enhanced surveillance video is obtained for abnormal analysis. The present invention accurately enhances the video by combining the motion changes and grayscale conditions of the pixels to obtain higher-quality surveillance videos, thereby making the results of abnormal analysis of the surveillance videos more accurate.

Please refer to Figure 2, which shows a structural diagram of a video AI analysis system provided by an embodiment of the present invention. The system includes: an image frame acquisition module 101, a region category acquisition module 102, a pixel enhancement acquisition module 103, a video Enhanced analysis module 104.

The image frame acquisition module 101 is used to acquire more than two monitoring image frames based on the mineral transportation monitoring video;

The area category acquisition module 102 is used to obtain the movement change value of each pixel point between each two adjacent monitoring image frames based on the movement change of each pixel point between each two adjacent monitoring image frames; According to the degree of change and overall distribution of all movement change values corresponding to each pixel, the degree of change of each pixel is obtained; the pixels are classified according to the distribution trend of the degree of change between all pixels, and the degree of change of the pixel is obtained Regional Category;

The pixel enhancement acquisition module 103 is used to, in each monitoring image frame, based on the degree of deviation of the gray value of each pixel and the gray difference between the area category where the corresponding pixel is located and other area categories, Obtain the grayscale impact index of each pixel in the corresponding monitoring image frame; according to the distribution of the degree of change of the area category where each pixel is located, and the grayscale impact index of the corresponding pixel in each monitoring image frame, Obtain the enhancement degree of each pixel in each monitoring image frame;

The video enhancement analysis module 104 is used to perform image enhancement on each monitoring image frame in the mineral transportation monitoring video according to the enhancement degree of the pixel point to obtain an enhanced monitoring video; and perform anomaly analysis through the enhanced monitoring video.

It should be noted that the corresponding video AI analysis system provided by the above embodiment can implement the technical solution described in the embodiment of the above video AI analysis method. The specific implementation principles of each of the above modules or units can be found in the above video AI analysis system. The corresponding content in the embodiment of the AI analysis method will not be described again here.

The present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the program or instructions are executed by the processor, a video AI analysis method in any of the above implementation modes can be implemented. steps in.

It should be noted that the above-mentioned order of the embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the figures do not necessarily require the specific order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain implementations.

Each embodiment in this specification is described in a progressive manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments.

Claims (10)

1. A video AI analysis method, characterized in that the method includes: Obtain more than two surveillance image frames based on mineral transportation surveillance video; According to the movement change of each pixel point between each two adjacent monitoring image frames, the movement change value of each pixel point between each two adjacent monitoring image frames is obtained; according to all the movement changes corresponding to each pixel point Move the degree of change and overall distribution of the change value to obtain the degree of change of each pixel; classify the pixels according to the distribution trend of the degree of change between all pixels, and obtain the regional category of the pixel; In each monitoring image frame, based on the deviation degree of the gray value of each pixel point and the gray level difference between the area category where the corresponding pixel point is located and other area categories, the value of each pixel point in the corresponding monitoring image frame is obtained. According to the distribution of the degree of change of the area category where each pixel is located, and the grayscale impact index of the corresponding pixel in each monitoring image frame, the influence index of each pixel in each monitoring image is obtained. The degree of enhancement in the frame; For each monitoring image frame in the mineral transportation monitoring video, the image is enhanced according to the enhancement degree of the pixel point to obtain the enhanced monitoring video; anomaly analysis is performed through the enhanced monitoring video. 2. A video AI analysis method according to claim 1, characterized in that the method for obtaining the degree of change includes: Each pixel is used as a target pixel in turn, and the average of all movement change values corresponding to the target pixel is used as the overall change index of the target pixel; Among all the movement change values corresponding to the target pixel point, the difference between the maximum value and the minimum value of the movement change value is taken as the change range of the target pixel point; the sum of the change range of the target pixel point and the preset adjustment parameter is taken as The change floating value of the target pixel point; calculate the product of the change floating value of the target pixel point and the maximum value of the corresponding movement change value of the target pixel point to obtain the change degree index of the target pixel point; the preset adjustment parameter is a positive number; According to the change degree index and the overall change index of the target pixel, the change degree of the target pixel is obtained; the change degree index and the overall change index are both positively correlated with the change degree. 3. A video AI analysis method according to claim 1, characterized in that the acquisition method of the area category includes: Sort the pixels in order from small to large to obtain a pixel change sequence; use the pixels in the pixel change sequence as points to be segmented, and calculate the degree of change between the point to be segmented and the next adjacent pixel. The difference, after normalization processing, the numerical difference index of the point to be divided is obtained; Calculate the average of the degree of change of all pixels before one pixel after the point to be divided as the first trend indicator; calculate the average of the degree of change of a preset trend number of pixels after the point to be divided as the second trend indicator; The difference between the first trend indicator and the second trend indicator is used as the trend difference indicator of the point to be divided; Normalize the product of the numerical difference index and the trend difference index of the point to be divided to obtain the segmentation degree of the point to be divided; In the pixel change sequence, the pixels are traversed in order. When the division degree of the current pixel is less than the division degree of the adjacent previous pixel, the traversal stops and the current pixel is used as the dividing point, and all pixels before the dividing point are Points are regarded as a regional category and the pixels of the regional category are segmented and removed from the pixel change sequence, and a new pixel change sequence is obtained for iterative segmentation to obtain the regional category; the iteration stops when the segmentation point cannot be obtained, and all remaining pixels are treated as a Regional Category. 4. A video AI analysis method according to claim 1, characterized in that the method for obtaining the grayscale impact index includes: For any surveillance image frame, each pixel point in the surveillance image frame is used as a reference pixel point in turn; the grayscale difference between the pixel value of the reference pixel point and the average grayscale value of the surveillance image frame is calculated to obtain the reference pixel point in The grayscale deviation in the monitoring image frame; Calculate the average grayscale value of the pixels corresponding to each regional category in the surveillance image frame, and obtain the regional grayscale value of each regional category in the surveillance image frame; calculate the difference between the regional category where the reference pixel point is located and each other regional category The difference in regional gray value is used to obtain the regional difference of the reference pixel; the average of all regional differences of the reference pixel is used as the regional deviation of the reference pixel in the monitoring image frame; After negative correlation mapping and normalization are performed on the product of the grayscale deviation of the reference pixel in the monitoring image frame and the regional deviation, the grayscale influence index of the reference pixel in the monitoring image frame is obtained. 5. A video AI analysis method according to claim 4, characterized in that the method for obtaining the enhancement degree includes: The average value of all changes in the area category where the reference pixel is located is used as the change impact index of the reference pixel; According to the grayscale impact index and change impact index of the reference pixel point in the surveillance image frame, the enhanced influence degree of the reference pixel point in the surveillance image frame is obtained; both the grayscale impact index and the change impact index are positively correlated with the enhanced impact degree. ; The cumulative value of the enhanced influence of all pixels in the monitoring image frame is used as the sum of the influence of the monitoring image frame; the ratio of the enhanced influence of the reference pixel in the monitoring image frame to the sum of the influence is calculated to obtain the reference pixel The enhancement degree of the point in this surveillance image frame. 6. A video AI analysis method according to claim 1, characterized in that the acquisition method of enhanced surveillance video includes: Each monitoring image frame is regarded as an image frame to be enhanced in turn. For any pixel in the image frame to be enhanced, the product of the enhancement degree and the gray value of the pixel is calculated to obtain the adjustment value of the pixel; The sum of the gray value of the point and the adjustment value is rounded down to obtain the new gray value of the pixel; The final enhanced image frame is obtained based on the new grayscale values of all pixels in the image frame to be enhanced; all the final enhanced image frames are arranged in time sequence to obtain the enhanced surveillance video. 7. A video AI analysis method according to claim 1, characterized in that the abnormality analysis through enhanced surveillance video includes: Take the enhanced surveillance video as input and input it into the trained neural network to output anomaly detection results. 8. A video AI analysis method according to claim 1, characterized in that the method for obtaining the movement change value includes: Two adjacent surveillance image frames are used as the input of the optical flow method, and the output is the moving distance of each pixel point between the corresponding two adjacent surveillance image frames; the moving distance is used as the moving distance of each pixel point between the corresponding two adjacent adjacent surveillance image frames. Monitor the movement changes between image frames. 9. A video AI analysis system, characterized by including: The image frame acquisition module is used to acquire more than two monitoring image frames based on the mineral transportation monitoring video; The area category acquisition module is used to obtain the movement change value of each pixel point between each two adjacent monitoring image frames based on the movement change of each pixel point between each two adjacent monitoring image frames; according to The degree of change and overall distribution of all movement change values corresponding to each pixel point are obtained to obtain the degree of change of each pixel point; the pixel points are classified according to the distribution trend of the degree of change between all pixel points, and the area of the pixel point is obtained category; The pixel enhancement acquisition module is used to obtain in each monitoring image frame based on the degree of deviation of the gray value of each pixel and the gray difference between the area category where the corresponding pixel is located and other area categories. The grayscale impact index of each pixel in the corresponding monitoring image frame; according to the distribution of the degree of change of the area category where each pixel is located, and the grayscale impact index of the corresponding pixel in each monitoring image frame, we obtain The enhancement degree of each pixel in each surveillance image frame; The video enhancement analysis module is used to perform image enhancement on each monitoring image frame in the mineral transportation monitoring video based on the enhancement degree of the pixels to obtain the enhanced monitoring video; and conduct abnormal analysis through the enhanced monitoring video. 10. A computer-readable storage medium, characterized in that it is used to store computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can realize any one of the above claims 1-8. Describe the steps in a video AI analysis method.