CN114820698A - Formation detection method and device for large-scale movable motion matrix - Google Patents
Formation detection method and device for large-scale movable motion matrix Download PDFInfo
- Publication number
- CN114820698A CN114820698A CN202210314674.3A CN202210314674A CN114820698A CN 114820698 A CN114820698 A CN 114820698A CN 202210314674 A CN202210314674 A CN 202210314674A CN 114820698 A CN114820698 A CN 114820698A
- Authority
- CN
- China
- Prior art keywords
- target
- image
- target video
- video frame
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 114
- 230000033001 locomotion Effects 0.000 title claims abstract description 87
- 238000001514 detection method Methods 0.000 title claims abstract description 69
- 230000015572 biosynthetic process Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 claims description 53
- 230000000694 effects Effects 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 14
- 230000009467 reduction Effects 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 5
- 230000003628 erosive effect Effects 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 abstract description 7
- 238000012937 correction Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000005286 illumination Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000003491 array Methods 0.000 description 4
- 230000007797 corrosion Effects 0.000 description 4
- 238000005260 corrosion Methods 0.000 description 4
- 230000004075 alteration Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 238000011897 real-time detection Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a formation detection method and a device of a large-scale movable movement matrix, wherein the formation detection method of the large-scale movable movement matrix comprises the following steps: acquiring a target video of a target motion matrix in a target environment; generating a jitter factor corresponding to a target video frame based on the target video frame in a target video and a diagonal gray projection standard array; and under the condition that the jitter factor is lower than a target jitter threshold value, generating a target image of the target motion matrix based on the target video frame. According to the formation detection method of the large-scale movable motion matrix, the jitter factor is generated based on the target video frame in the target video and the diagonal gray level projection standard array, the target video frame is screened based on the jitter factor so as to keep the target video frame with lower jitter amplitude, the target video frame with lower jitter amplitude is processed, and the target image of the target motion matrix is generated, so that the imaging definition is effectively improved, and the detection result is improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a formation detection method and a formation detection device for a large-scale moving matrix.
Background
The motion matrix real-time detection based on video images in large-scale activities is mainly used for capturing video images through a panoramic camera erected at a higher position in an outdoor place and detecting the motion matrix formation with a larger area in real time. However, in the related art, the shooting device (especially outdoors) is easily shaken by strong wind or shaking of the support frame, which causes problems of shaking and ghosting of video images, and causes position deviation and image distortion of objects in the images, which affects the definition of the images, thereby affecting the detection result.
Disclosure of Invention
The invention provides a formation detection method and a formation detection device for a large-scale moving square matrix, which are used for solving the defect of poor detection effect of the moving square matrix in the prior art and improving the detection effect.
The invention provides a formation detection method of a large-scale movable motion matrix, which comprises the following steps:
acquiring a target video of a target motion matrix in a target environment;
generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array;
and under the condition that the jitter factor is lower than a target jitter threshold value, generating a target image of the target motion matrix based on the target video frame.
According to the formation detection method of the large-scale moving matrix provided by the invention, the generating of the jitter factor corresponding to the target video frame based on the target video frame in the target video and the diagonal gray level projection standard array comprises the following steps:
generating a first diagonal gray projection array of a target region of the target video frame based on the target region, the target region not including features of the motion matrix;
generating the dithering factor based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array.
According to the formation detection method of the large-scale moving square matrix provided by the invention, the generating of the jitter factor based on the first diagonal gray projection array and the diagonal gray projection standard array comprises the following steps:
applying the formula:
generating the dithering factor, wherein θ is the dithering factor, Gr _ bi (m) is a diagonal grayscale projection standard value of an mth position in the diagonal grayscale projection standard array, Gr _ ci (m) is a first diagonal grayscale projection value of the mth position in the first diagonal grayscale projection standard array, and d is a side length of the target area.
According to the formation detection method of the large-scale moving matrix provided by the invention, the generation of the target image of the target moving matrix based on the target video frame comprises the following steps:
performing frame difference operation on the target video frame and the target background model to generate a first image;
carrying out binarization processing and noise reduction processing on the first image to generate a third image;
and extracting a foreground image in the target video frame based on the third image, and generating a target image of the target motion matrix based on the foreground image.
According to the formation detection method of the large-scale moving matrix provided by the invention, the generation of the target image of the target moving matrix based on the foreground image comprises the following steps:
converting the foreground image into a target format to generate a fourth image;
and carrying out target shadow threshold-based shadow region segmentation processing and erosion expansion processing on the fourth image to generate a target image of the target motion matrix.
According to the formation detection method of the large-scale moving matrix provided by the invention, before the target video of the target moving matrix in the target environment is obtained, the method further comprises the following steps:
acquiring a multi-frame environment background image in a first target time period under the target environment;
generating a target background model based on the multi-frame environment background image;
and generating a diagonal gray projection standard array corresponding to the target background model based on the target background model.
The invention also provides a formation detection device of the large-scale movable motion matrix, which comprises:
the first acquisition module is used for acquiring a target video of a target motion matrix in a target environment;
the first generation module is used for generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array;
and the second generation module is used for generating a target image of the target motion matrix based on the target video frame under the condition that the jitter factor is lower than a target jitter threshold value.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein when the processor executes the program, the steps of the formation detection method of the large-scale moving square matrix are realized.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of formation detection for a sports square of a large activity as described in any of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a method for formation detection of a square matrix of movements for a large activity as described in any of the above.
The method and the device for detecting the formation of the large-scale moving square matrix generate the jitter factor based on the target video frame in the target video and the diagonal gray level projection standard array, screen the target video frame based on the jitter factor to keep the target video frame with lower jitter amplitude, process the target video frame with lower jitter amplitude to generate the target image of the target moving square matrix, overcome the defect that other gray level projection methods are easily influenced by local motion, realize effective jitter detection on the target video frame in a moving state, remarkably improve the rapidity and the real-time performance of the video frame jitter detection, and improve the imaging effect of the target image, thereby being beneficial to improving the accuracy and the precision of the detection result.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a formation detection method for a motion matrix of a large-scale activity according to the present invention;
FIG. 2 is a second schematic flow chart of the method for detecting formation of a sports square matrix for large activities according to the present invention;
FIG. 3 is a schematic diagram of a method for detecting formation of a large-scale moving matrix according to the present invention;
FIG. 4 is a second schematic diagram of the formation detection method of the exercise matrix of the large-scale activity according to the present invention;
FIG. 5 is a third schematic diagram of the formation detection method of the motion matrix of the large-scale activity provided by the present invention;
FIG. 6 is a fourth schematic diagram of the formation detection method of the motion matrix of the large-scale activity provided by the present invention;
FIG. 7 is a fifth schematic diagram of the formation detection method of the motion matrix of the large-scale activity provided by the present invention;
FIG. 8 is a sixth schematic diagram illustrating the formation detection method of the motion matrix of the large-scale activity according to the present invention;
FIG. 9 is a schematic structural diagram of a formation detection device of a large-scale moving matrix provided by the invention;
fig. 10 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The formation detection method of the large-scale active motion matrix of the present invention is described below with reference to fig. 1 to 6.
The execution main body of the formation detection method of the large-scale moving square matrix can be a server or a terminal of a user, such as a mobile phone or a computer.
As shown in fig. 1, the formation detection method of the motion matrix of the large-scale activity includes: step 110, step 120 and step 130.
in this step, the target environment is the scene in which the target motion matrix is located.
The target motion matrix is a matrix which needs to be detected in a plurality of motion matrixes.
The target video is a real-time video stream including a target motion matrix, and the target video may include multiple frames of images including the target motion matrix.
For example, in a sporting event, the target environment is the environment of a sports field, and the sports square is the sports square formed by players.
It will be appreciated that the motion matrix may be in motion.
The target video may be collected by an image sensor, where the image sensor may be a panoramic camera or other image sensor, which is not limited in this application.
In the actual implementation process, the image sensor needs to be erected at a higher position to ensure that the image of the whole target motion matrix can be acquired.
In some embodiments, after step 110, the method may further comprise: and carrying out distortion correction on each frame of video frame of the target video.
In this embodiment, the aberration correction includes radial aberration and tangential aberration.
Can be determined by the formula:
The radial distortion correction is carried out, and the radial distortion correction,
wherein, (x ', y') represents the distorted pixel coordinates of each frame of video frame in the target video, (x, y) represents the pixel coordinates after distortion correction, and k1, k2 and k3 are the distortion parameters calibrated by the image sensor.
Can be determined by the formula:
The tangential distortion correction is carried out, and the tangential distortion correction,
wherein, (x ', y') represent the distorted pixel coordinates of each frame of video frame in the target video, (x, y) represent the pixel coordinates after distortion correction, and p1 and p3 are distortion parameters obtained by calibration of the image sensor.
It should be noted that the distortion parameter can be obtained by calibrating the image sensor.
In actual implementation, before step 110, the image sensor may be preferentially calibrated to obtain distortion parameters and an internal reference matrix of the image sensor.
In the embodiment, the distortion correction is performed on the target video, so that the picture distortion caused in the process of using the panoramic camera to acquire images can be avoided, and the imaging effect and the imaging authenticity can be improved.
After the target video is obtained, the target video can be stored in a local server or a cloud server and can be called when needed.
Step 120, generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and the diagonal gray projection standard array;
in this step, the target video frame is a video frame to be detected.
The target video frame may be any one of the image frames in the target video.
The diagonal gray projection standard array is a diagonal gray projection value under the condition that the image is not shaken, namely an initial value of the diagonal gray projection value.
The diagonal gray projection standard array is a numerical value determined in advance before detection, and the determination steps of the diagonal gray projection standard array in the subsequent embodiment of the invention are explained in detail, which is not repeated herein at first.
The dithering factor is used to characterize the degree of dithering of the target video.
It will be appreciated that different video frames in the same target video may have different levels of jitter, subject to external environmental factors.
In this embodiment, based on the target video frame in the target video and the diagonal gray projection standard array, the jitter factor corresponding to the target video frame may be determined.
The following describes a specific implementation of this step.
In some embodiments, step 120 may include:
generating a first diagonal gray projection array of a target area based on the target area of the target video frame, wherein the target area does not comprise the characteristics of a motion matrix;
and generating a dithering factor based on the first diagonal gray level projection array and the diagonal gray level projection standard array.
In this embodiment, the target area is any area except for the motion matrix area in the target video frame, and the number of the target areas may be one or more.
The target area may be user-defined.
For example, the target area may be set as square areas at four corners of the target video frame, each square area having a side length of d; or setting the target area as a square area on any two corners of the target video frame; or the target area is set as a square area at any corner of the target video frame.
By selecting square areas on four corners of the target video frame, the influence on the shake detection caused by the movement of the movement matrix with larger volume at the middle position of the video frame can be avoided.
The first diagonal gray projection array is an actual diagonal gray projection value corresponding to a target video frame in the target video.
The first diagonal gray projection arrays may be represented by Gr, where the number of the first diagonal gray projection arrays coincides with the number of target regions.
For example, in the case where square areas at the four corners of the target video frame are determined as target areas, first diagonal gray scale projection arrays of the four target areas of the target video frame are calculated and respectively denoted by Gr _ c1, Gr _ c2, Gr _ c3, and Gr _ c 4. The Gr _ c1 is used for representing a diagonal gray projection array in the upper left corner region of the target video, the Gr _ c2 is used for representing a diagonal gray projection array in the upper right corner region of the target video, the Gr _ c3 is used for representing a diagonal gray projection array in the lower right corner region of the target video, and the Gr _ c4 is used for representing a diagonal gray projection array in the lower left corner region of the target video.
And then respectively calculating weighted difference square factors between the first diagonal gray level projection array and the diagonal gray level projection standard array of the four target areas of the current target video frame.
In some embodiments, the maximum of the weighted difference square factors for the four target regions may be selected as the dithering factor θ to reflect the truest shaking situation.
That is to say that the first and second electrodes,
θ max ＝max(θ 1 ,θ 2 ,θ 3 ,θ 4 )；
wherein, theta max Is a dithering factor, theta 1 ,θ 2 ,θ 3 ,θ 4 And the weighted difference square factors corresponding to the four target areas respectively.
It can be understood that, when the diagonal grayscale projection standard arrays are multiple sets, the first diagonal grayscale projection array and each diagonal grayscale projection standard array are respectively calculated, and the maximum value of the obtained multiple dithering factors is used as the final dithering factor.
For example, in the case that the diagonal grayscale projection standard array is 1000 sets, the first diagonal grayscale projection array and the 1000 sets are calculated for the diagonal grayscale projection standard array, respectively, and the maximum value is selected as the dithering factor.
In some embodiments, generating the dithering factor based on the first diagonal grayscale projection array and the diagonal grayscale projection criteria array includes:
applying the formula:
generating a dithering factor, where θ is the dithering factor, Gr _ bi (m) is a diagonal grayscale projection standard value of the mth position in the diagonal grayscale projection standard array corresponding to the target area i, Gr _ ci (m) is a first diagonal grayscale projection value of the mth position in the first diagonal grayscale projection standard array corresponding to the target area i, d is the side length of the target area, "? And m is not more than d-1:2 d-m-1' and is used for determining the value of m at the position of the denominator in the formula, and the specific steps are as follows: when m is less than or equal to d-1, m in the denominator m +1 is d-1; otherwise, m in the denominator "m + 1" is 2 d-m-1.
After the dithering factor is obtained, step 130 may be performed.
In the research and development process, the inventor finds that, in the related art, a method based on feature point matching and a method based on optical flow detect jitter, but both methods depend on detected feature points, which usually involve a very large amount of computation, and the accuracy of the optical flow algorithm is affected by the quality of feature point detection, and the movement of an object in a screen may cause the optical flow algorithm to generate an incorrect estimation, thereby causing a poor image effect of final display.
In the application, the target area (such as the four corners of the target video frame) of the target video frame is extracted for shake detection, so that the influence of local motion (such as the influence of a central motion matrix of the target video frame) can be effectively avoided, and the method is suitable for a dynamic shooting environment.
In addition, the value with the maximum jitter degree is selected as the jitter degree of the whole target video frame, so that the interference of a moving target in the target video frame on jitter detection can be effectively avoided.
And step 130, under the condition that the jitter factor is lower than the target jitter threshold value, generating a target image of the target motion matrix based on the target video frame.
In this step, the target shake threshold is the maximum shake value of the image in a state considered approximately as still.
In the event that the jitter factor is below the target jitter threshold, then the target video frame to which the jitter factor corresponds may be approximately considered stationary.
The target image is an image which is obtained by processing an original image and comprises moving pedestrians and shadows of the pedestrians in the target motion matrix.
The target image is used for representing the real-time state of the target motion matrix.
In this embodiment, when the jitter factor of the target video frame is lower than the target jitter threshold, the target video frame is approximately regarded as non-jittered, and the target video frame is processed in the next step.
In other embodiments, when the jitter factor of the target video frame is not lower than the target jitter threshold, the target video frame corresponding to the jitter factor is skipped and the next frame video frame is detected if the jitter degree of the target video frame is considered to be greater.
Next, a specific description will be given of a method of generating the target image in this step.
In some embodiments, step 130 may further include:
performing frame difference operation on a target video frame and a target background model to generate a first image;
carrying out binarization processing and noise reduction processing on the first image to generate a third image;
and extracting a foreground image in the target video frame based on the third image, and generating a target image of the target motion matrix based on the foreground image.
In this embodiment, the foreground image includes the moving pedestrian in the target motion matrix and the projection of the moving pedestrian in the target motion matrix on the ground.
In the actual implementation process, for a target video frame without jitter, coarse-grained detection of a foreground region can be performed first.
For example, as shown in fig. 4, a frame difference operation is performed on a target video frame and a target background model to generate a first image.
The target background model is a model established in advance, and the establishing step of the target background model in the subsequent embodiments will be explained, which is not repeated herein.
Then, firstly, carrying out binarization processing on the frame difference result, and then carrying out noise reduction processing to eliminate the influence of noise of the first image and intrusion interference of a smaller moving object so as to generate a third image, wherein the third image is used for representing the contour characteristics of a motion matrix in a target video frame, as shown in fig. 5;
the noise reduction process may include basic morphological operations of erosion and dilation, among others.
Then, a foreground image in the target video frame within the contour corresponding to the third image is extracted based on the third image, and a target image is generated based on the foreground region, as shown in fig. 8.
The foreground region at this time will contain the pedestrians in the square matrix and the shadows of the pedestrians, as shown in fig. 6.
According to the formation detection method of the large-scale moving square matrix provided by the embodiment of the invention, the moving target is detected by combining the background modeling method and the frame difference method, so that the improvement on details is realized.
In some embodiments, generating the target image of the target motion matrix based on the foreground image may further include:
converting the foreground image into a target format to generate a fourth image;
and performing shadow region segmentation processing and corrosion expansion processing based on a target shadow threshold value on the fourth image to generate a target image of a target motion matrix.
In this embodiment, color space conversion and fine detection of the square matrix region can be performed on the foreground image after coarse-grained detection of the foreground region, so as to eliminate the shadow of the foreground region and make the extraction of the foreground more accurate.
Wherein the target shadow threshold may be user-defined.
For example, the third image is converted from an RGB color space to an HSV color space to generate a fourth image. Among them, the HSV color space includes color (H), saturation (S), and brightness (V).
By converting into the HSV color space, the square matrix area and the shadow area in the foreground area in the third image can be effectively distinguished based on the luminance value. As shown in fig. 7, the difference between the luminance values (V space) is significant.
By using the distribution characteristic of the third image in the HSV color space, after the third image is converted from the RBG color space to the HSV color space to generate a fourth image, the shadow region of the image in the foreground region in the fourth image is segmented at the brightness V space by setting a threshold value, and then the erosion expansion operation is performed, so that the target image without shadow interference is obtained, as shown in fig. 8.
In the embodiment, the method of distortion correction, corrosion expansion and the like is combined, the influence of interference such as jitter, illumination, shadow and the like under outdoor natural conditions is overcome, and the real-time detection of the motion matrix formation based on the video frame is finally realized.
In some embodiments, the method may further comprise: the target shading threshold is updated every third target time period.
For example, the numerical value distribution of the foreground region (square matrix and shadow region) on the brightness space V can be used, the prior knowledge determines that the value range of the pixel point of the shadow region in the V space is between 50 and 100, the interval of 10 to 200 of the V space numerical value distribution is taken, the least square method is used for fitting a curve, the minimum value of the trough is taken as the target shadow threshold Ψ for segmenting the shadow region, and the target shadow threshold Ψ is updated at intervals of a third target time period.
Setting all values of image pixel point values corresponding to the brightness space V, which are lower than the target shadow threshold psi, as 0, sequentially carrying out basic morphological operations of corrosion and expansion, eliminating the influence of image noise, and then extracting a finer foreground region and a target image.
According to the method for detecting the formation of the large-scale movable motion matrix, provided by the embodiment of the invention, the target shadow threshold is updated in real time in a self-adaptive target shadow threshold updating mode, and the target shadow threshold can be adjusted to be the optimal threshold in time based on the illumination intensity at different time, so that the condition that the best effect cannot be kept when the natural illumination intensity is changed due to the fixed threshold is avoided, the problem that the image shadow in an outdoor natural environment is easily influenced by the change of the illumination intensity is also solved, the method can adapt to the motion matrix formation detection under the long-time outdoor natural condition, the segmentation effect of a shadow area is favorably improved, the imaging effect of a target image is improved, and the detection effect is further improved.
In the step, the target video frame is processed for multiple times, so that the noise can be effectively reduced, shadow interference is eliminated, a clear target image is obtained, and the accuracy of a detection result is improved.
As shown in fig. 2, in some embodiments, after determining that the jitter factor is below the target jitter threshold in step 130, the method may further include:
acquiring the definition corresponding to the target video frame;
and under the condition that the definition is not lower than the definition threshold, generating a target image of the target motion matrix based on the target video frame.
In the case that the sharpness is below the sharpness threshold, then a dithering factor for the next frame target video frame is detected.
In this embodiment, the sharpness threshold is the minimum value of the sharpness of the image in a state approximately regarded as sharp.
The sharpness threshold may be user-defined.
In the actual implementation process, the definition of the target video frame can be evaluated by adopting a Laplacian gradient function.
The method can be specifically realized by the following formula:
determining, wherein M, N represents the width and height of the target video frame, respectively, T is a given edge detection threshold, and z (x, y) is the convolution of Laplacian operator at pixel point (x, y).
The Laplacian operator matrix is defined as the following formula:
it will be appreciated that for a target video frame that has been converted to a grey scale map, the sharper the image of the input convolved target video frame, the larger the Laplacian gradient function value.
Obtaining the definition omega corresponding to the target video frame t The sharpness is then compared to a sharpness threshold Ω, at Ω t And if the target video frame is determined to be clear, generating a target image of the target motion matrix based on the target video frame.
In other embodiments, at Ω t If the omega is less than omega, the target video frame is determined to be unclear, the target video frame is skipped,a dithering factor for a next frame of video frames is detected.
In the embodiment, by detecting the definition of the target video frame, the target video frame which does not meet the definition requirement is skipped on the basis that the jitter factor is lower than the target jitter threshold, so that the influence on the square matrix detection caused by camera jitter, overexposure and other conditions can be avoided, the imaging definition is improved, and the accuracy of the detection result is improved.
According to the method for detecting the formation of the large-scale moving square matrix, provided by the embodiment of the invention, the jitter factor is generated based on the target video frame in the target video and the diagonal gray level projection standard array, the target video frame is screened based on the jitter factor so as to keep the target video frame with lower jitter amplitude, and the target video frame with lower jitter amplitude is processed to generate the target image of the target moving square matrix.
The following describes the generation procedure of the target background model by using a specific example.
In some embodiments, prior to step 110, the method further comprises:
acquiring a multi-frame environment background image in a first target time period under a target environment;
generating a target background model based on the multi-frame environment background image;
and generating a diagonal gray projection standard array corresponding to the target background model based on the target background model.
In this embodiment, the first target time period may be user-defined.
The environment background image is the same background image as the target video.
The ambient background image may be captured by an image sensor, such as by a panoramic camera.
The number of the environment background images can be customized based on the user, such as setting to 1000 frames.
After the environment background image is acquired, the distortion correction may be preferentially performed on the environment background image, and the specific implementation manner is the same as that in the above embodiment, which is not described herein again.
For example, in actual implementation, a gaussian background model may be first used to model the first 1000 consecutive frames of the environmental background image to generate the target background model.
After the target background model is generated, the target background model can be saved in a local server or a cloud server and can be called when needed.
The following describes a specific implementation manner of generating a diagonal grayscale projection standard array corresponding to the target background model based on the target background model.
Firstly, each frame of environment background image is converted into a gray scale image, four square areas with the same side length of d are respectively taken as target areas at the upper left corner, the lower left corner, the upper right corner and the lower right corner of the background model image, and therefore the influence of a large-size moving target at the middle position of a future picture on jitter detection is avoided.
It should be noted that the target area in the present embodiment needs to be consistent with the position and number of the target area in the target video frame extracted in the above embodiment.
And respectively carrying out histogram equalization processing on the four target areas.
The method can be specifically realized by the following formula:
performing histogram equalization processing, wherein f is monotonic nonlinear mapping, G I Representing the gray value of a pixel point in an environment background image I, L representing the gray level number, I n Is the number of pixel points in the environment background image, H I Represents the gray-scale histogram distribution of the environmental background image I, and u represents the gray-scale value.
L is typically 256.
After histogram equalization processing, gray projection is performed on the four areas subjected to histogram equalization to generate a gray projection array standard value.
As shown in fig. 3, the grayscale projection array standard value is used to represent the comprehensive grayscale information of the four corner regions of the background image in the current environment in the horizontal and vertical directions.
In some embodiments, the data may be represented by the formula:
calculating the gray average value of the environment background image, wherein I _ mean represents the gray value of a pixel point in the environment background image,
representing the gray value of a pixel point (x, y) in the environment background image, d representing the side length of each target area, I n The number of the pixel points in the environment background image is shown.
After obtaining the gray level average value of the environment background image, based on the gray level average value, the method comprises the following steps:
calculating to obtain diagonal gray projection array standard values corresponding to the target areas;
wherein Gr _ b1 represents the standard value of the kth position of the diagonal gray projection array of the upper left corner region of the environment background image, d represents the side length of the target region,
and representing the gray value of the pixel point (i, j) in the environment background image.
It is understood that Gr _ b2 represents a standard value of the kth position of the diagonal gray projection array in the upper right corner region of the environment background image, Gr _ b3 represents a standard value of the kth position of the diagonal gray projection array in the lower right corner region of the environment background image, and Gr _ b4 represents a standard value of the kth position of the diagonal gray projection array in the lower left corner region of the environment background image.
After the diagonal gray projection standard array is obtained, the diagonal gray projection standard array can be stored for calculation in the subsequent actual detection process.
In some embodiments, the target background model may be updated every second target time period t1 to avoid the change of information such as background brightness caused by the outdoor environment being susceptible to the sun illumination.
In the embodiment, the method of distortion correction, histogram equalization and the like is combined, so that the influence of interference such as jitter, illumination, shadow and the like under outdoor natural conditions can be overcome.
According to the object detection method of the motion policy provided by the embodiment of the invention, the diagonal gray projection standard array is generated based on the multi-frame environment background image in the first target time period and is used for the shake detection of the subsequent target video frame, so that the flexibility and the detection accuracy are high.
The following describes the formation detection device of the exercise matrix of the large-scale activity according to the present invention, and the formation detection device of the exercise matrix of the large-scale activity described below and the formation detection method of the exercise matrix of the large-scale activity described above may be referred to in correspondence with each other.
As shown in fig. 9, the formation detection device for the motion matrix of the large-scale activity includes: a first acquisition module 910, a first generation module 920, and a second generation module 930.
A first obtaining module 910, configured to obtain a target video of a target motion matrix in a target environment;
a first generating module 920, configured to generate a dithering factor corresponding to a target video frame based on the target video frame in the target video and a diagonal grayscale projection standard array;
a second generating module 930 configured to generate a target image of the target motion matrix based on the target video frame if the shake factor is lower than the target shake threshold.
According to the object detection device of the motion policy provided by the embodiment of the invention, the jitter factor is generated based on the target video frame in the target video and the diagonal gray level projection standard array, the target video frame is screened based on the jitter factor so as to reserve the target video frame with lower jitter amplitude, and the target video frame with lower jitter amplitude is processed to generate the target image of the target motion square matrix, so that the imaging definition is effectively improved, and the detection result is favorably improved.
In some embodiments, the first generating module 920 is configured to:
generating a first diagonal gray projection array of a target area based on the target area of the target video frame;
and generating a dithering factor based on the first diagonal gray level projection array and the diagonal gray level projection standard array.
In some embodiments, the first generating module 920 is configured to: applying the formula:
generating a dithering factor, wherein theta is the dithering factor, Gr _ bi (m) is a diagonal gray scale projection standard value of the mth position in the diagonal gray scale projection standard array, Gr _ ci (m) is a first diagonal gray scale projection value of the mth position in the first diagonal gray scale projection standard array, and d is the side length of the target area.
In some embodiments, the second generating module 930 is configured to:
performing frame difference operation on a target video frame and a target background model to generate a first image;
carrying out binarization processing and noise reduction processing on the first image to generate a third image;
and extracting a foreground image in the target video frame based on the third image, and generating a target image of the target motion matrix based on the foreground image.
In some embodiments, the second generating module 930 is configured to:
converting the foreground image into a target format to generate a fourth image;
and performing shadow region segmentation processing and corrosion expansion processing based on a target shadow threshold value on the fourth image to generate a target image of a target motion matrix.
In some embodiments, the apparatus further comprises:
the second acquisition module is used for acquiring a multi-frame environment background image in a first target time period in the target environment before acquiring a target video of a target motion matrix in the target environment;
the third generation module is used for generating a target background model based on the multi-frame environment background image;
and the fourth generation module is used for generating a diagonal gray projection standard array corresponding to the target background model based on the target background model.
Fig. 10 illustrates a physical structure diagram of an electronic device, and as shown in fig. 10, the electronic device may include: a processor (processor)1010, a communication Interface (Communications Interface)1020, a memory (memory)1030, and a communication bus 1040, wherein the processor 1010, the communication Interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. Processor 1010 may invoke logic instructions in memory 1030 to perform a method of formation detection for a large active motion matrix, the method comprising: acquiring a target video of a target motion matrix in a target environment; generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array; and under the condition that the jitter factor is lower than a target jitter threshold value, generating a target image of the target motion matrix based on the target video frame.
Furthermore, the logic instructions in the memory 1030 can be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method for detecting formation of a large-scale active exercise matrix provided by the above methods, the method comprising: acquiring a target video of a target motion matrix in a target environment; generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array; and under the condition that the jitter factor is lower than a target jitter threshold value, generating a target image of the target motion matrix based on the target video frame.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method for formation detection of a large-scale activity sports square provided above, the method comprising: acquiring a target video of a target motion matrix in a target environment; generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array; and under the condition that the jitter factor is lower than a target jitter threshold value, generating a target image of the target motion matrix based on the target video frame.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A formation detection method of a large-scale movable motion matrix is characterized by comprising the following steps:
acquiring a target video of a target motion matrix in a target environment;
generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array;
and under the condition that the jitter factor is lower than a target jitter threshold value, generating a target image of the target motion matrix based on the target video frame.
2. The method for detecting formation of a large moving square matrix according to claim 1, wherein the generating a dithering factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray projection standard array comprises:
generating a first diagonal gray projection array of a target region of the target video frame based on the target region, the target region not including features of the motion matrix;
generating the dithering factor based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array.
3. The method of claim 2, wherein the generating the dithering factor based on the first diagonal gray projection array and the diagonal gray projection standard array comprises:
applying the formula:
generating the dithering factor, wherein θ is the dithering factor, Gr _ bi (m) is a diagonal grayscale projection standard value of an mth position in the diagonal grayscale projection standard array, Gr _ ci (m) is a first diagonal grayscale projection value of the mth position in the first diagonal grayscale projection standard array, and d is a side length of the target area.
4. The method of claim 1, wherein the generating the target image of the target motion matrix based on the target video frame comprises:
performing frame difference operation on the target video frame and the target background model to generate a first image;
carrying out binarization processing and noise reduction processing on the first image to generate a third image;
and extracting a foreground image in the target video frame based on the third image, and generating a target image of the target motion matrix based on the foreground image.
5. The method according to claim 4, wherein the generating the target image of the target motion matrix based on the foreground image comprises:
converting the foreground image into a target format to generate a fourth image;
and carrying out target shadow threshold-based shadow region segmentation processing and erosion expansion processing on the fourth image to generate a target image of the target motion matrix.
6. The method for formation detection of a large active motion matrix according to any one of claims 1-5, wherein before said obtaining the target video of the target motion matrix in the target environment, the method further comprises:
acquiring a multi-frame environment background image in a first target time period under the target environment;
generating a target background model based on the multi-frame environment background image;
and generating a diagonal gray projection standard array corresponding to the target background model based on the target background model.
7. A formation detection device of a large-scale movable motion matrix is characterized by comprising:
the first acquisition module is used for acquiring a target video of a target motion matrix in a target environment;
the first generation module is used for generating a jitter factor corresponding to a target video frame based on the target video frame in the target video and a diagonal gray level projection standard array;
and the second generation module is used for generating a target image of the target motion matrix based on the target video frame under the condition that the jitter factor is lower than a target jitter threshold value.
8. An electronic device comprising a memory, a processor and a computer program stored on said memory and executable on said processor, wherein said processor when executing said program implements a method of formation detection of a sports square of a large activity according to any of claims 1 to 6.
9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements a method for formation detection of a large active sports square according to any of claims 1 to 6.
10. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements a method for formation detection of a sports square for a large activity according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210314674.3A CN114820698A (en) | 2022-03-28 | 2022-03-28 | Formation detection method and device for large-scale movable motion matrix |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210314674.3A CN114820698A (en) | 2022-03-28 | 2022-03-28 | Formation detection method and device for large-scale movable motion matrix |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114820698A true CN114820698A (en) | 2022-07-29 |
Family
ID=82531310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210314674.3A Pending CN114820698A (en) | 2022-03-28 | 2022-03-28 | Formation detection method and device for large-scale movable motion matrix |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114820698A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102202164A (en) * | 2011-05-20 | 2011-09-28 | 长安大学 | Motion-estimation-based road video stabilization method |
| CN102930559A (en) * | 2012-10-23 | 2013-02-13 | 华为技术有限公司 | Image processing method and device |
| CN103955930A (en) * | 2014-04-28 | 2014-07-30 | 中国人民解放军理工大学 | Motion parameter estimation method based on gray integral projection cross-correlation function characteristics |
| US20180027161A1 (en) * | 2014-05-06 | 2018-01-25 | Urugus S.A. | Imaging device for scenes in apparent motion |
| CN108492287A (en) * | 2018-03-14 | 2018-09-04 | 罗普特（厦门）科技集团有限公司 | A kind of video jitter detection method, terminal device and storage medium |
| CN111741186A (en) * | 2020-06-11 | 2020-10-02 | 浙江大华技术股份有限公司 | Video jitter detection method, device and system |
-
2022
- 2022-03-28 CN CN202210314674.3A patent/CN114820698A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102202164A (en) * | 2011-05-20 | 2011-09-28 | 长安大学 | Motion-estimation-based road video stabilization method |
| CN102930559A (en) * | 2012-10-23 | 2013-02-13 | 华为技术有限公司 | Image processing method and device |
| CN103955930A (en) * | 2014-04-28 | 2014-07-30 | 中国人民解放军理工大学 | Motion parameter estimation method based on gray integral projection cross-correlation function characteristics |
| US20180027161A1 (en) * | 2014-05-06 | 2018-01-25 | Urugus S.A. | Imaging device for scenes in apparent motion |
| CN108492287A (en) * | 2018-03-14 | 2018-09-04 | 罗普特（厦门）科技集团有限公司 | A kind of video jitter detection method, terminal device and storage medium |
| CN111741186A (en) * | 2020-06-11 | 2020-10-02 | 浙江大华技术股份有限公司 | Video jitter detection method, device and system |
Non-Patent Citations (1)
| Title |
|---|
| 路霄汉等: "一种快速消除鬼影的目标检测算法", 计算机应用与软件, vol. 36, no. 10, 12 October 2019 (2019-10-12), pages 101 - 106 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111402135B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| US10339643B2 (en) | Algorithm and device for image processing | |
| CN113313661B (en) | Image fusion method, device, electronic equipment and computer readable storage medium | |
| US11538175B2 (en) | Method and apparatus for detecting subject, electronic device, and computer readable storage medium | |
| US9202263B2 (en) | System and method for spatio video image enhancement | |
| CN110852997B (en) | Dynamic image definition detection method and device, electronic equipment and storage medium | |
| CN112785534A (en) | Ghost-removing multi-exposure image fusion method in dynamic scene | |
| CN113469897B (en) | Training of image enhancement model, image enhancement method and device and electronic equipment | |
| US12039767B2 (en) | Subject detection method and apparatus, electronic device, and computer-readable storage medium | |
| CN111784609A (en) | Image dynamic range compression method and device and computer readable storage medium | |
| CN112351195B (en) | Image processing method, device and electronic system | |
| WO2023273868A1 (en) | Image denoising method and apparatus, terminal, and storage medium | |
| CN111445487A (en) | Image segmentation method and device, computer equipment and storage medium | |
| CN111062331A (en) | Mosaic detection method and device for image, electronic equipment and storage medium | |
| CN116433496A (en) | Image denoising method, device and storage medium | |
| CN114820394A (en) | Image noise filtering method and device based on robust parameter estimation | |
| CN113438386B (en) | Dynamic and static judgment method and device applied to video processing | |
| CN111539975B (en) | Method, device, equipment and storage medium for detecting moving object | |
| JP4496005B2 (en) | Image processing method and image processing apparatus | |
| CN110334606B (en) | Picture-in-picture positioning method and device | |
| CN116597252A (en) | Picture generation method, device, computer equipment and storage medium | |
| CN114820698A (en) | Formation detection method and device for large-scale movable motion matrix | |
| CN111147693B (en) | Noise reduction method and device for full-size photographed image | |
| CN101364303B (en) | Edge pixel extracting and processing method | |
| CN117422656B (en) | Low-illumination fuzzy traffic image enhancement method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |