CN117319747A - Video splicing method for fully-mechanized mining face of coal mine - Google Patents
Video splicing method for fully-mechanized mining face of coal mine Download PDFInfo
- Publication number
- CN117319747A CN117319747A CN202311302439.5A CN202311302439A CN117319747A CN 117319747 A CN117319747 A CN 117319747A CN 202311302439 A CN202311302439 A CN 202311302439A CN 117319747 A CN117319747 A CN 117319747A
- Authority
- CN
- China
- Prior art keywords
- video
- image
- point
- fully
- mining face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000003245 coal Substances 0.000 title claims abstract description 78
- 238000005065 mining Methods 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000012545 processing Methods 0.000 claims abstract description 42
- 230000004927 fusion Effects 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims description 32
- 230000008859 change Effects 0.000 claims description 27
- 230000009466 transformation Effects 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 10
- 230000014759 maintenance of location Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000002817 coal dust Substances 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a video stitching method for a fully mechanized coal mining face, and belongs to the technical field of video fusion. The method comprises the following steps: video acquisition is carried out on the fully mechanized mining face of the coal mine according to the pre-deployment multi-camera, and video grouping processing is carried out on the acquired video according to the deployment position of the pre-deployment multi-camera; frame extraction is carried out on each group of videos, and each frame of extracted image is numbered to obtain an image sequence to be spliced of each corresponding group, wherein the frame number of frame extraction of each group of videos is the same; establishing grouping target characteristics of the multi-camera at the deployment position, and carrying out standardized processing on images in the corresponding image sequences to be spliced according to the grouping target characteristics to obtain standard images; and video stitching is carried out based on all the standard images, and the stitched video of the fully-mechanized coal mining face of the coal mine is obtained. And the complete video of the fully-mechanized mining face of the coal mine can be conveniently obtained.
Description
Technical Field
The invention relates to the technical field of video fusion, in particular to a video splicing method for a fully-mechanized mining face of a coal mine.
Background
Currently, in the working environment of a fully-mechanized mining face of a coal mine, a plurality of cameras are generally arranged for monitoring and recording conditions of a working site. The cameras may cover different angles and positions to obtain more comprehensive video information. However, the frames captured by one camera are limited, and the situation of the whole fully-mechanized mining face cannot be completely presented. The condition of abnormal analysis errors caused by incomplete pictures can be caused, so that the management of the fully mechanized mining face is difficult and the safety problem is caused.
Therefore, the invention provides a video splicing method for the fully-mechanized coal mining face.
Disclosure of Invention
The invention provides a video splicing method for a fully-mechanized coal mining face, which is used for monitoring and shooting the fully-mechanized coal mining face through a plurality of cameras, processing shooting videos of a plurality of cameras, matching the shooting videos according to the coincidence degree, and splicing the shooting videos into a complete video of the fully-mechanized coal mining face, so that the follow-up management is convenient.
The invention provides a video splicing method for a fully-mechanized coal mining face, which comprises the following steps:
step 1: video acquisition is carried out on the fully mechanized mining face of the coal mine according to the pre-deployment multi-camera, and video grouping processing is carried out on the acquired video according to the deployment position of the pre-deployment multi-camera;
step 2: frame extraction is carried out on each group of videos, and each frame of extracted image is numbered to obtain an image sequence to be spliced of each corresponding group, wherein the frame number of frame extraction of each group of videos is the same;
step 3: establishing grouping target characteristics of the multi-camera at the deployment position, and carrying out standardized processing on images in the corresponding image sequences to be spliced according to the grouping target characteristics to obtain standard images;
step 4: and video stitching is carried out based on all the standard images, and the stitched video of the fully-mechanized coal mining face of the coal mine is obtained.
In one possible implementation, video acquisition is performed on a fully-mechanized mining face of a coal mine according to pre-deployment of a plurality of cameras, including:
based on the basic environment condition of the fully-mechanized mining face of the coal mine and the region needing coverage shooting, an initial deployment position and a shooting angle of the multi-camera are drawn, and a serial number corresponding relation of the initial deployment position and the multi-camera of each initial deployment position is established;
and carrying out video acquisition on the fully-mechanized coal mining face by using a plurality of cameras deployed based on the corresponding relation of the numbers.
In one possible implementation, video packet processing is performed on the acquired video, including:
according to the corresponding relation of the numbers of each working area of the fully-mechanized coal mining face, all acquired videos are subjected to grouping processing;
and obtaining the grouping video of the corresponding working area according to the grouping processing result.
In one possible implementation manner, frame extraction is performed on each group of video, and each frame of extracted image is numbered to obtain a sequence of images to be spliced of each corresponding group, including:
decoding each group of videos into an original image sequence by using a video decoder, wherein the original image sequence is formed by unequal image frame amounts under a plurality of acquired videos;
determining average duration according to the video duration of each acquired video in each grouped video, and determining the extraction frame number of the current group according to an average-extraction frame table;
extracting a plurality of images in the same acquired video according to the time interval of the corresponding acquired video to the original image sequence under the same grouped video according to the extraction frame number to obtain a still image, and preprocessing the extracted still image to obtain an initial image sequence to be spliced, wherein the initial image sequence to be spliced is formed by equal image frame amounts under a plurality of acquired videos;
numbering each frame of image in the initial image sequence to be spliced according to a time logic sequence to obtain the image sequence to be spliced of the target acquisition video.
In one possible implementation, establishing a grouping target feature in a deployment location of the multi-view camera includes:
performing pixel point alignment processing on the still images extracted from the same acquired video in the same grouped video to acquire multi-feature images under the corresponding acquired video, wherein each acquired video corresponds to one multi-feature image;
splicing and edge fusion are carried out on all the multi-feature images under the same grouping video to obtain a target fusion image;
calculating the gradient of each pixel point in the target fusion image, and constructing a gradient set of the target fusion image:let the target fusion image have n pixels in total, < >>Representing the gradient of the i11 th pixelAnd the value range of i11 is [1, n]；
Taking a point s of the target fusion image as the point s, and selecting surrounding 3x3 pixel points to construct a gradient covariance matrix：
The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing covariance under a point s-based gradient; />Representing covariance based on the upper left side gradient of point s; />Representing covariance at the left-hand gradient based on point s; />Representing covariance at the lower left gradient based on point s; />Representing covariance based on the upper right side gradient of point s; />Representing covariance based on the gradient to the right of point s; />Representing covariance at the lower right gradient based on point s; />Representing covariance based on the side gradient above point s; />Representing covariance at the gradient of the lower side based on the point s;
based on gradient covariance matrixCalculate +.>Retention value of each point:
the method comprises the steps of carrying out a first treatment on the surface of the Wherein r represents->A reserved value of the corresponding point in (a); />Representation ofCovariance of corresponding points in (a); />Representation->Maximum of all covariances in (a); />Representation->Average of all covariances in (a); />Covariance matrix representing corresponding points +.>Is a modulus of (2); />Representing covariance matrix->Is a modulus of (2); max represents the maximum value symbol;
determining a gradient covariance matrixAnd taking the points corresponding to the reserved values larger than the average value as feature points, so as to obtain all feature points on the target feature image, and further obtain grouping target features.
In one possible implementation manner, the normalizing processing is performed on the images in the corresponding image sequence to be spliced according to the grouping target feature, including:
performing perspective change processing on multi-feature images under the same grouping video to obtain a perspective change matrix, substituting the grouping target features into the perspective change matrix, and obtaining a standard conversion function of the corresponding multi-feature images;
converting the standard conversion function with the isoframe still images under the corresponding acquired video in the images to be spliced under the same group to obtain a first image sequence of the corresponding acquired video;
and collecting the first image sequences to obtain standard image sequences under corresponding groups.
In one possible implementation manner, video stitching is performed based on all standard image sequences, and a stitched video of the fully-mechanized mining face of the coal mine is obtained, including:
performing edge matching on the kth standard image of the left video and the kth standard image of the right video which have adjacent relations in the same group to generate a transformation matrix;
fusing an original video image corresponding to a kth standard image of the left video and a kth standard image of the right video by utilizing a transformation matrix to obtain a kth frame fused image of the belonging packet spliced video;
and splicing all the frame fusion images under all the groups to obtain a spliced video of the fully-mechanized coal mining face.
In one possible implementation manner, the process of performing edge matching on the kth standard image of the left video and the kth standard image of the right video in the adjacent relation in the same group further includes:
image alignment processing is carried out on the kth standard image of the left video and the kth standard image of the right video, equidistant edge points are screened on the left edge line and the right edge line, a point array is constructed, and a linear change sequence based on the point array is obtainedWherein->A linear value representing the j01 th column; />A value representing a first point of a j01 th column in the point array; />A value representing a second point of the j01 th column in the point array; />Representing the standard value of the j01 th column in the point array; m01 represents the number of columns included in the corresponding point array, and each column includes an edge point pair;
according to the linear change sequence, determining an error coefficient of edge matching:
the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing the corresponding error coefficient; m02 represents +.>A number that is not within a preset range; />A difference function representing a linear value that is not within a preset range;representing a corresponding point arrayIs a difference function of (2); />Representing the jth 02 linear value which is not within a preset range;
and adjusting edge point pairs which are not in a preset range based on the error coefficient, so as to realize subsequent edge matching.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
fig. 1 is a flowchart of a video stitching method for a fully-mechanized coal mining face in an embodiment of the invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
Example 1:
the invention provides a video stitching method for a fully-mechanized coal mining face, which is shown in fig. 1 and comprises the following steps:
step 1: video acquisition is carried out on the fully mechanized mining face of the coal mine according to the pre-deployment multi-camera, and video grouping processing is carried out on the acquired video according to the deployment position of the pre-deployment multi-camera;
step 2: frame extraction is carried out on each group of videos, and each frame of extracted image is numbered to obtain an image sequence to be spliced of each corresponding group, wherein the frame number of frame extraction of each group of videos is the same;
step 3: establishing grouping target characteristics of the multi-camera at the deployment position, and carrying out standardized processing on images in the corresponding image sequences to be spliced according to the grouping target characteristics to obtain standard images;
step 4: and video stitching is carried out based on all the standard images, and the stitched video of the fully-mechanized coal mining face of the coal mine is obtained.
In this embodiment, a multi-view camera refers to a plurality of video cameras or cameras mounted on the same device. The cameras can simultaneously capture scene pictures of the fully-mechanized coal mining face at different angles or positions, so that a more comprehensive view angle and richer working face information are provided, wherein a plurality of cameras exist, namely a plurality of deployment positions are deployed in advance, and all pictures of the fully-mechanized coal mining face can be acquired.
In this embodiment, for example, there are a coal mine area 1, an area 2, and an area 3, and there are multiple cameras 01 and 02 in the area 1, a multiple camera 03 in the area 2, and multiple cameras 04 and 05 in the area 3, and at this time, the multiple cameras 01 and 02 are set, the multiple camera 03 is set, and the multiple cameras 04 and 05 are set.
In this embodiment, the video acquisition refers to the use of converting video signals of a fully-mechanized coal mining face photographed in real time into digital data through a multi-camera photographing device input device so as to perform processing, storage and transmission.
In the embodiment, the deployment position of the pre-deployment multi-camera is that before the deployment of the cameras, the fully-mechanized mining face of the current coal mine is firstly surveyed and detected, and the deployment positions and shooting angles of all the multi-camera are determined.
In this embodiment, frame extraction refers to a process of selecting a specific video frame from a continuous video stream as a separate image frame for processing or presentation.
In this embodiment, the image sequence to be spliced is a group of image sequences to be spliced obtained after frame extraction and time sequential labeling are performed on all videos under a current group, for example, a group 1 has a video a1 and a video a2, at this time, s0 frame images are extracted from the video a1 and s0 frame images are extracted from the video a2, and thus, the extracted frame images are time sequential labeled: ta11. ta1s0 and ta21. ta2s0 are the image sequences to be stitched.
In this embodiment, the grouping target feature refers to a grouping target feature of a current grouping formed by a plurality of feature points after feature processing, that is, the grouping target feature is formed by performing centralized stitching on the most feature points existing under each video in the same grouping.
In this embodiment, the normalization process is a process of subjecting an isoframe still image under an acquired video to a perspective change process, which is a standard conversion function.
In this embodiment, the standard image is an image obtained by subjecting an equal frame still image under the captured video to normalization processing.
In this embodiment, video stitching refers to performing edge connection processing on images with adjacent relationships to obtain a stitched video.
The working principle and the beneficial effects of the technical scheme are as follows: the video splicing method for the fully-mechanized coal mining face based on shooting input of the multi-view camera can monitor and splice videos of the fully-mechanized coal mining face in real time, ensure that the videos can be completely presented, and realize effective monitoring of the fully-mechanized coal mining face.
Example 2:
on the basis of the above embodiment 1, the present embodiment provides a method for video stitching of a fully-mechanized coal mining face, in step 1, video acquisition is performed on the fully-mechanized coal mining face according to a pre-deployed multi-camera, including:
based on the basic environment condition of the fully-mechanized mining face of the coal mine and the region needing coverage shooting, an initial deployment position and a shooting angle of the multi-camera are drawn, and a serial number corresponding relation of the initial deployment position and the multi-camera of each initial deployment position is established;
and carrying out video acquisition on the fully-mechanized coal mining face by using a plurality of cameras deployed based on the corresponding relation of the numbers.
In this embodiment, the basic environment condition is the basic environment condition of the fully-mechanized mining face of the coal mine, which refers to some environmental indexes when the comprehensive mining operation is performed underground the coal mine, including: geological conditions, the size and shape of a working surface, the mechanical properties of coal and rock mass, the condition of gas and coal dust, environmental temperature and humidity and other environmental indexes.
In this embodiment, the number correspondence is to set unique numbers of the initial deployment position and the multiple cameras, and the number of the deployment position and the number of the multiple cameras need to be in one-to-one correspondence for preparation before deployment.
The working principle and the beneficial effects of the technical scheme are as follows: the multi-view camera can omnidirectionally record and collect the working state and the production condition of the fully-mechanized coal mining face, and the defect that an ordinary camera is not capable of capturing the picture of the fully-mechanized coal mining face and the acquired data cannot support complete analysis is avoided.
Example 3:
on the basis of the above embodiment 1, the present embodiment provides a video stitching method for a fully-mechanized coal mining face, in step 1, video grouping processing is performed on an acquired video, including:
according to the corresponding relation of the numbers of each working area of the fully-mechanized coal mining face, all acquired videos are subjected to grouping processing;
and obtaining the grouping video of the corresponding working area according to the grouping processing result.
In this embodiment, the working area of the fully-mechanized mining face refers to a specific area for performing comprehensive mining operations of a coal mine, and includes: the system comprises a coal seam exploitation area, a coal mining area, a supporting area and a transportation area.
The working principle and the beneficial effects of the technical scheme are as follows: by carrying out video grouping processing based on the corresponding relation of the numbers, the collected video data can be utilized more effectively, monitoring, evaluation and analysis of each working area are realized, and the problems of heavy splicing process and overlong splicing time consumption caused by excessive videos can be solved.
Example 4:
on the basis of the above embodiment 1, the present embodiment provides a method for video stitching of a fully-mechanized coal mining face, in step 2, frame extraction is performed on each group of videos, and each frame of extracted images is numbered to obtain a to-be-stitched image sequence of each corresponding group, including:
decoding each group of videos into an original image sequence by using a video decoder, wherein the original image sequence is formed by unequal image frame amounts under a plurality of acquired videos;
determining average duration according to the video duration of each acquired video in each grouped video, and determining the extraction frame number of the current group according to an average-extraction frame table;
extracting a plurality of images in the same acquired video according to the time interval of the corresponding acquired video to the original image sequence under the same grouped video according to the extraction frame number to obtain a still image, and preprocessing the extracted still image to obtain an initial image sequence to be spliced, wherein the initial image sequence to be spliced is formed by equal image frame amounts under a plurality of acquired videos;
numbering each frame of image in the initial image sequence to be spliced according to a time logic sequence to obtain the image sequence to be spliced of the target acquisition video.
In this embodiment, the video decoder is a software or hardware device for decoding video files in compressed format. And decoding the compressed and encoded coal mine collected video data, and restoring the data into images and sounds.
In this embodiment, the original image sequence is an original image group obtained by processing the coal mine collected video under each group by a video decoder, for example: under the group 1, there is an a1 video, and the a1 video can obtain a group of continuous image frames a1e1, a1e2 … a1en through a video decoder, and the n continuous image frames are original image groups of the a1 video.
In this embodiment, the unequal image frame amounts mean that the number of image frames included in the video per second is not constant, for example, s0 frame images are present in advance in the video a1, and s1 frame images are present in advance in the video a 2.
In this embodiment, the average-frame extraction table is a corresponding relation table based on average duration of all videos under a group and the corresponding required frame extraction number, so that the frame extraction number can be obtained by direct matching after the average duration is obtained.
In this embodiment, preprocessing is processing means for extracting a frame number from one video, and then, resizing, cropping, filtering, and the like, are performed on the extracted still image.
In this embodiment, the initial image sequence to be stitched is a group of still images obtained after preprocessing, for example, the group 1 has a1 video and a2 video, the size of the image extracted from the a1 video is 1000x1000 pixels, the size of the image extracted from the a2 video is 700x700, and the a1 image and the a2 image need to be unified in image size to facilitate subsequent operations.
The working principle and the beneficial effects of the technical scheme are as follows: and a proper amount of image frames are extracted from each acquired video for processing, so that unnecessary repeated or redundant images can be reduced, the correct sequence of the images during splicing can be ensured by sequencing in time sequence, and a foundation is provided for subsequent image processing and editing.
Example 5:
on the basis of the above embodiment 1, the present embodiment provides a method for video stitching of a fully-mechanized coal mining face, in step 3, the method includes the steps of:
performing pixel point alignment processing on the still images extracted from the same acquired video in the same grouped video to acquire multi-feature images under the corresponding acquired video, wherein each acquired video corresponds to one multi-feature image;
splicing and edge fusion are carried out on all the multi-feature images under the same grouping video to obtain a target fusion image;
calculating the gradient of each pixel point in the target fusion image, and constructing a gradient set of the target fusion image:let the target fusion image have n pixels in total, < >>Represents the gradient of the ith 11 pixel point, and the value range of i11 is [1, n]；
Taking a point s of the target fusion image as the point s, and selecting surrounding 3x3 pixel points to construct a gradient covariance matrix：
The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing covariance under a point s-based gradient; />Representing covariance based on the upper left side gradient of point s; />Representing covariance at the left-hand gradient based on point s; />Representing covariance at the lower left gradient based on point s; />Representing covariance based on the upper right side gradient of point s; />Representing covariance based on the gradient to the right of point s; />Representing covariance at the lower right gradient based on point s; />Representing covariance based on the side gradient above point s; />Representing covariance at the gradient of the lower side based on the point s;
based on gradient covariance matrixCalculate +.>Retention value of each point:
the method comprises the steps of carrying out a first treatment on the surface of the Wherein r represents->A reserved value of the corresponding point in (a); />Representation ofCovariance of corresponding points in (a); />Representation->Maximum of all covariances in (a); />Representation->Average of all covariances in (a); />Covariance matrix representing corresponding points +.>Is a modulus of (2); />Representing covariance matrix->Is a modulus of (2); max represents the maximum value symbol;
determining a gradient covariance matrixAnd taking the points corresponding to the reserved values larger than the average value as feature points, so as to obtain all feature points on the target feature image, and further obtain grouping target features.
In this embodiment, the pixel alignment process refers to aligning all still images in the same group video with a predefined grid or coordinate system, and the pixel refers to the smallest display unit in an image, which is one of the constituent elements of the image.
In this embodiment, the multi-feature image is a superimposed image obtained by processing the same group of video through pixels.
In this embodiment, edge blending is the seamless merging of edges of multiple feature images under the same group to create an overall consistent image.
In this embodiment, the target fusion image is the final image obtained by stitching and edge fusion of all the multi-feature images under the same group video.
In this embodiment, the gradient of the pixel points refers to the gray scale rate of change at each pixel location in the image. The degree of change in brightness or color of the image at that location is shown.
In this embodiment, the reserved value is a value in which the neighborhood around the current pixel point represents the characteristic of the pixel point by calculation.
In this embodiment, the feature point is to average all the retention values of 8 points in the pixel area, and if the retention value of the current pixel is greater than the average value, it is determined that the current pixel is the feature point.
The working principle and the beneficial effects of the technical scheme are as follows: the pixel points of the images under the videos in the group are processed through superposition, the multi-feature images of the videos are spliced and fused, the fusion image of the current group is extracted, the group target feature of the current group is obtained through gradient calculation based on the fusion image of the group, and a foundation is provided for subsequent video splicing.
Example 6:
on the basis of the above embodiment 1, the present embodiment provides a video stitching method for a fully-mechanized coal mining face, in step 3, the normalizing processing is performed on the images in the corresponding image sequences to be stitched according to the grouping target features, including:
performing perspective change processing on multi-feature images under the same grouping video to obtain a perspective change matrix, substituting the grouping target features into the perspective change matrix, and obtaining a standard conversion function of the corresponding multi-feature images;
converting the standard conversion function with the isoframe still images under the corresponding acquired video in the images to be spliced under the same group to obtain a first image sequence of the corresponding acquired video;
and collecting the first image sequences to obtain standard image sequences under corresponding groups.
In this embodiment, the perspective change process is an image processing technique for mapping an image on one plane onto another plane and performing shape warping and transformation according to a perspective relationship so that an original image can be completely projected onto the other plane.
In this embodiment, the perspective change matrix is a mathematical representation, typically a 3x3 matrix, for performing perspective transformation, and is a calculation matrix generated based on the characteristics of the perspective transformation that preserve linearity.
In this embodiment, the standard transformation function is a function of normalizing the group target feature into a specific form by substituting the group target feature into the perspective transformation matrix, and the function represents the perspective change calculation function of the current group, so as to provide a basis for subsequent image transformation.
In this embodiment, the first image sequence is obtained by performing perspective change processing-standard conversion function processing operation on all still images under the corresponding acquired video in the corresponding images to be spliced under the same group, that is, the standard conversion function is obtained after determining the group target feature under the corresponding group, where all feature conditions may exist, for example, the feature point 1 is characterized by y1, the converted function is r1, and at this time, the corresponding standard conversion function is: r1=zh (y 1, 1), where zh (y 1, 1) is a conversion function with a characteristic y1 for the feature point 1, and then the standard conversion function is used as an image conversion basis to obtain a first image sequence, that is, the image is converted first and then numbered, so as to obtain the first image sequence.
The working principle and the beneficial effects of the technical scheme are as follows: by applying the multi-feature images to the same perspective transformation matrix, the consistency of the images in space can be ensured, and the influence of factors such as different visual angles, distortion or scale difference on the images is eliminated, so that the images are more comparable.
Example 7:
on the basis of the above embodiment 1, the present embodiment provides a method for video stitching of a fully-mechanized coal mining face, in step 4, video stitching is performed based on all standard image sequences, and a stitched video of the fully-mechanized coal mining face is obtained, including:
performing edge matching on the kth standard image of the left video and the kth standard image of the right video which have adjacent relations in the same group to generate a transformation matrix;
fusing an original video image corresponding to a kth standard image of the left video and a kth standard image of the right video by utilizing a transformation matrix to obtain a kth frame fused image of the belonging packet spliced video;
and splicing all the frame fusion images under all the groups to obtain a spliced video of the fully-mechanized coal mining face.
In this embodiment, edge matching may identify and match edges or contours of objects in the image. And (3) finding similar edge features in two adjacent standard images through edge matching, and matching the similar edge features.
In this embodiment, the transformation matrix is a geometric matrix in which edge variation is obtained after edge matching is performed on two adjacent standard images.
The working principle and the beneficial effects of the technical scheme are that through edge matching and fusion operation, the discontinuity and dislocation between left and right images in the spliced video can be reduced, so that the image quality is improved, and the condition of the fully-mechanized coal face of the coal mine can be observed more clearly. And the left and right images are converted more smoothly based on the change matrix, and the pictures of the spliced video are more coherent.
Example 8:
on the basis of the above embodiment 1, the present embodiment provides a method for video stitching of a fully-mechanized coal mining face, where the process of performing edge matching on a kth standard image of a left video and a kth standard image of a right video in adjacent relation in the same group further includes:
image alignment processing is carried out on the kth standard image of the left video and the kth standard image of the right video, equidistant edge points are screened on the left edge line and the right edge line, a point array is constructed, and a linear change sequence based on the point array is obtainedWherein->A linear value representing the j01 th column; />A value representing a first point of a j01 th column in the point array; />A value representing a second point of the j01 th column in the point array; />Representing the standard value of the j01 th column in the point array; m01 represents the number of columns included in the corresponding point array, and each column includes an edge point pair;
according to the linear change sequence, determining an error coefficient of edge matching:
the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing the corresponding error coefficient; m02 represents +.>A number that is not within a preset range; />A difference function representing a linear value that is not within a preset range;a difference function representing the corresponding point array; />Representing the jth 02 linear value which is not within a preset range;
and adjusting edge point pairs which are not in a preset range based on the error coefficient, so as to realize subsequent edge matching.
In this embodiment, equidistant edge points refer to key pixel points distributed at equidistant intervals on left and right edge lines in two adjacent still images.
In this embodiment, the dot array refers to a sequence of a set of pixel points obtained by equidistant edge point screening on the left edge line and the right edge line of two adjacent still images.
In this embodiment, the linear change sequence refers to the corresponding point arrays obtained according to the two images, and there is a linear change relationship between the two corresponding edge points, so the linear change relationship between the two corresponding edge points of all the point arrays forms a linear change sequence.
In this embodiment, the linear value represents the degree to which two corresponding edge points of the linear variation sequence gradually vary along a linear relationship.
In this embodiment, the edge point pair is a point group constituted by two edge points representing that there is a correspondence relationship in the point group.
In this embodiment, the error coefficient is the degree of error between the quantized matching edge point result and the true edge.
In this embodiment, the difference function is a function representing the difference between two linear values.
The working principle and the beneficial effects of the technical scheme are as follows: through edge matching and error calculation, gaps at the splicing positions of left and right videos can be effectively eliminated, so that spliced videos are smoother and more continuous in vision, the problems of information faults and visual incoherence caused by splicing gaps are avoided, and the problems of information loss caused by video discontinuities, such as video monitoring, operation guidance and fault elimination, are reduced.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (8)
1. The video splicing method for the fully-mechanized coal mining face is characterized by comprising the following steps of:
step 1: video acquisition is carried out on the fully mechanized mining face of the coal mine according to the pre-deployment multi-camera, and video grouping processing is carried out on the acquired video according to the deployment position of the pre-deployment multi-camera;
step 2: frame extraction is carried out on each group of videos, and each frame of extracted image is numbered to obtain an image sequence to be spliced of each corresponding group, wherein the frame number of frame extraction of each group of videos is the same;
step 3: establishing grouping target characteristics of the multi-camera at the deployment position, and carrying out standardized processing on images in the corresponding image sequences to be spliced according to the grouping target characteristics to obtain standard images;
step 4: and video stitching is carried out based on all the standard images, and the stitched video of the fully-mechanized coal mining face of the coal mine is obtained.
2. The method for video stitching of a fully-mechanized coal mining face according to claim 1, wherein in step 1, video acquisition is performed on the fully-mechanized coal mining face according to pre-deployment of a plurality of cameras, and the method comprises the following steps:
based on the basic environment condition of the fully-mechanized mining face of the coal mine and the region needing coverage shooting, an initial deployment position and a shooting angle of the multi-camera are drawn, and a serial number corresponding relation of the initial deployment position and the multi-camera of each initial deployment position is established;
and carrying out video acquisition on the fully-mechanized coal mining face by using a plurality of cameras deployed based on the corresponding relation of the numbers.
3. The method for video stitching of a fully-mechanized coal mining face according to claim 2, wherein in step 1, video grouping processing is performed on the acquired video, including:
according to the corresponding relation of the numbers of each working area of the fully-mechanized coal mining face, all acquired videos are subjected to grouping processing;
and obtaining the grouping video of the corresponding working area according to the grouping processing result.
4. The method for video stitching of a fully-mechanized coal mining face according to claim 1, wherein in step 2, frame extraction is performed on each grouping video, and each frame of extracted image is numbered to obtain a sequence of images to be stitched of each corresponding grouping, and the method comprises the steps of:
decoding each group of videos into an original image sequence by using a video decoder, wherein the original image sequence is formed by unequal image frame amounts under a plurality of acquired videos;
determining average duration according to the video duration of each acquired video in each grouped video, and determining the extraction frame number of the current group according to an average-extraction frame table;
extracting a plurality of images in the same acquired video according to the time interval of the corresponding acquired video to the original image sequence under the same grouped video according to the extraction frame number to obtain a still image, and preprocessing the extracted still image to obtain an initial image sequence to be spliced, wherein the initial image sequence to be spliced is formed by equal image frame amounts under a plurality of acquired videos;
numbering each frame of image in the initial image sequence to be spliced according to a time logic sequence to obtain the image sequence to be spliced of the target acquisition video.
5. The method for video stitching of a fully-mechanized coal mining face according to claim 4, wherein in step 3, establishing a grouping target feature in a deployment position of the multi-camera comprises:
performing pixel point alignment processing on the still images extracted from the same acquired video in the same grouped video to acquire multi-feature images under the corresponding acquired video, wherein each acquired video corresponds to one multi-feature image;
splicing and edge fusion are carried out on all the multi-feature images under the same grouping video to obtain a target fusion image;
calculating the gradient of each pixel point in the target fusion image, and constructing a gradient set of the target fusion image:let the target fusion image have n pixels in total, < >>Represents the gradient of the ith 11 pixel point, and the value range of i11 is [1, n]；
Taking a point s of the target fusion image as the point s, and selecting surrounding 3x3 pixel points to construct a gradient covariance matrix：
The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing covariance under a point s-based gradient; />Representing covariance based on the upper left side gradient of point s; />Representing covariance at the left-hand gradient based on point s; />Representing covariance at the lower left gradient based on point s; />Representing covariance based on the upper right side gradient of point s; />Representing covariance based on the gradient to the right of point s; />Representing covariance at the lower right gradient based on point s; />Representing covariance based on the side gradient above point s; />Representing covariance at the gradient of the lower side based on the point s;
based on gradient covariance matrixCalculate +.>Retention value of each point:
the method comprises the steps of carrying out a first treatment on the surface of the Wherein r represents->A reserved value of the corresponding point in (a); />Representation->Covariance of corresponding points in (a); />Representation->Maximum of all covariances in (a); />Representation->Average of all covariances in (a); />Covariance matrix representing corresponding points +.>Is a modulus of (2); />Representing covariance matrix->Is a modulus of (2); max represents the maximum value symbol;
determining a gradient covariance matrixAnd taking the points corresponding to the reserved values larger than the average value as feature points, so as to obtain all feature points on the target feature image, and further obtain grouping target features.
6. The method for video stitching of a fully-mechanized coal mining face according to claim 1, wherein in step 3, the image in the corresponding image sequence to be stitched is normalized according to the grouping target feature, including:
performing perspective change processing on multi-feature images under the same grouping video to obtain a perspective change matrix, substituting the grouping target features into the perspective change matrix, and obtaining a standard conversion function of the corresponding multi-feature images;
converting the standard conversion function with the isoframe still images under the corresponding acquired video in the images to be spliced under the same group to obtain a first image sequence of the corresponding acquired video;
and collecting the first image sequences to obtain standard image sequences under corresponding groups.
7. The method for video stitching of a fully-mechanized coal mining face according to claim 1, wherein in step 4, video stitching is performed based on all standard image sequences to obtain stitched video of the fully-mechanized coal mining face, and the method comprises the following steps:
performing edge matching on the kth standard image of the left video and the kth standard image of the right video which have adjacent relations in the same group to generate a transformation matrix;
fusing an original video image corresponding to a kth standard image of the left video and a kth standard image of the right video by utilizing a transformation matrix to obtain a kth frame fused image of the belonging packet spliced video;
and splicing all the frame fusion images under all the groups to obtain a spliced video of the fully-mechanized coal mining face.
8. The method for video stitching on a fully-mechanized coal mining face according to claim 7, wherein in the process of performing edge matching on a kth standard image of a left video and a kth standard image of a right video in adjacent relation in the same group, the method further comprises:
image alignment processing is carried out on the kth standard image of the left video and the kth standard image of the right video, equidistant edge points are screened on the left edge line and the right edge line, a point array is constructed, and a linear change sequence based on the point array is obtainedWherein->A linear value representing the j01 th column; />A value representing a first point of a j01 th column in the point array; />A value representing a second point of the j01 th column in the point array; />Representing the standard value of the j01 th column in the point array; m01 represents the number of columns included in the corresponding point array, and each column includes an edge point pair;
according to the linear change sequence, determining an error coefficient of edge matching:
the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing the corresponding error coefficient; m02 represents +.>A number that is not within a preset range; />Representing differences in linear values that are not within a preset rangeA function; />A difference function representing the corresponding point array; />Representing the jth 02 linear value which is not within a preset range;
and adjusting edge point pairs which are not in a preset range based on the error coefficient, so as to realize subsequent edge matching.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311302439.5A CN117319747A (en) | 2023-10-09 | 2023-10-09 | Video splicing method for fully-mechanized mining face of coal mine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311302439.5A CN117319747A (en) | 2023-10-09 | 2023-10-09 | Video splicing method for fully-mechanized mining face of coal mine |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117319747A true CN117319747A (en) | 2023-12-29 |
Family
ID=89296882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311302439.5A Pending CN117319747A (en) | 2023-10-09 | 2023-10-09 | Video splicing method for fully-mechanized mining face of coal mine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117319747A (en) |
-
2023
- 2023-10-09 CN CN202311302439.5A patent/CN117319747A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103517041B (en) | Based on real time panoramic method for supervising and the device of polyphaser rotation sweep | |
| CN102132323B (en) | System and method for automatic image straightening | |
| CN112365404B (en) | Contact net panoramic image splicing method, system and equipment based on multiple cameras | |
| JP2004515832A (en) | Apparatus and method for spatio-temporal normalization matching of image sequence | |
| KR101759798B1 (en) | Method, device and system for generating an indoor two dimensional plan view image | |
| EP1999685B1 (en) | Method of and system for storing 3d information | |
| CN105554447A (en) | Image processing technology-based coal mining face real-time video splicing system | |
| CN104392416A (en) | Video stitching method for sports scene | |
| EP0866606B1 (en) | Method for temporally and spatially integrating and managing a plurality of videos, device used for the same, and recording medium storing program of the method | |
| CN103986854A (en) | Image processing apparatus, image capturing apparatus, and control method | |
| CN109919038A (en) | Power distribution cabinet square pressing plate state identification method based on machine vision and deep learning | |
| CN111524083A (en) | Active and passive combined underwater aerial imaging image recovery method based on structured light | |
| CN110120012B (en) | Video stitching method for synchronous key frame extraction based on binocular camera | |
| CN114973028A (en) | Aerial video image real-time change detection method and system | |
| US6181815B1 (en) | Subject image extraction device | |
| KR20230065014A (en) | Automated analysis method and apparatus of fittings in power equipment using keyframe | |
| CN117319747A (en) | Video splicing method for fully-mechanized mining face of coal mine | |
| CN105404127A (en) | Holographic imaging system and imaging method for taking photos to collect three-dimensional information | |
| CN116051681B (en) | Processing method and system for generating image data based on intelligent watch | |
| CN116132636A (en) | Video splicing method and device for fully-mechanized coal mining face | |
| CN116091506A (en) | Machine vision defect quality inspection method based on YOLOV5 | |
| CN116132610A (en) | Fully-mechanized mining face video stitching method and system | |
| CN112287787B (en) | Crop lodging grading method based on gradient histogram characteristics | |
| CN115035466A (en) | Infrared panoramic radar system for safety monitoring | |
| CN110658844B (en) | Ultra-high voltage direct current line channel unmanned aerial vehicle monitoring method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |