CN116233477B - Live broadcast real-time transmission image preprocessing system based on algorithm and application - Google Patents
Live broadcast real-time transmission image preprocessing system based on algorithm and application Download PDFInfo
- Publication number
- CN116233477B CN116233477B CN202310221381.5A CN202310221381A CN116233477B CN 116233477 B CN116233477 B CN 116233477B CN 202310221381 A CN202310221381 A CN 202310221381A CN 116233477 B CN116233477 B CN 116233477B
- Authority
- CN
- China
- Prior art keywords
- image
- dictionary
- sparse
- video
- shooting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007781 pre-processing Methods 0.000 title claims abstract description 27
- 230000005540 biological transmission Effects 0.000 title claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 31
- 238000004891 communication Methods 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 claims abstract description 8
- 239000013598 vector Substances 0.000 claims description 27
- 230000006870 function Effects 0.000 claims description 23
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000005516 engineering process Methods 0.000 claims description 12
- 230000014509 gene expression Effects 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 claims description 6
- 239000000654 additive Substances 0.000 claims description 5
- 230000000996 additive effect Effects 0.000 claims description 5
- 238000013075 data extraction Methods 0.000 claims description 5
- 238000013178 mathematical model Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 3
- 230000015556 catabolic process Effects 0.000 claims 1
- 238000006731 degradation reaction Methods 0.000 claims 1
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 230000006855 networking Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4334—Recording operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Image Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The live broadcast real-time transmission image preprocessing system based on the algorithm comprises an image video shooting module, a communication module, an image data preprocessing module and an output and display module, wherein the image video shooting module finishes shooting of an image or recording of a video by using a mobile camera, the communication module uploads the image or the video to a data processing computer, the image data preprocessing module processes and analyzes the image or the video data, and finally a result is displayed by the output and display module. The invention has the beneficial effects that: the mobile shooting device is free to hold, images or videos shot by a user can be uploaded in real time in a wireless transmission mode, preprocessing of data is completed, fine analysis is further performed on the images for subsequent researchers, efficiency is improved, and rapid statistics is performed to achieve classification of image data.
Description
Technical Field
The invention belongs to the field of camera shooting technology and network live broadcast, and particularly relates to a live broadcast real-time transmission image preprocessing system based on an algorithm.
Background
Along with the gradual acceleration of the life pace of people in the development of science and technology, the continuous alternation of the internet industry opens up a high-speed channel for our life, the requirements of people in the fast rhythm at present can not be met by traditional shooting and image processing services, the local huge memory space of resources can be occupied by images and videos shot by photographers along with the accumulation of quantity, and overstocked images and videos can bring excessive burden in the later image processing work of the photographers. The online transmission of images is a hot topic in the present era, and a MESH network technology is adopted, so that a wireless MESH network has evolved into an effective solution suitable for a wide-band home network, a community network, an enterprise network, a metropolitan area network and other wireless access networks by virtue of multi-hop interconnection and MESH topology characteristics. The wireless Mesh routers form an ad hoc network in a multi-hop interconnection mode, so that higher reliability, wider service coverage and lower early investment cost are provided for WMN networking.
Disclosure of Invention
Aiming at the problems, the invention aims to provide an algorithm-based live broadcast real-time transmission image preprocessing system.
The aim of the invention is realized by the following technical scheme:
the live broadcast real-time transmission image preprocessing system based on the algorithm comprises an image video shooting module, a communication module, an image data preprocessing module and an output and display module, wherein the image video shooting module finishes shooting of images or recording of videos by using a mobile camera, the mobile camera is provided with a wireless network card, wireless network communication is established through a connection hot spot, the communication module has network authority, a photographer can open or close the network authority according to requirements, if the network authority is opened, the images or videos shot by the photographer are uploaded in real time, if the network authority is closed, the images or videos shot by the photographer are stored in a local memory card, the communication module builds a communication bridge between the mobile camera and a data processing computer, the images or videos shot in real time in shooting are uploaded in the data processing computer, the image data preprocessing module processes and analyzes the images or videos, the frame data of the videos are extracted and restored, the result is displayed through the output and display module, and a user can access the data processing computer to check the preprocessed image information so as to pave the deeper image processing.
Further, the image video shooting module finishes shooting of images or recording of videos by using a mobile camera, the mobile camera is provided with a wireless network card, wireless network communication is established through a connection hotspot, a communication module has network authority, a photographer can open or close the network authority according to requirements, if the network authority is opened, the images or videos shot by the photographer are uploaded in real time, and if the network authority is closed, the images or videos shot by the photographer are stored in a local memory card.
Further, if the network authority is opened in shooting, an image or video shot by a photographer is uploaded in real time, a communication bridge between the mobile camera and the data processing computer is built by the communication module, the image or video shot in real time in shooting is uploaded to the data processing computer by utilizing the MESH technology, wherein the MESH technology can realize high-speed data communication in a relatively close range, and the effective bandwidth can reach 6 Mbit/s by utilizing the 2.4 GHz frequency band.
Furthermore, the image or video data is processed and analyzed by the image data preprocessing module, including frame data extraction of video and restoration of image data, because the image may be affected by the inside and outside of the mobile shooting device, environment, etc. during shooting, the image data is interfered by additive noise, and the shooting scene needs to be preprocessed by using the image restoration technology, the image restoration problem can be regarded as: often assume thatWhite noise and uncorrelated with the image, based on degraded imageAnd degenerate operator->Is to solve the original image in reverse>Expressed as a mathematical model:
for "noise" in a photographed scene, which is a variety of factors that prevent the visual system from perceiving and understanding the received image information, noise in the image is generally treated as additive noise, and then the noise image model may be expressed as:
wherein:for noise images, each frame of image signals in a shot video can be accurately represented through a small number of basis functions by using a sparse matrix method, a set of the basis functions is called a dictionary, each basis function in the dictionary is called an atom, in order to obtain higher sparsity, an image block is taken as an atom to construct the dictionary, and the image signals are projected on the dictionary, so that a good effect is obtainedSparse effect. The expression is: />Wherein: />A signal that is sparsely represented; />Is the->An atom; />Is the%>A coefficient vector; />In an image sparse representation model, because the image size is large, the image is generally divided into small blocks, the small blocks are represented by column vectors, and then the small blocks are respectively processed, so that the image coefficients are sparse as much as possible, and the following formula is defined:
wherein:for each frame of live video +.>Middle image block->Sparse dictionary corresponding to the whole image +.>For image block->Corresponding sparse coefficients, ++>For image block->Is introduced into the regularization parameter +.>Solving a sparse coding problem of each image block about a sparse dictionary as shown in the following formula: />
At the heart of the sparse matrix is a dictionaryUsing a global dictionary learning K-SVD algorithm to find a best set of bases to enable images to be sparsely represented, the objective function of the K-SVD algorithm is expressed as:
wherein: dictionary and coefficient matrix can not be convex at the same time, and sparse representation error is carried outSub singular value decomposition with simultaneous updating of dictionary atoms and coefficients ++>The mathematical model representing the number of decompositions, the sparse representation error, can be expressed as:
wherein,is->Set of->Is->Set of->Is F-norm>Error, expressed asAnd Λ is a unit array, dictionary atoms are updated by using a left vector U, coefficient vectors are updated by using a right vector V, and meanwhile, atoms and the coefficient vectors are updated, so that the error of each step is minimized, and the iteration efficiency is improved.
Further, each frame of image of the live video is divided into different pixel characteristicsClass, and training a dictionary for each class, get +.>A local dictionary, assuming each class +.>Corresponds to a sparse dictionary->For->There is a sharp image patch +.>And original image tiles->Minimal error between the two, sparse dictionary and mathematical expression of coefficient vectorThe following are provided: />
Adopting an iterative solving method to set initial dictionary values asCoefficient vector +.>Then fix +.>Returning to find dictionary->The optimal dictionary can be finally obtained through repeated iteration, and in order to avoid the great influence of initial values on the optimal of the final dictionary, an expression of the relation between the dictionary and the coefficients is defined: />
Representing a substituted dictionary D into the above-mentioned dictionary：
In the middle ofIs->Each frame of image matrix of live video in each local dictionary, and weight values are defined>The similarity between pixel characteristic structures is represented, so weight is added in the solution of the sparse coefficient, and the coefficient solution formula is expressed as follows:
further, the results are displayed through the output and display module, and the user can access the data processing computer to view the preprocessed image information so as to lay down for deeper image processing research.
The invention has the beneficial effects that: the live broadcast real-time transmission image preprocessing system based on the algorithm is characterized in that the shooting of an image or the recording of a video is completed through a mobile camera, the mobile camera is convenient and portable, the requirements of a photographer can be completed at any time and in real time, and the mobile camera has network rights; supporting the functions of MeshController hot backup link, automatic roaming switching and the like; the centralized management of the MeshController user terminal and various verification modes are supported, so that the system is safer; the method supports the MeshController user flow control function, can freely distribute flow according to user types, and supports the functions of speed limit, flow limit, network time limit and the like. The invention provides an algorithm-based live broadcast real-time transmission image preprocessing system which can well improve the working efficiency of post-processing images of a photographer, can upload image information to a data processing computer for synchronous processing in real time in the shooting process, and is innovative in that each frame of image data in the shot video is accurately represented by a small amount of basis functionsThe set of basis functions is called a dictionary, each basis function in the dictionary is called an atom, in order to obtain higher sparsity, an image block is taken as an atomic construction dictionary, an image signal is projected on the dictionary, a good sparse effect is obtained, in an image sparse representation model, because the image size is large, an image is generally divided into small blocks, the small blocks are respectively processed after being expressed by column vectors, and regularization parameters are introducedSolving the sparse coding problem of each image block about a sparse dictionary, searching a best group of bases by using a global dictionary learning K-SVD algorithm so that the image can be sparsely represented, updating atomic and coefficient vectors simultaneously, minimizing errors of each step and improving iteration efficiency. The invention has the advantages that the shot image information can be transmitted to the data processing computer for preprocessing in real time in the shooting process, the precision analysis of the images is further carried out by subsequent researchers, the efficiency is improved, the statistics is fast carried out to realize the classification of the image data, and the invention brings convenience to the subsequent work in shooting.
Drawings
The invention will be further described with reference to the accompanying drawings, in which embodiments do not constitute any limitation on the invention, and other drawings can be obtained by one of ordinary skill in the art without undue effort from the following drawings.
Fig. 1 is a schematic diagram of the structure of the present invention.
Detailed Description
The invention is further described in connection with the following examples.
Referring to fig. 1, the invention aims to provide an algorithm-based live broadcast real-time transmission image preprocessing system, which comprises an image video shooting module, a communication module, an image data preprocessing module and an output and display module, wherein the image video shooting module finishes shooting of images or recording of videos by using a mobile camera, the mobile camera is provided with a wireless network card, wireless network communication is established through a connection hotspot, the communication module has network authority, a photographer can open or close the network authority according to requirements, if the network authority is opened, the images or videos shot by the photographer are uploaded in real time, if the network authority is closed, the images or videos shot by the photographer are stored in a local memory card, the communication module builds a communication bridge between the mobile camera and a data processing computer, the images or videos shot in real time in shooting are uploaded in the data processing computer, the image or video data is processed and analyzed by the image data preprocessing module, the result comprises frame data extraction of the videos and restoration of the image data is displayed by the output and display module, and the user can access the image information preprocessed by the data processing computer so as to pave the image processing research.
Preferably, the image video shooting module uses a mobile camera to complete shooting of an image or recording of a video, the mobile camera is provided with a wireless network card, wireless network communication is established through a connection hotspot, a communication module has network authority, a photographer can open or close the network authority according to requirements, if the network authority is opened, the image or the video shot by the photographer is uploaded in real time, and if the network authority is closed, the image or the video shot by the photographer is stored in a local memory card.
Preferably, if the network authority is opened in shooting, the image or video shot by the photographer is uploaded in real time, the communication module builds a communication bridge between the mobile camera and the data processing computer, and the image or video shot in real time in shooting is uploaded to the data processing computer by utilizing the MESH technology, wherein the MESH technology can realize high-speed data communication in a relatively close range, and the effective bandwidth can reach 6 Mbit/s by utilizing the 2.4 GHz frequency band.
Specifically, the image or video data is processed and analyzed by the image data preprocessing module, including frame data extraction of video and restoration of image data, because the image may be affected by the inside and outside of the mobile shooting device, environment, etc. during shooting, the interference of additive noise is received, and the shooting scene needs to be preprocessed by using the image restoration technology, the image restoration problem can be regarded as: often assume thatIs white noise and is uncorrelated with the image, according to the degraded image +.>And degenerate operator->Is to solve the original image in reverse>Expressed as a mathematical model:
for "noise" in a photographed scene, which is a variety of factors that prevent the visual system from perceiving and understanding the received image information, noise in the image is generally treated as additive noise, and then the noise image model may be expressed as:
wherein:for noise images, each frame of image signals in a shot video can be accurately represented through a small number of basis functions by using a sparse matrix method, a set of the basis functions is called a dictionary, each basis function in the dictionary is called an atom, in order to obtain higher sparsity, an image block is taken as an atom to construct the dictionary, and the image signals are projected on the dictionary, so that a good sparse effect is obtained. The expression is: />Wherein: />A signal that is sparsely represented; />Is the->An atom; />Is the%>A coefficient vector; />In an image sparse representation model, because the image size is large, the image is generally divided into small blocks, the small blocks are represented by column vectors, and then the small blocks are respectively processed, so that the image coefficients are sparse as much as possible, and the following formula is defined:
wherein:for each frame of live video +.>Middle image block->Sparse dictionary corresponding to the whole image +.>For image block->Corresponding sparse coefficients, ++>For image block->Is introduced into the regularization parameter +.>Solving a sparse coding problem of each image block about a sparse dictionary as shown in the following formula: />
At the heart of the sparse matrix is a dictionaryUsing a global dictionary learning K-SVD algorithm to find a best set of bases to enable images to be sparsely represented, the objective function of the K-SVD algorithm is expressed as:
wherein: dictionary and coefficient matrix can not be convex at the same time, and sparse representation error is carried outSub singular value decomposition with simultaneous updating of dictionary atoms and coefficients ++>The mathematical model representing the number of decompositions, the sparse representation error, can be expressed as:
wherein,is->Set of->Is->Set of->Is F-norm>Error, expressed asAnd Λ is a unit array, dictionary atoms are updated by using a left vector U, coefficient vectors are updated by using a right vector V, and meanwhile, atoms and the coefficient vectors are updated, so that the error of each step is minimized, and the iteration efficiency is improved.
Preferably, each frame of image of the live video is divided into different pixel characteristicsClass, and training a dictionary for each class, get +.>A local dictionary, assuming each class +.>Corresponds to a sparse dictionary->For the followingThere is a sharp image patch +.>And original image tiles->The error between the two is minimum, and the mathematical expressions of the sparse dictionary and the coefficient vector are as follows: />
Adopting an iterative solving method to set initial dictionary values asCoefficient vector +.>Then fix +.>Returning to find dictionary->The optimal dictionary can be finally obtained through repeated iteration, and in order to avoid the great influence of initial values on the optimal of the final dictionary, an expression of the relation between the dictionary and the coefficients is defined: />
Representing a substituted dictionary D into the above-mentioned dictionary：
In the middle ofIs->Each frame of image matrix of live video in each local dictionary, and weight values are defined>The similarity between pixel characteristic structures is represented, so weight is added in the solution of the sparse coefficient, and the coefficient solution formula is expressed as follows:
preferably, the results are presented via an output and presentation module, and the user can access the data processing computer to view the pre-processed image information to pad further image processing studies.
The invention has the beneficial effects that: live broadcast real-time transmission image preprocessing system based on algorithm and through mobile cameraThe mobile video camera is convenient to complete shooting of images or recording of videos, the mobile video camera is convenient to carry, the requirements of a photographer can be met at any time in real time, the mobile video camera has network rights, shooting operation of the photographer can be uploaded to a data processing computer in real time for preprocessing pictures when the photographer opens the network rights, shooting operation of the photographer can not be uploaded in real time and can only be stored in a local memory card when the photographer closes the network rights, safety is brought to the photographer, personal privacy and special requirements of the user are enhanced, a communication module adopts an MESH technology, the technology has simple link design, flexible networking and convenient maintenance, the centralized mode management of MeshController is supported, terminal data does not need to be configured, and a solution is automatically generated; supporting the functions of MeshController hot backup link, automatic roaming switching and the like; the centralized management of the MeshController user terminal and various verification modes are supported, so that the system is safer; the method supports the MeshController user flow control function, can freely distribute flow according to user types, and supports the functions of speed limit, flow limit, network time limit and the like. The invention provides an algorithm-based live transmission image preprocessing system which can well improve the working efficiency of post-processing images of a photographer, can upload image information into a data processing computer for synchronous processing in real time in the shooting process, and is characterized in that each frame of image data in the shot video is accurately represented by a small number of basis functions, the set of basis functions is called a dictionary, each basis function in the dictionary is called an atom, in order to obtain higher sparsity, an image block is used as an atomic structure dictionary, image signals are projected on the dictionary to obtain good sparse effect, in an image sparse representation model, the image is generally divided into small blocks and is represented by column vectors, and the small blocks are processed respectively, and rule parameters are introducedSolving the sparse coding problem of each image block about a sparse dictionary, searching a best group of bases by using a global dictionary learning K-SVD algorithm so that the image can be sparsely represented, updating atomic and coefficient vectors simultaneously, minimizing errors of each step and improving iteration efficiency. The invention has the advantages that the shot image information can be transmitted to the data processing computer for preprocessing in real time in the shooting process, the precision analysis of the images is further carried out by subsequent researchers, the efficiency is improved, the statistics is fast carried out to realize the classification of the image data, and the invention brings convenience to the subsequent work in shooting.
Finally, it should be noted that the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the scope of the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (2)
1. The live broadcast real-time transmission image preprocessing system based on the algorithm is characterized by comprising an image video shooting module, a communication module, an image data preprocessing module and an output and display module; the image video shooting module is used for shooting images or recording videos by using a mobile camera, the mobile camera is provided with a wireless network card, wireless network communication is established through a connection hot spot, the communication module has network authority, a photographer opens or closes the network authority according to the requirement, if the network authority is opened, the images or videos shot by the photographer are uploaded in real time, if the network authority is closed, the images or videos shot by the photographer are stored in a local memory card, the communication module builds a communication bridge between the mobile camera and a data processing computer, the images or videos shot in real time in shooting are uploaded in the data processing computer, the image data preprocessing module processes and analyzes the image or video data, the frame data extraction of the video and the restoration of the image data are included, finally, the result is displayed through an output and display module, and a user checks the preprocessed image information through accessing the data processing computer;
if the network authority is opened in shooting, uploading an image or video shot by a photographer in real time; the communication module builds a communication bridge between the mobile camera and the data processing computer, and uploads the image or video shot in real time in shooting to the data processing computer by utilizing the MESH technology;
the specific algorithm of frame data extraction and image data restoration of the video is as follows:
assuming that n (x, y) is white noise and is uncorrelated with the image, the process of solving the original image f (x, y) in reverse from the form of the degraded image g (x, y) and the degradation operator H (), is expressed as:
g(x，y)＝H(f(x，y))+n(x，y)
for "noise" in a photographed scene, which is a variety of factors that prevent the vision system from perceiving and understanding the received image information, the noise in the image is treated as additive noise, and then the noise image model is expressed as:
X n (x，y)＝f(x，y)+n(x，y)
wherein: x is X n (x, y) is a noise image, each frame of image signal in a shot video can be accurately represented by a small amount of basis functions by using a sparse matrix method, a set of the basis functions is called a dictionary, each basis function in the dictionary is called an atom, in order to obtain higher sparsity, an image block is taken as an atom to construct the dictionary, and the image signal is projected on the dictionary, so that a sparse effect is obtained; the expression is: x= Σ i∈I d i a i Wherein: x is a sparsely represented signal; d, d i An ith atom in the sparse dictionary; a, a i Is the i-th coefficient vector in the coefficient matrix; i is a subscript set of all non-zero coefficients in a coefficient matrix, in an image sparse representation model, because the image size is large, the image is divided into small blocks, the small blocks are represented by column vectors, and then the small blocks are respectively processed, so that the image coefficients are sparse, and the following formula is defined:
min||a|| 0 s.t.||x-Da||≤ε
wherein: x is an image block in each frame of image X of the live video, D is a sparse dictionary corresponding to the whole image, a is a sparse coefficient corresponding to the image block X, epsilon is a representation error of the image block X, and a regularization parameter lambda is introduced to solve a sparse coding problem of each image block relative to the sparse dictionary, wherein the sparse coding problem is represented by the following formula:
the core of the sparse matrix is the selection of a dictionary D, a global dictionary learning K-SVD algorithm is utilized to find a best group of bases so that the image can be sparsely represented, and an objective function of the K-SVD algorithm is expressed as follows:
s.t.||a i || 0 ≤T 0
wherein: the dictionary and the coefficient matrix cannot be convex at the same time, k times of singular value decomposition is carried out on the sparse representation error, and simultaneously dictionary atoms and coefficients are updated, wherein k represents the decomposition times, and a mathematical model of the sparse representation error can be expressed as follows:
wherein Y is a set of Y, A is a set of a, |·|| F Is F-norm, E k Error, denoted E k ＝UΛV T And Λ is a unit array, dictionary atoms are updated by using a left vector U, coefficient vectors are updated by using a right vector V, and meanwhile, atoms and the coefficient vectors are updated, so that the error of each step is minimized, and the iteration efficiency is improved.
2. An algorithm-based live video real-time transmission image preprocessing system according to claim 1, wherein each frame of image of said live video is processed by different pixel specificationsDividing the signs into K classes, training a dictionary for each class to obtain K local dictionaries, and supposing that each class is omega K Corresponds to a sparse dictionary D (k) For omega K Each of the tiles in the image is provided with a clear image tileAnd original image patch y i The error between the two is minimum, and the mathematical expressions of the sparse dictionary and the coefficient vector are as follows:
adopting an iterative solving method to set the initial value of the dictionary as D (0) Obtaining coefficient vector a by minimizing the above formula, then fixing a, and returning to the dictionary D (1) The optimal dictionary can be finally obtained through repeated iteration, and in order to avoid the great influence of initial values on the optimal of the final dictionary, an expression of the relation between the dictionary and the coefficients is defined:
the dictionary D is expressed by substituting a into the dictionary D to directly obtain the dictionary D:
in the middle ofDefining a weight w for each frame of image matrix of live video in the K-th local dictionary ij The similarity between pixel characteristic structures is represented, so weight is added in the solution of the sparse coefficient, and the coefficient solution formula is expressed as follows:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310221381.5A CN116233477B (en) | 2023-03-09 | 2023-03-09 | Live broadcast real-time transmission image preprocessing system based on algorithm and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310221381.5A CN116233477B (en) | 2023-03-09 | 2023-03-09 | Live broadcast real-time transmission image preprocessing system based on algorithm and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116233477A CN116233477A (en) | 2023-06-06 |
| CN116233477B true CN116233477B (en) | 2023-12-01 |
Family
ID=86578473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310221381.5A Active CN116233477B (en) | 2023-03-09 | 2023-03-09 | Live broadcast real-time transmission image preprocessing system based on algorithm and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116233477B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101505434A (en) * | 2009-03-12 | 2009-08-12 | 浙江大学 | High resolution intelligent network camera array system having global synchronization function |
| CN102256127A (en) * | 2011-07-20 | 2011-11-23 | 东南大学 | Real-time video transmission system for multimode wireless communication network |
| CN107483892A (en) * | 2017-09-08 | 2017-12-15 | 北京奇虎科技有限公司 | Video data real-time processing method and device, computing device |
| CN107818325A (en) * | 2017-09-22 | 2018-03-20 | 浙江工业大学 | Image sparse method for expressing based on integrated dictionary learning |
| CN108491868A (en) * | 2018-03-12 | 2018-09-04 | 广东工业大学 | A kind of image processing method and device based on k-means cluster and dictionary learning |
| CN115086560A (en) * | 2022-06-20 | 2022-09-20 | 深圳星禾设计有限公司 | Image management optimization method of mobile terminal and mobile terminal |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9418318B2 (en) * | 2013-08-30 | 2016-08-16 | Siemens Aktiengesellschaft | Robust subspace recovery via dual sparsity pursuit |
-
2023
- 2023-03-09 CN CN202310221381.5A patent/CN116233477B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101505434A (en) * | 2009-03-12 | 2009-08-12 | 浙江大学 | High resolution intelligent network camera array system having global synchronization function |
| CN102256127A (en) * | 2011-07-20 | 2011-11-23 | 东南大学 | Real-time video transmission system for multimode wireless communication network |
| CN107483892A (en) * | 2017-09-08 | 2017-12-15 | 北京奇虎科技有限公司 | Video data real-time processing method and device, computing device |
| CN107818325A (en) * | 2017-09-22 | 2018-03-20 | 浙江工业大学 | Image sparse method for expressing based on integrated dictionary learning |
| CN108491868A (en) * | 2018-03-12 | 2018-09-04 | 广东工业大学 | A kind of image processing method and device based on k-means cluster and dictionary learning |
| CN115086560A (en) * | 2022-06-20 | 2022-09-20 | 深圳星禾设计有限公司 | Image management optimization method of mobile terminal and mobile terminal |
Non-Patent Citations (1)
| Title |
|---|
| 以图像分类为目标的字典学习算法;徐健 等;现代电子技术;第36卷(第2期);23-28 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116233477A (en) | 2023-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113242469B (en) | Self-adaptive video transmission configuration method and system | |
| CN110968426B (en) | Edge cloud collaborative k-means clustering model optimization method based on online learning | |
| CN109697697B (en) | Reconstruction method of spectral imaging system based on optimization heuristic neural network | |
| CN114092769B (en) | Transformer substation multi-scene inspection analysis method based on federal learning | |
| CN101009835A (en) | Background-based motion estimation coding method | |
| Zheng et al. | A high-efficiency compressed sensing-based terminal-to-cloud video transmission system | |
| CN112906047B (en) | Image privacy information protection system and method based on deep learning | |
| CN113850272A (en) | Local differential privacy-based federal learning image classification method | |
| Peng et al. | LVE-S2D: Low-light video enhancement from static to dynamic | |
| CN111901532B (en) | Video stabilization method based on recurrent neural network iteration strategy | |
| CN115375589B (en) | Model for removing image shadow and construction method, device and application thereof | |
| CN110223245B (en) | Method and system for processing blurred picture in sharpening mode based on deep neural network | |
| Monika et al. | Coefficient permuted adaptive block compressed sensing for camera enabled underwater wireless sensor nodes | |
| CN116579966B (en) | Image reconstruction method and device, electronic equipment and storage medium | |
| CN116233477B (en) | Live broadcast real-time transmission image preprocessing system based on algorithm and application | |
| CN116522988B (en) | Federal learning method, system, terminal and medium based on graph structure learning | |
| WO2023217138A1 (en) | Parameter configuration method and apparatus, device, storage medium and product | |
| Chen et al. | Inverse atmoshperic scattering modeling with convolutional neural networks for single image dehazing | |
| CN115936108A (en) | Knowledge distillation-based neural network compression method for multivariate time series prediction graph | |
| Tan et al. | Two-stage CNN model for joint Demosaicing and Denoising of burst Bayer images | |
| CN113222953B (en) | Natural image enhancement method based on depth gamma transformation | |
| CN113643212B (en) | Depth map noise reduction method based on map neural network | |
| CN113011324B (en) | Target tracking method and device based on feature map matching and super-pixel map sorting | |
| Wang et al. | Application of Image Denoising Method Based on Two-Way Coupling Diffusion Equation in Public Security Forensics | |
| Han et al. | Auto Complementary Exposure Control for High Dynamic Range Video Capturing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231113 Address after: Room 101, Building 9, No. 233 Huanshi Middle Road, Yuexiu District, Guangzhou City, Guangdong Province, 510000 Applicant after: Guangzhou Zhongyi Enstek Electronic Technology Co.,Ltd. Address before: 4th Floor, Comprehensive Service Center, Zhenxing Street Office, Chiping District, Liaocheng City, Shandong Province, 252100 Applicant before: Liaocheng Chiping Runde Survey and Mapping Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |