CN115908190A - Method and system for enhancing image quality of video image - Google Patents
Method and system for enhancing image quality of video image Download PDFInfo
- Publication number
- CN115908190A CN115908190A CN202211571852.7A CN202211571852A CN115908190A CN 115908190 A CN115908190 A CN 115908190A CN 202211571852 A CN202211571852 A CN 202211571852A CN 115908190 A CN115908190 A CN 115908190A
- Authority
- CN
- China
- Prior art keywords
- data
- filtering
- model
- filtering model
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000002708 enhancing effect Effects 0.000 title claims abstract description 18
- 238000001914 filtration Methods 0.000 claims abstract description 199
- 238000012549 training Methods 0.000 claims abstract description 46
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims description 83
- 238000012360 testing method Methods 0.000 claims description 28
- 230000002123 temporal effect Effects 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 description 19
- 238000004364 calculation method Methods 0.000 description 12
- 210000001519 tissue Anatomy 0.000 description 10
- 230000000694 effects Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000001727 in vivo Methods 0.000 description 5
- 210000004204 blood vessel Anatomy 0.000 description 4
- 230000001575 pathological effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000001356 surgical procedure Methods 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 238000002324 minimally invasive surgery Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 210000000115 thoracic cavity Anatomy 0.000 description 2
- 230000003313 weakening effect Effects 0.000 description 2
- 210000000683 abdominal cavity Anatomy 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002682 general surgery Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000011503 in vivo imaging Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 210000000936 intestine Anatomy 0.000 description 1
- 238000002357 laparoscopic surgery Methods 0.000 description 1
- 210000004185 liver Anatomy 0.000 description 1
- 210000004877 mucosa Anatomy 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a method and a system for enhancing the image quality of a video image, and relates to the technical field of medical image quality enhancement. The algorithm model establishment method comprises the following steps: firstly, making a data set, wherein the data set comprises original video data and positive sample image data, and acquiring the original video data to obtain training data; secondly, receiving training data by using a preprocessed statistical module, training a preset spatial filtering model and a preset time-domain filtering model, and outputting training image data after the training data is processed by the spatial filtering module and the time-domain filtering module in sequence; and finally, calculating the loss of the training image data and the positive sample image data, and correcting the parameters of the spatial filtering model and the time-domain filtering model. The image quality enhancement method adopted by the invention can greatly retain image details such as brightness, edges, details, dynamics and the like while reducing noise, so that the processed image can present more details in the body of a patient.
Description
Technical Field
The invention relates to the technical field of image quality enhancement of clinical medical images, in particular to a method and a system for enhancing the image quality of a video image.
Background
With the continuous progress of medical technology, judging pathology, observing lesion or performing surgery according to minimally invasive surgery imaging becomes one of the most common means in modern medicine.
For example, minimally invasive techniques based on endoscopic systems are widely used in various departments such as general surgery, obstetrics and gynecology, thoracic surgery, urology, otorhinolaryngology, and pediatrics, and endoscopes are indispensable diagnostic and surgical devices for medical use. The endoscope is generally used in internal cavities of a human body such as an abdominal cavity, a chest cavity, a uterine cavity and the like, the illumination condition is poor, the absorption capacity of human tissues, particularly blood, to natural light is strong, so that a detection sensitive element of the endoscope needs to obtain good imaging brightness by large gain, the image noise is large due to large gain, the imaging quality of an image is seriously influenced, the distinguishing and judging of various tissues of the human body in the operation process of a doctor are influenced, even misdiagnosis or missed diagnosis of the doctor can be caused, and particularly in the aspect of minimally invasive surgery, a clear internal environment needs to be provided for the doctor.
In addition, the shooting environment in the human body cavity is complex: 1. the shooting distance is short, the shadow area is easy to be too dark and has no details due to over-bright and even over-exposure and long shooting distance; 2. the scenes of picture motion are many; 3. the texture is fine but weak and not easily distinguishable from noise. The existing common noise reduction method has good noise reduction effect, but the processed image has smear and no details, and the processing speed is low, so that the clinical requirement cannot be met; or the processing speed is high, no smear is generated, but the noise cannot be removed completely, so that the detail and the noise cannot be distinguished, and the clinical requirement of a doctor cannot be met.
Through retrieval, two patents named as a multi-frame digital image denoising method based on spatial domain and temporal domain combined filtering, with the publication number of CN103606132B and named as a video denoising device and method combining the temporal domain and the spatial domain, with the publication number of CN102769722A, the method of denoising a video by combining the temporal domain and the spatial domain is proposed, but in the medical field, due to the complexity of the internal environment of a human body, the denoising method is not enough to satisfy the clinical requirements of doctors, the internal light problem of the human body, the complexity of human tissues, various blood vessels and the like, especially pathological images in the operation stage, once the details are not clear or the local brightness is not obvious, the operation effect of the doctors is easy to image.
Therefore, the present application is directed to enhancing the image quality of an in-vivo image, and particularly enhancing the image quality of a pathological image.
Disclosure of Invention
The purpose of the invention is as follows: based on the problems mentioned in the background art, the method and the system for enhancing the image quality of the in-vivo image are provided, and an improved spatial filtering model and an improved temporal filtering model are adopted to process the in-vivo video image data according to a preset sequence, so that the image quality content including details, brightness, edges and the like is enhanced, the image quality content is applied to clinic, a doctor can obtain clear internal images of a human body in real time, and then accurate judgment is made or an operation is accurately completed by combining the medical experience of the doctor.
The technical scheme is as follows: a method for enhancing the quality of a video image, comprising the steps of:
s1, using a data set, wherein the data set comprises original video data and positive sample image data; acquiring original video data to obtain training data;
s2, receiving training data by using a preprocessed statistical module, training a preset filtering algorithm model, and outputting training image data after the training data are processed by the filtering algorithm model;
s3, calculating loss of the training image data and the positive sample image data, correcting parameters of a filter algorithm model, and returning to the step S2 until preset training times are finished;
and outputting the trained filter algorithm model.
Further, the method also comprises the following steps:
s4, selecting a part of the data set as a test set, and obtaining test data from the test set;
s5, processing the test data by using the trained filtering algorithm model, and outputting test image data;
s6, calculating loss of the test image data and the positive sample image data, and comparing by using a preset threshold value;
when the loss of the test image data and the positive sample image data is not more than a threshold value, finishing training;
and when the loss of the test image data and the positive sample image data is larger than the threshold value, returning to the step S2.
Further, the preprocessing content of the statistical module in step S2 includes:
at least introducing a motion unit, a brightness statistical unit of each channel, a local detail statistical unit, an edge gradient unit and a local similarity unit of a front frame and a rear frame into a statistical module;
the filtering algorithm model preset in the step S2 at least comprises a spatial filtering model and a time domain filtering model.
Further, the step of constructing the spatial filtering model comprises:
s211, using the guided filtering as a frame, wherein the guided filtering formula is as follows:
Out(i,j,k)= λ*In(i,j,k)+(1-λ)*Mean(k)
wherein ,In(i,j,k)andOut(i,j,k)respectively representing the current frame format dataiLine and firstjInput and output of the column;krepresents a component, which can be R, G, and B;λrepresents the filtering weight of the spatial filtering model, and the weight range is [0,1 ]]；Mean (k)Representing sum components except for a central pointkThe mean of all pixels of the same color.
Further, the step of training the spatial filtering model in step S2 includes:
s212, the spatial filtering model interacts with a statistic module, and the spatial filtering model is trained by using at least one of a motion unit, a channel brightness statistic unit, a local detail unit and an edge gradient unit;
s33, updating the filtering weight of the spatial filtering modelλ。
Further, the step of constructing the temporal filtering model comprises:
s221, using the first-order low-pass filtering as a frame, and using a filtering formula as follows:
Out(i,j)= δ*In(i,j)+(1-δ)*In_previous (i,j)
wherein ,In(i,j) andOut(i,j)respectively representing the input and the output of the ith row and the jth column of the format data of the current frame of the image;In_previous (i,j) representing the first frame, and obtaining the first frame time domain processing data after the time domain filtering model processingiGo to the firstjColumn output results;δrepresenting time-domain filtering modesFilter weight of type (M), weight range is [0,1 ]]。
Further, the step of training the time-domain filtering model in step S2 includes:
s223, the time-domain filtering model interacts with a statistic module, and the time-domain filtering model is trained by using at least one of a channel brightness statistic unit, an edge gradient unit and a local similarity unit;
s224, updating the filtering weight of the time domain filtering modelδ。
Further, the step of processing the video image based on the spatial filtering model and the temporal filtering model comprises:
s7, setting a processing sequence of a spatial filtering model and a time domain filtering model;
the statistical module receives and stores the time domain processing data of the previous frame obtained after the image data of the previous frame is processed by the spatial filtering model and the time domain filtering model;
s8, inputting image data of a current video frame into a spatial filtering model for processing, and outputting spatial processing data of the current video frame; then simultaneously inputting the spatial domain processing data of the current frame and the time domain processing data of the previous frame into a time domain filtering model for processing, and outputting the time domain processing data of the current frame;
s9, a statistical module receives and stores time domain processing data of a current frame; and outputting the time domain processing data of the current frame as enhanced image data.
A system for enhancing the quality of a video image, based on the method for enhancing the quality of the video image, comprising:
the acquisition module is used for acquiring a video to obtain preset image data;
the statistical module at least comprises a motion unit, a luminance statistical unit of each channel, a local detail statistical unit, an edge gradient unit and a local similarity unit of front and back frames, and is used for training a spatial filtering model and a time-domain filtering model;
the image processing module at least comprises a spatial filtering module and a temporal filtering module; the spatial filtering module is set based on the spatial filtering model, and the time-domain filtering module is set based on the time-domain filtering model.
Furthermore, the acquisition module is used for acquiring image data of a current frame of the video, processing the image data of the current frame through the spatial filtering module, and outputting spatial processing data of the current frame; inputting the time domain processing data of the previous frame and the space domain processing data of the current frame into a time domain filtering module for processing to obtain the time domain processing data of the current frame, and finally outputting the time domain processing data; the time domain processing data of the current frame is cached to the statistical module, and is called through the statistical module when the statistical module is used.
Has the advantages that:
1. the image quality enhancement method adopted by the invention can greatly retain image details such as brightness, edges, details, dynamics and the like while reducing noise, so that the processed picture can present more details of the video image in the body of a patient, and the details can be transmitted to a doctor to help the doctor to give more accurate diagnosis and positioning, thereby providing better auxiliary action for the doctor;
2. according to the invention, a spatial filtering model and a time domain filtering model are improved and combined, so that compared with the traditional 3D noise reduction, the speed is high, no frame delay exists, and no smear exists;
3. the reference frame of the time domain noise reduction is the processing result of the previous frame after the spatial domain noise reduction and the time domain noise reduction, the noise of the reference frame is removed completely, and finally, the image data processing effect of the current frame is better and is enough for clinical images;
4. when the method is actually used, only one frame of reference frame needs to be cached, the reference frame is a comprehensive result of space domain and time domain noise reduction, and compared with time domain noise reduction of multi-frame calculation, the method has the advantages of higher calculation speed and better processing effect.
Drawings
FIG. 1 is a flow chart of spatial filtering model and temporal filtering model training in the present invention.
FIG. 2 is a flow chart of the inspection of the spatial filtering model and the temporal filtering model trained in the present invention.
Fig. 3 is a video image processing flow diagram of the present invention.
FIG. 4 is a flow chart of the statistical module correction factor calculation in the present invention.
FIG. 5 is a flow chart of an embodiment of the filter correction coefficient parameter training of the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
Example 1
Based on the problems mentioned in the background art, the image quality details, noise, brightness, and the like of in-vivo imaging are difficult to meet the clinical requirements of doctors, and this embodiment provides an algorithm model for enhancing the image quality of in-vivo video images, and this embodiment, in combination with an endoscope to acquire surgical images, further elaborates on the basis of a spatial filtering model and a temporal filtering model, wherein the algorithm model in this embodiment is constructed by the steps of:
s1, making a data set, wherein the data set comprises original video data and positive sample image data, taking an endoscope as an example, acquiring an operation adaptive image with clear scene, making a training data set, or adopting the existing operation related data set, and the data set comprises the original video data and the positive sample image data after image quality enhancement; acquiring original video data to obtain training data, for example, acquiring data of each frame of an original video, wherein an image grid is Bayer, a clear operating Bayer format image is used as a positive sample, and Gaussian and Poisson noise are added as negative samples to perform training; compared with the traditional RGB format noise reduction, the noise of Bayer format data is closer to the theoretical random noise model of the sensor, and the processing effect on Bayer format images is better theoretically.
S2, receiving training data by using a preprocessed statistical module, wherein the statistical module at least comprises a motion unit, a brightness statistical unit of each channel, a local detail statistical unit, an edge gradient unit and a front-frame and rear-frame local similarity unit, and trains a spatial filtering model of a preset spatial filtering module and a time-domain filtering model of a time-domain filtering module, wherein network models of the local detail statistical unit, the motion unit, the local detail statistical unit and the edge gradient unit are trained and used for correcting parameters of the spatial filtering model; the network models of the brightness statistical unit, the edge gradient unit and the front and rear frame local similarity unit of each channel are trained and used for correcting the parameters of the time-domain filtering model, and the training data in the embodiment are processed by the spatial filtering module and the time-domain filtering module in sequence to output training image data;
s3, calculating the loss of the training image data and the positive sample image data by using a loss function, correcting the parameters of the spatial filtering model and the time domain filtering model, returning to the step S2, and training for preset times; and outputting the trained spatial filtering model and the trained time domain filtering model.
Example 2
In embodiment 1, after the spatial filtering model and the temporal filtering model are trained, a test is performed on the models to ensure that the final algorithm model meets the requirement of enhancing the image quality, and in this embodiment, the step of detecting the trained models in the embodiment includes:
s4, when the data set is used, taking a part of the data set as a test set, wherein the test set comprises test image data and corresponding positive sample image data, and acquiring the test data in the test set;
s5, processing the test data according to a preset sequence by using the spatial filtering model and the time-domain filtering model trained in the embodiment 1, and outputting the test image data;
s6, calculating the loss of the test image data and the positive sample image data by using a loss function, wherein a threshold value is introduced in the embodiment and is compared by using a preset threshold value because the ideal value cannot be perfectly realized by the algorithm model;
when the loss of the test image data and the positive sample image data is not greater than a threshold value, finishing training;
when the loss of the test image data and the positive sample image data is greater than the threshold value, the method returns to step S2 in the embodiment, and continues to use the training data for training until the loss of the test image data and the positive sample image data is not greater than the threshold value.
In the parameter training process in the embodiment 1 and the embodiment 2, the operation video data of each scene is filtered and intercepted, and the operation video data comprises each brightness scene, a motion scene of each motion speed, a static scene, details, an edge scene and the like. In the training process, coefficients of a spatial filtering model and a temporal filtering model are obtained by respectively correcting coefficients of statistical motion coefficients of front and rear continuous frames of the surgical video, luminance information of each channel, local detail gradient, edge gradient, local similarity of the front and rear frames and the like. The training model in this embodiment includes, but is not limited to, a fully-connected neural network with more than 3 layers.
Example 3
In embodiment 1, the preprocessing content of the statistic module includes at least introducing a motion unit, a luminance statistic unit of each channel, a local detail statistic unit, an edge gradient unit, and a local similarity unit of previous and subsequent frames into the statistic module.
Example 4
Based on embodiments 1 to 3, this embodiment provides a spatial filtering model, which includes the following contents:
and S211, the spatial filtering takes the guide filtering as a frame, and the filtering is performed alternately according to the central color channel of the filter window. The guided filtering formula is:
Out(i,j,k)= λ*In(i,j,k)+(1-λ)*Mean(k)
wherein ,In(i,j,k)andOut(i,j,k)respectively representing the input and output of the ith row and the jth column of the current frame format data;krepresents a component, can beR、 GAndB(ii) a λ represents the filter weight of the spatial filtering model, with a weight range of [0,1%]；Mean (k)Representing sum components other than the central pointkThe mean of all pixels of the same color.
A Bayer format image of a current frame of a video is used as input and enters a spatial filtering module.
Example 5
Based on embodiments 1 to 3, this embodiment provides a spatial filtering model, which includes the following contents:
s221, using first-order low-pass filtering as a framework, wherein the adopted filtering formula is as follows:
Out(i,j)= δ*In(i,j)+(1-δ)*In_previous (i,j)
wherein ,In(i,j) andOut(i,j)respectively represent the current frame format data of the imageiGo to the firstjInput and output of the column;In_previous (i,j) the first frame is represented and processed by the time domain filtering module to obtain the first frame time domain processing dataiGo to the firstjColumn output results;δrepresents the filtering weight of the time-domain filtering model, and the weight range is [0,1 ]]。
In this embodiment, bayer pattern data of a current frame and output of a previous frame after passing through a spatial filtering module and a temporal filtering module may be used as input.
Example 6
At present, smear is generated due to the movement of the acquisition equipment or the movement of human tissues; the detail information of dark areas and bright areas in the human body is less; texture detail information such as tiny blood vessels is difficult to maintain; it is difficult to maintain the image edge gradient, etc. due to the above reasons, the spatial filtering is difficult to reach the clinical requirement when processing the image, and this embodiment improves the spatial filtering model, including the following steps:
s212, interacting the spatial filtering model with a statistical module, and training the spatial filtering model by using at least one of a motion unit, a channel brightness statistical unit, a local detail unit and an edge gradient unit;
s33, updating the filtering weight of the spatial filtering modelλFiltering weightλThe method at least comprises the weight of one of a motion unit, a channel brightness statistic unit, a local detail unit and an edge gradient unit.
In this embodiment, the four units may exist independently and be calculated; the calculation can also be performed according to a preset rule, the calculation sequence can be changed randomly, and the calculation can also be performed simultaneously, and the action contents of the four units in the implementation are as follows:
the motion unit is used for counting the motion coefficient of a current image pixel through comparison calculation of Bayer format images of front and back frames of a video, the motion coefficient is larger, the stronger the motion is, the filtering strength is stronger, otherwise, the motion coefficient is larger, the smaller the motion is, the weaker the filtering is, and the motion unit is adopted because the time domain filtering cannot be too strong in a region with stronger motion, otherwise, motion smear can be caused; conversely, in a region with weak motion, the time domain can be de-noised more strongly, the motion is weaker, the smear is not easy to occur, and the spatial filtering is weaker, and more details are kept. And balancing the noise reduction intensity of the spatial filtering and the temporal filtering on the motion region or the static region through the motion unit so as to achieve the purpose of optimal noise reduction.
For example, in the clinical use of an endoscope, a motion scene such as the movement of an endoscope by an endoscope hand or the movement of human tissues is inevitable, and the noise reduction of the associated motion unit can reduce the noise of instruments or human tissues in the motion process and simultaneously keep the edges clear without causing smear.
And each channel brightness statistical unit is mainly used for performing statistics on brightness information of each channel of a frame of a video and can be obtained by calculating a local area mean value or a weighted mean value. Because the detail information of the dark area and the bright area of the internal image of the human body is less, when the brightness value is smaller or larger, the filtering is stronger, the noise reduction is stronger, the general details under the condition of moderate brightness value can be more, the noise reduction is weaker, and the purpose of optimally reducing the noise of the image under different brightness is achieved through the brightness information of each channel according to the feeling of human eyes on different brightness scenes of the image.
For example, in the laparoscopic surgery, the thoracoscopic surgery and the like in clinic, a human body cavity is shot by an endoscope, tissues close to the endoscope are too bright, and remote or shadow areas are too dark, and the statistics of the brightness of each channel is closed to achieve that the noise of each distance scene is small, so that the visual requirements of doctors can be met.
And the local detail statistical unit is used for calculating the coefficient of the local detail statistical unit through the spatial distribution of the local region gradient, and if the detail is stronger, the noise reduction is weaker, and the texture detail information such as a small blood vessel and the like is kept as far as possible. For example to preserve texture details such as fine blood vessels, fat particles, etc. in the image frame.
And the edge gradient unit is used for carrying out gradient statistics on the local region gradient, the statistical method comprises but is not limited to carrying out gradient statistics by using methods such as a sobel gradient, a cany edge gradient, a laplace operator and the like, if the gradient is larger, the probability of representing the edge is higher, the noise reduction is weaker, otherwise, the probability of representing a flat region is higher, the gradient is smaller, and the noise reduction is stronger. This makes it possible to maintain image edge information as much as possible while removing flat area noise. By preserving image edge gradients, such as instrument edges, boundaries between tissues, etc., the surface noise is low, such as locally smooth liver surfaces, mucosa, intestines, fat, etc.
Example 7
On the basis of embodiment 6, in order to complement the spatial filtering model to achieve the optimal noise reduction, the present embodiment further improves the spatial filtering model, and includes the following steps:
s223, interacting the time-domain filtering model with the statistical module, and training the time-domain filtering module by using at least one of a channel brightness statistical unit, an edge gradient unit and a local similarity unit;
s224, updating a filtering weight delta of the time-domain filtering model, wherein the filtering weight delta at least comprises the weight of one of a channel brightness statistical unit, an edge gradient unit and a local similarity unit.
In the present embodiment, the three units can exist and calculate independently; the calculation can also be performed according to a preset rule, the calculation sequence can be changed randomly, and the calculation can also be performed simultaneously, and the action contents of the three units in the implementation are as follows:
the local similarity unit calculates the local similarity through output of Bayer format data of a current frame and a previous frame after being processed by the spatial filtering module and the time-domain filtering module, the method comprises but is not limited to a similarity calculation mode of a non-local mean value, time-domain filtering is weakened in a region with small similarity, motion smear is reduced, time-domain filtering is strengthened in a region with large similarity, noise is reduced, and the purpose of optimal noise reduction is achieved through complementary use with the spatial filtering;
the channel brightness statistical unit is used for enhancing the time-domain filtering in the region with overhigh or overlow brightness and weakening the time-domain filtering in the middle brightness scene;
and the edge gradient unit is used for enhancing the time domain filtering and effectively weakening the edge noise when the edge gradient is larger and the spatial filtering is weaker in order to keep the edge. Effectively improves the definition of the edges among the tissues of the human body and the instruments.
Example 8
Based on the spatial filtering model and the temporal filtering model improved in the above embodiments 1 to 7, this embodiment provides a method for processing the content of a video image by using the spatial filtering model and the temporal filtering model, including the following steps:
s7, the statistical module receives and stores the time domain processing data of the previous frame, which is obtained after the image data of the previous frame is processed by the spatial filtering module and the time domain filtering module;
s8, collecting image data of a current frame of the video, inputting the image data into a spatial filtering module for processing, and outputting spatial processing data of the current frame; simultaneously inputting the spatial domain processing data of the current frame and the time domain processing data of the previous frame into a time domain filtering module for processing, and outputting the time domain processing data of the current frame;
the time-domain processing data of the previous frame is data synchronously processed by a space-domain filtering model and a time-domain filtering model, the data is a reference frame, the details, brightness, color and the like of the reference frame are enhanced, the noise is removed completely, and the reference frame and the time-domain processing data of the current frame are simultaneously input into the time-domain filtering model for processing, so that the setting sequence of the space-domain filtering model and the time-domain filtering model in the embodiment further improves the denoising effect of the image;
s9, a statistical module receives and stores time domain processing data of a current frame; the time-domain processing data of the current frame is output as enhanced image data, and the output image data can reach the clinical operation of a doctor.
Example 9
Based on the spatial filtering model and the temporal filtering model provided above, this embodiment provides a system for enhancing image quality based on the method for enhancing image quality of the video image, which at least includes an acquisition module, a statistical module, and an image processing module, the acquisition module is used to acquire video and obtain preset image data, the statistical module at least includes a motion unit, a luminance statistical unit of each channel, a local detail statistical unit, an edge gradient unit, and a front-and-back frame local similarity unit, and is used to train the spatial filtering model and the temporal filtering model, the image processing module at least includes a spatial filtering module and a temporal filtering module, the spatial filtering module is set based on the spatial filtering model in the above embodiment, and the temporal filtering module is set based on the temporal filtering model in the above embodiment.
In this embodiment, the statistical module is configured to cache time-domain processing data of previous and subsequent frames, and in order to increase the calculation speed, the statistical module in this application preferably caches one frame, that is, the time-domain processing data of the previous and subsequent frames are replaced.
Example 10
On the basis of embodiments 1 to 9, this embodiment provides a processing procedure of an image quality enhancement system, which includes the following steps:
firstly, an acquisition module acquires image data of a current frame of a video;
secondly, processing the image data of the current frame by a spatial filtering module, and outputting spatial processing data of the current frame;
next, the time-domain processing data of the previous frame is called from the statistical module, the time-domain processing data of the previous frame and the spatial-domain processing data of the current frame are input to the time-domain filtering module for processing, and the time-domain processing data of the current frame is output as processed image data.
The processing process is used for real-time video processing and outputting high-quality video content, for example, real-time output of pathological images inside a patient during an operation.
Example 11
Unlike embodiments 1-10, for early pathological symptoms, because the lesion itself is small and there are tissue details, which are convenient for doctors to find and confirm, this embodiment is based on the above embodiments, and this embodiment provides an enhanced image enhancement method, which includes the following steps:
the acquisition module acquires the image data of the current frame, and sets the image data of the current frame into n parts, such as copying and the like;
constructing n parallel spatial filtering modules, and respectively receiving image data of corresponding previous frames;
and improving the corresponding time domain filtering model to further construct an improved time domain filtering module, wherein the improved time domain filtering module simultaneously receives the n image data and the image of the previous frame after the image is processed by the time domain filtering module, outputs the processed image of the current frame, and caches the processed image for processing and calling the next frame.
By the treatment of the embodiment, the micro focus is easier to be found and is suitable for early in vivo examination.
Although the present application has been described in conjunction with specific features and embodiments thereof, it will be evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and figures are merely exemplary of the present application as defined in the appended claims and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include such modifications and variations.
The above is only a preferred embodiment of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (10)
1. A method for enhancing the quality of a video image, comprising the steps of:
s1, using a data set, wherein the data set comprises original video data and positive sample image data; acquiring original video data to obtain training data;
s2, receiving training data by using a preprocessed statistical module, training a preset filtering algorithm model, and outputting training image data after the training data are processed by the filtering algorithm model;
s3, calculating loss of the training image data and the positive sample image data, correcting parameters of a filter algorithm model, and returning to the step S2 until preset training times are finished;
and outputting the trained filtering algorithm model.
2. The method of claim 1, further comprising the steps of:
s4, selecting a part of the data set as a test set, and obtaining test data from the test set;
s5, processing the test data by using the trained filter algorithm model, and outputting test image data;
s6, calculating the loss of the test image data and the positive sample image data, and comparing by using a preset threshold value;
when the loss of the test image data and the positive sample image data is not greater than a threshold value, finishing training;
and when the loss of the test image data and the positive sample image data is greater than the threshold value, returning to the step S2.
3. The method as claimed in claim 1, wherein the preprocessing of the statistical module in step S2 comprises:
at least introducing a motion unit, a brightness statistical unit of each channel, a local detail statistical unit, an edge gradient unit and a local similarity unit of a front frame and a rear frame into a statistical module;
the filtering algorithm model preset in the step S2 at least comprises a spatial filtering model and a time domain filtering model.
4. The method of claim 3, wherein the step of constructing the spatial filtering model comprises:
s211, using the guided filtering as a frame, wherein the guided filtering formula is as follows:
Out(i,j,k)= λ*In(i,j,k)+(1-λ)*Mean(k)
wherein ,In(i,j,k)andOut(i,j,k)respectively representing the current frame format dataiLine and firstjInput and output of the column;krepresents a component, which can be R, G and B;λrepresents the filtering weight of the spatial filtering model, and the weight range is [0,1 ]]；Mean(k)Representing sum components except for a central pointkThe mean of all pixels of the same color.
5. The method of claim 3, wherein the step of training the spatial filtering model in step S2 comprises:
s212, the spatial filtering model interacts with a statistic module, and the spatial filtering model is trained by using at least one of a motion unit, a channel brightness statistic unit, a local detail unit and an edge gradient unit;
s33, updating the filtering weight of the spatial filtering modelλ。
6. The method of claim 3, wherein the step of constructing the temporal filtering model comprises:
s221, using first-order low-pass filtering as a framework, wherein the adopted filtering formula is as follows:
Out(i,j)= δ*In(i,j)+(1-δ)*In_previous (i,j)
wherein ,In(i,j) andOut(i,j)respectively representing the input and output of ith row and jth column of current frame format data of the image;In_previous (i,j) representing the first frame, and obtaining the first frame time domain processing data after the time domain filtering model processingiGo to the firstjColumn output results;δrepresents the filtering weight of the time-domain filtering model, and the weight range is [0,1 ]]。
7. The method as claimed in claim 6, wherein the step of training the temporal filtering model in step S2 comprises:
s223, the time-domain filtering model interacts with a statistic module, and the time-domain filtering model is trained by using at least one of a channel brightness statistic unit, an edge gradient unit and a local similarity unit;
s224, updating the filtering weight of the time domain filtering modelδ。
8. The method of claim 3, wherein the step of processing the video image based on the spatial filtering model and the temporal filtering model comprises:
s7, setting a processing sequence of a spatial filtering model and a time-domain filtering model;
the statistical module receives and stores the time domain processing data of the previous frame obtained after the image data of the previous frame is processed by the spatial filtering model and the time domain filtering model;
s8, inputting image data of a current video frame into a spatial filtering model for processing, and outputting spatial processing data of the current video frame; then simultaneously inputting the spatial domain processing data of the current frame and the time domain processing data of the previous frame into a time domain filtering model for processing, and outputting the time domain processing data of the current frame;
s9, a statistical module receives and stores time domain processing data of the current frame; and outputting the time domain processing data of the current frame as enhanced image data.
9. A system for enhancing the quality of a video image according to any one of the claims 1 to 8, comprising:
the acquisition module is used for acquiring a video and acquiring preset image data;
the statistical module at least comprises a motion unit, a brightness statistical unit of each channel, a local detail statistical unit, an edge gradient unit and a front and rear frame local similarity unit, and is used for training a spatial filtering model and a time domain filtering model;
the image processing module at least comprises a spatial filtering module and a temporal filtering module; the spatial filtering module is set on the basis of a spatial filtering model, and the time-domain filtering module is set on the basis of a time-domain filtering model.
10. The system according to claim 9, wherein the collection module is configured to collect image data of a current frame of the video, process the image data of the current frame through the spatial filtering module, and output spatial processing data of the current frame; inputting the time domain processing data of the previous frame and the space domain processing data of the current frame into a time domain filtering module for processing to obtain the time domain processing data of the current frame, and finally outputting the time domain processing data; the time domain processing data of the current frame is cached to the statistical module, and is called through the statistical module when in use.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211571852.7A CN115908190B (en) | 2022-12-08 | 2022-12-08 | Method and system for enhancing image quality of video image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211571852.7A CN115908190B (en) | 2022-12-08 | 2022-12-08 | Method and system for enhancing image quality of video image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115908190A true CN115908190A (en) | 2023-04-04 |
| CN115908190B CN115908190B (en) | 2023-10-13 |
Family
ID=86485419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211571852.7A Active CN115908190B (en) | 2022-12-08 | 2022-12-08 | Method and system for enhancing image quality of video image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908190B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769722A (en) * | 2012-07-20 | 2012-11-07 | 上海富瀚微电子有限公司 | Time-space domain hybrid video noise reduction device and method |
| CN103024248A (en) * | 2013-01-05 | 2013-04-03 | 上海富瀚微电子有限公司 | Motion-adaptive video image denoising method and device |
| CN103413276A (en) * | 2013-08-07 | 2013-11-27 | 清华大学深圳研究生院 | Depth enhancing method based on texture distribution characteristics |
| CN103679641A (en) * | 2012-09-26 | 2014-03-26 | 株式会社理光 | Depth image enhancing method and apparatus |
| CN109344725A (en) * | 2018-09-04 | 2019-02-15 | 上海交通大学 | A kind of online tracking of multirow people based on space-time attention rate mechanism |
| CN109688351A (en) * | 2017-10-13 | 2019-04-26 | 华为技术有限公司 | A kind of image-signal processing method, device and equipment |
| CN113223059A (en) * | 2021-05-17 | 2021-08-06 | 浙江大学 | Weak and small airspace target detection method based on super-resolution feature enhancement |
| CN113674316A (en) * | 2021-08-04 | 2021-11-19 | 浙江大华技术股份有限公司 | Video noise reduction method, device and equipment |
| WO2022021999A1 (en) * | 2020-07-27 | 2022-02-03 | 虹软科技股份有限公司 | Image processing method and image processing apparatus |
| CN114219722A (en) * | 2021-11-12 | 2022-03-22 | 广州大学 | Low-illumination image enhancement method by utilizing time-frequency domain hierarchical processing |
-
2022
- 2022-12-08 CN CN202211571852.7A patent/CN115908190B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769722A (en) * | 2012-07-20 | 2012-11-07 | 上海富瀚微电子有限公司 | Time-space domain hybrid video noise reduction device and method |
| CN103679641A (en) * | 2012-09-26 | 2014-03-26 | 株式会社理光 | Depth image enhancing method and apparatus |
| CN103024248A (en) * | 2013-01-05 | 2013-04-03 | 上海富瀚微电子有限公司 | Motion-adaptive video image denoising method and device |
| CN103413276A (en) * | 2013-08-07 | 2013-11-27 | 清华大学深圳研究生院 | Depth enhancing method based on texture distribution characteristics |
| CN109688351A (en) * | 2017-10-13 | 2019-04-26 | 华为技术有限公司 | A kind of image-signal processing method, device and equipment |
| CN109344725A (en) * | 2018-09-04 | 2019-02-15 | 上海交通大学 | A kind of online tracking of multirow people based on space-time attention rate mechanism |
| WO2022021999A1 (en) * | 2020-07-27 | 2022-02-03 | 虹软科技股份有限公司 | Image processing method and image processing apparatus |
| CN113223059A (en) * | 2021-05-17 | 2021-08-06 | 浙江大学 | Weak and small airspace target detection method based on super-resolution feature enhancement |
| CN113674316A (en) * | 2021-08-04 | 2021-11-19 | 浙江大华技术股份有限公司 | Video noise reduction method, device and equipment |
| CN114219722A (en) * | 2021-11-12 | 2022-03-22 | 广州大学 | Low-illumination image enhancement method by utilizing time-frequency domain hierarchical processing |
Non-Patent Citations (3)
| Title |
|---|
| 叶庆伟;谢永昌;狄红卫;: "基于Non_local means的时空联合视频降噪算法", 微型电脑应用, no. 05 * |
| 姚哲维;杨丰;黄靖;刘娅琴;: "改进型循环生成对抗网络的血管内超声图像增强", 计算机科学, no. 05 * |
| 阎春生;廖延彪;田芊;: "层析成像图像重建算法综述", 中国光学, no. 05 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115908190B (en) | 2023-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8515141B2 (en) | Medical image processing apparatus and method for detecting locally protruding lesion | |
| JP5113841B2 (en) | Computer-aided analysis using video from an endoscope | |
| US8351667B2 (en) | Methods of contrast enhancement for images having blood vessel structures | |
| CN110163809A (en) | Confrontation network DSA imaging method and device are generated based on U-net | |
| CN111626962A (en) | CMOS endoscope image enhancement method | |
| US8666135B2 (en) | Image processing apparatus | |
| Suman et al. | Image enhancement using geometric mean filter and gamma correction for WCE images | |
| WO1992008405A1 (en) | Device for processing image of endoscope and method thereof | |
| CN111292307A (en) | Digestive system gallstone recognition method and positioning method | |
| CN115908190B (en) | Method and system for enhancing image quality of video image | |
| CN102760283B (en) | Image processing method, image processing device and medical imaging equipment | |
| JP7387859B2 (en) | Medical image processing device, processor device, endoscope system, operating method and program for medical image processing device | |
| CN111815617A (en) | Hyperspectrum-based eye fundus image detection method | |
| EP2313848A1 (en) | Methods for enhancing vascular patterns in cervical imagery | |
| CN116205814A (en) | Medical endoscope image enhancement method, system and computer equipment | |
| CN115115553A (en) | Endoscope image noise reduction processing method, electronic equipment and system | |
| CN114549368A (en) | Endoscope imaging system and method based on image sensor | |
| CN111080547A (en) | Endoscope image enhancement method | |
| CN113256502A (en) | Ultrasonic image adjusting method, terminal device and readable storage medium | |
| JP5543871B2 (en) | Image processing device | |
| CN114387191B (en) | Endoscope image enhancement method and endoscope device | |
| JP3126996B2 (en) | Endoscope image processing method | |
| CN117934371A (en) | Chest X-ray image bone suppression method based on conditional diffusion model | |
| Qi et al. | Vessel Image Enhancement Method Using Nonlinear Transformation Based on Guided Filtering | |
| JP2002077728A (en) | Method for processing time base subtracted image and device for the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |