CN115396570A - High-temperature industrial endoscope with faint light - Google Patents
High-temperature industrial endoscope with faint light Download PDFInfo
- Publication number
- CN115396570A CN115396570A CN202210816693.6A CN202210816693A CN115396570A CN 115396570 A CN115396570 A CN 115396570A CN 202210816693 A CN202210816693 A CN 202210816693A CN 115396570 A CN115396570 A CN 115396570A
- Authority
- CN
- China
- Prior art keywords
- image
- channel
- module
- video stream
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 claims abstract description 134
- 238000003384 imaging method Methods 0.000 claims abstract description 52
- 238000012634 optical imaging Methods 0.000 claims abstract description 28
- 238000001816 cooling Methods 0.000 claims description 56
- 238000004364 calculation method Methods 0.000 claims description 45
- 230000015572 biosynthetic process Effects 0.000 claims description 28
- 238000003786 synthesis reaction Methods 0.000 claims description 27
- 239000000498 cooling water Substances 0.000 claims description 18
- 238000001914 filtration Methods 0.000 claims description 17
- 239000002131 composite material Substances 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 15
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 2
- 230000006978 adaptation Effects 0.000 claims 2
- 230000006870 function Effects 0.000 description 48
- 238000000034 method Methods 0.000 description 12
- 238000005286 illumination Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 239000011049 pearl Substances 0.000 description 5
- 230000002441 reversible effect Effects 0.000 description 4
- 238000005260 corrosion Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- PXHVJJICTQNCMI-UHFFFAOYSA-N Nickel Chemical compound [Ni] PXHVJJICTQNCMI-UHFFFAOYSA-N 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 239000011324 bead Substances 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 239000000956 alloy Substances 0.000 description 1
- 229910045601 alloy Inorganic materials 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 229910052759 nickel Inorganic materials 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B17/00—Details of cameras or camera bodies; Accessories therefor
- G03B17/55—Details of cameras or camera bodies; Accessories therefor with provision for heating or cooling, e.g. in aircraft
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/73—Colour balance circuits, e.g. white balance circuits or colour temperature control
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Endoscopes (AREA)
Abstract
The invention discloses a high-temperature industrial endoscope of a faint light, which comprises a shell component, a faint light level optical imaging system and an image brightness processing module, wherein the faint light level optical imaging system and the image brightness processing module are arranged in the shell component, and the faint light level optical imaging system comprises: the dim light level optical imaging system is used for acquiring image video stream in the furnace; the invention solves the technical problems of insufficient image imaging brightness, blurred details and unclear contours in an industrial furnace under the condition of extremely weak light, and utilizes the strong correlation between the optimized image and the original image to achieve the purpose of eliminating double images appearing in the enhanced image while ensuring the enhanced image brightness, thereby effectively improving the image quality of the enhanced image.
Description
Technical Field
The invention mainly relates to the technical field of high-temperature industrial endoscopes, in particular to a high-temperature industrial endoscope with faint light.
Background
The acquisition of the internal information of the industrial kiln is the key for realizing the accurate control, ensuring the high-efficiency operation and achieving the purposes of consumption reduction and emission reduction. The high-temperature industrial endoscope is core equipment capable of directly acquiring internal information of an industrial kiln. However, the high temperature and strong corrosive harsh environment inside the industrial kiln has severe requirements on the temperature resistance and corrosion resistance of the endoscope. Meanwhile, due to an extremely weak light environment (namely, a faint light level brightness environment with the illumination degree lower than 0.0001 Lux) inside the industrial kiln, the problems of detail loss, fuzzy outline, poor definition and the like of an image shot by the endoscope exist. This will influence the accurate judgement of operating personnel to the kiln operation state, can't select in time the appropriate means control kiln, leads to the fluctuation of furnace conditions, equipment failure, finally causes huge economic loss. The novel faint light high-temperature industrial endoscope can bear high-temperature and strong corrosion environment, overcomes the extremely weak light environment (the illumination is lower than 0.0001 Lux) in the industrial kiln, realizes faint light level imaging, provides high-quality in-furnace imaging for operators, clearly shows the running condition in the kiln, provides an operation basis for adjusting the running state of the industrial kiln, and avoids accidents.
The endoscope equipment of getting like in the kiln mainly includes: a non-light source industrial endoscope and a light source industrial endoscope. The active light source industrial endoscope is also classified into a telescopic type and a non-telescopic type. Now, the explanation is made one by one.
Industrial endoscope without light source: an optical camera is directly mounted at the front end of an endoscope, and a measured object is shot by using ambient light. The collected image information is transmitted out from the tail part of the endoscope through a signal line and is externally connected with an upper computer for displaying. The mode is clear in imaging and simple in structure, the endoscope tube body with the movable wide-angle assembly and the angle adjusting unit can be designed, the shooting visual angle is increased, the picture width is increased, and the angle with clear pictures is selected for shooting and imaging. However, since no light source is provided, and no image brightness improvement algorithm is designed, it is difficult to capture a clear and bright image under the condition of insufficient ambient light. Meanwhile, the camera is directly arranged at the front end of the endoscope, and even if a cooling system is designed, the camera at the front end can still be burnt by high temperature close to the center in the furnace, so that the camera cannot normally operate, and the equipment is not suitable for shooting inside an industrial kiln with a high-temperature environment.
Telescopic active light source industrial endoscope: by designing the telescopic device, the detection device can be extended only when the shooting is needed. The equipment can prevent the camera from being in a severe environment in the furnace for a long time and prolong the service life of the endoscope; on the other hand, the pollution of dust in the kiln to the camera lens is reduced, and the imaging definition is ensured. However, the equipment cannot provide continuous real-time in-furnace pictures, is difficult to adjust in time according to the operating state of the industrial kiln, and has certain potential safety hazards.
Non-telescopic active light source industrial endoscope: utilize a plurality of LED lamp pearls, with the circumference array distribution around the endoscope stack shell, for the endoscope provides the hi-lite to shine, be convenient for carry out high bright high definition to dark or low light object and shoot. However, because the LED lamp beads are distributed circumferentially, light rays are scattered, light loss is serious in the irradiation process, and sufficient brightness cannot be provided for shooting inside the kiln in a faint light environment. Meanwhile, the LED lamp bead array is positioned at the front part of the endoscope and is extremely easy to damage when being in a high-temperature environment inside the industrial kiln for a long time. Therefore, the equipment is not suitable for shooting inside the industrial kiln in a high-temperature and dim environment.
The utility model with publication number CN213338205U is an endoscope camera head convenient for observing and imaging clearly. In order to obtain clear and stable imaging for the endoscope, the device comprises a motor, a sliding block, a fixing plate, an adjusting rod and the like. The main function and the realization principle of the endoscope are divided into two parts, wherein one part realizes the fixation of the endoscope main body at the inner side of the detected object by utilizing components such as a slide block, a slide rail, a first motor, a fixed plate, a support rod, an adjusting rod and the like; the other part is to use components such as a second motor, an angle rod, a rotating tooth and an adjusting tooth to realize the fine adjustment of the angle of the camera. The specific workflow of the equipment is as follows: firstly, the first motor drives the threaded rod to rotate, so that the endoscope sliding block slides along the sliding rail, the fixing plate, the supporting rod and the adjusting rod which are originally attached to the outer side of the main body abut against the inner side of the detection area, the whole endoscope main body is fixed, and a stable and clear image is obtained. Then, the second motor is started, and the angle rod drives the camera to rotate, so that the camera angle is adjusted. This patent provides a new equipment fixed form to it does not receive the external influence to have guaranteed indirectly the acquireing of image after fixed, can stably obtain clear image. However, the equipment lacks a cooling component, cannot bear high-temperature and high-pressure severe environment inside the industrial kiln, cannot ensure the air tightness of the industrial kiln under long-time work due to the existence of movable components such as a sliding rail and a sliding block, and has certain potential safety hazard, so that the equipment cannot be applied to the industrial kiln.
Utility model patent publication CN213399057U is a high brightness industrial endoscope for viewing large spaces. This patent is in order surrounding the mode of arranging, settles LED lamp pearl array around endoscope camera lens, for the shooting of endoscope provides the illumination, improves the imaging brightness. Simultaneously, this patent has also designed corresponding heat radiation structure for LED lamp pearl array, prevents that LED from overheated influence endoscope normal work. This patent provides one kind and regards LED lamp pearl array as endoscope illumination source, finally improves the new thinking of formation of image effect, nevertheless to the inside faint light level luminance environment of industrial kiln (illuminance is less than 0.0001 Lux), the illumination light dispersion, the light loss that LED lamp pearl array provided are serious, can't provide high bright effectual illumination for the inside clear shooting of industrial kiln. Therefore, the device is not suitable for image acquisition in the interior of an industrial kiln in a dim environment.
The invention patent with publication number CN114026495A is an endoscope capable of finely adjusting a photographing angle, and its working principle is: the bending unit of the camera is designed, the endoscope camera is connected with the endoscope shell, and the internal hinge of the camera is connected with the angle adjusting units, so that the rotation of the endoscope camera at any angle is realized, and the detection range is maximized. Meanwhile, the equipment is provided with an air supply pipe to prevent the endoscope camera from being damaged due to high temperature. But this equipment is only through mobilizable wide angle subassembly and angle adjustment unit, and the increase endoscope shoots the visual angle, promotes the picture width, does not carry out extra brightness to the image and handles, still can't obtain under the not enough circumstances of illumination, the inside sharp picture of industrial kiln.
Disclosure of Invention
The invention provides a faint light high-temperature industrial endoscope, which solves the technical problems of insufficient brightness, blurred details and unclear contours of images obtained by the existing industrial endoscope in an industrial kiln which is sealed at high temperature and low in illuminance.
In order to solve the technical problem, the invention provides a high-temperature industrial endoscope for faint light, which comprises a shell component, a faint light level optical imaging system and an image brightness processing module, wherein the faint light level optical imaging system and the image brightness processing module are installed in the shell component, and the image brightness processing module comprises:
the imaging system comprises an imaging system, a light level optical imaging system and a light level optical imaging system, wherein the imaging system is used for acquiring image video flow in a furnace;
and the image brightness processing module is connected with the dim level optical imaging system and used for receiving the in-furnace image video stream output by the dim level optical imaging system and performing brightness processing on the in-furnace image video stream so as to obtain a highlight clear in-furnace image video stream.
Further, the housing assembly comprises a front end housing assembly and a rear end housing assembly, wherein:
the front-end housing assembly is of a double-layer structure, an air-cooling circulation channel is arranged on the outer layer of the double-layer structure, and a double-spiral wound water-cooling circulation flow channel is arranged on the outer side of the inner layer;
and the rear end housing assembly is used for mounting the CCD imaging chip, the imaging driving circuit and the image brightness processing module, and is provided with an air cooling air inlet, a water cooling water inlet and a water cooling water outlet.
Further, the pyloric high temperature industrial endoscope further comprises a cooling system, wherein:
the cooling system adopts an air cooling-water cooling double cooling system, cold air in the air cooling-water cooling double cooling system enters an air cooling circulation channel from an air cooling air inlet, and air circulates in the air cooling circulation channel to form an air cooling effect;
water flow in the air-cooling-water-cooling double-cooling system enters the water-cooling circulation flow channel from the water-cooling water inlet and is discharged from the water-cooling water outlet, and the discharged water flow can flow in again after being treated so as to form a water-cooling effect through circulation.
Further, the image capture objective lens group, the relay lens group, the focusing lens group and the CCD imaging chip are sequentially connected to the light level optical imaging system, wherein:
the image capturing objective lens group comprises a negative power lens group and a positive power lens group, the negative power lens group is used for scattering incident light, the acquired optical information with large field angle and large light incoming amount is uniformly scattered to the positive power lens group, an initial condition is provided for long-distance parallel transmission of the optical information, and the positive power lens group is used for re-gathering the scattered light to form light beams parallel to an optical axis;
the relay lens group is used for transmitting the in-furnace image acquired by the image taking objective lens group to the CCD imaging chip;
the focusing lens group is used for adjusting the focal length when the CCD imaging chip is blurred, so that the imaging is clear;
and the CCD imaging chip is used for converting the clear image transmitted by the focusing mirror group into a digital image.
Further, the image brightness processing module comprises an input processing unit, and an FPGA video stream image processing unit and an output processing unit which are sequentially connected with the input processing unit, wherein:
the input processing unit is used for decoding the in-furnace image video stream output by the ghost-level optical imaging system and storing the decoded in-furnace image video stream into a DDR SDRAM storage module;
the FPGA video stream image processing unit is used for carrying out image preprocessing and brightness synthesis on the decoded in-furnace image video stream, and specifically comprises a MicroBlaze processing module, a signal processing task triggering module connected with the MicroBlaze processing module, a preset processing task module and a DDR SDRAM storage module, wherein:
the signal processing task triggering module is used for judging whether the number of image frames in the furnace image video stream meets the algorithm requirement in the preset processing task module or not according to the furnace image video stream stored by the DDR SDRAM storage module, and sending a task command to the MicroBlaze processing module to realize task triggering, task configuration and task management;
the preset processing task module is used for pre-storing an algorithm of the image preprocessing and brightness synthesis processing tasks;
the MicroBlaze processing module is used for receiving task triggering, task configuration and task management commands sent by the signal processing task triggering module, and carrying out image preprocessing and brightness synthesis on the decoded in-furnace image video stream according to an algorithm of an image preprocessing and brightness synthesis processing task stored in the preset processing task module in advance;
the DDR SDRAM storage module is used for storing the in-furnace image video stream decoded by the input processing unit and performing data interaction with the signal processing task triggering module and the MicroBlaze processing module;
and the output processing unit is used for outputting the in-furnace image video stream processed by the FPGA video stream image processing unit.
Further, the preset processing task module comprises a three-dimensional digital comb filtering algorithm submodule, a video highlight synthesis algorithm submodule based on time-space domain self-adaption and a white balance algorithm submodule based on a multi-color dynamic domain algorithm which are connected in sequence, wherein:
the three-dimensional digital comb filtering algorithm submodule is used for carrying out three-dimensional digital comb filtering on the decoded in-furnace video stream;
a video highlight synthesis algorithm submodule based on time-space domain self-adaption and used for carrying out video highlight synthesis on the video stream in the furnace after three-dimensional digital comb filtering;
and the white balance algorithm submodule based on the multi-color dynamic domain algorithm is used for carrying out white balance processing on the video stream in the furnace after the video brightness is synthesized.
Further, the video highlight synthesis algorithm submodule based on the time-space domain self-adaption comprises a video frame acquisition submodule, a nonlinear transformation submodule, an overlapping submodule and a synthesized video frame acquisition submodule which are sequentially connected, wherein:
the video frame acquisition submodule is used for acquiring video frames in the three-dimensional comb-shaped filtered video stream, and the video frames comprise a current video frame and a video frame adjacent to the current video frame;
the nonlinear transformation submodule is used for carrying out nonlinear transformation on the video frame;
the superposition submodule is used for superposing the video frames after the nonlinear transformation to obtain an enhanced image;
and the synthesized video frame acquisition sub-module is used for optimizing the enhanced image by adopting a maximum posterior model to obtain a synthesized video frame so as to obtain the highlight video stream.
Further, the composite video frame acquisition sub-module comprises a fidelity function calculation sub-module, a consistency function calculation sub-module, a regularization function calculation sub-module, a posterior probability calculation sub-module and a composite video frame calculation sub-module which are connected in sequence, wherein:
the fidelity function calculation submodule is used for calculating the fidelity function, wherein the calculation formula of the fidelity function is as follows:
wherein psi (O) t ,Y j ) Is a fidelity function and is used for measuring the optimized image O at the moment t t And j time instant enhanced image Y j The similarity of (a) to (b) is,
is O t And Y j B and D represent the blur matrix and the down-sampled matrix, respectively, S represents the covariance matrix,
optimized image O representing time t t Enhanced image Y to time j j The motion compensation matrix of (a);
the consistency function calculation submodule is used for calculating a consistency function, wherein the calculation formula of the consistency function is as follows:
wherein,
representing the output optimized image at time t-1,
an optimized image O representing a consistency function and measuring the time t t And output optimized image at time t-1
The similarity between the two groups is similar to each other,
is O t And
the variance of the joint distribution of (a),
is the optimized image O at time t t Output optimized image to time t-1
The motion compensation matrix of (a);
the regularization function calculation sub-module is used for calculating a regularization function, wherein the calculation formula of the regularization function is as follows:
wherein, L (O) t ) For the regularization function, P is the size of the moving window, a is used to constrain the degree of smoothness of the optimized image,H l and V j Indicating that the time t is optimized to the image O t Operators for shifting pixels l and j in the horizontal and vertical directions, respectively;
a posterior probability calculation submodule for obtaining an optimized image O according to the fidelity function, the consistency function and the regularization function t And the calculation formula of the posterior probability is as follows:
wherein, p (O) t ) Is to optimize the image O t R is the video image frame number of the original video used by the output optimized image before the time t, b is the video image frame number of the original video used by the output optimized image after the time t;
the composite video frame calculation submodule is used for obtaining a composite video frame according to the posterior probability of the output optimized image, and the specific calculation formula is as follows:
further, the white balance algorithm submodule includes a space conversion submodule, an absolute deviation mean calculation submodule, a region to be processed acquisition submodule, a mean and deviation calculation submodule, a candidate white point acquisition submodule and a fusion submodule which are connected in sequence, wherein:
the space conversion sub-module is used for converting image frames in the highlight video stream from an RGB space to an HSV space and a YCbCr space respectively to obtain an H channel, an S channel, a V channel, a Y channel, a Cb channel and a Cr channel;
the absolute deviation mean value calculation submodule is used for dividing the image frame into a preset number of areas and calculating the absolute deviation mean values of the H channel, the S channel, the Cb channel and the Cr channel of each area respectively;
the to-be-processed region acquisition submodule is used for acquiring a to-be-processed region which needs white balance processing according to the absolute deviation mean value of the H channel, the S channel, the Cb channel and the Cr channel;
the mean value and deviation calculation submodule is used for calculating the mean values and deviations of an H channel, an S channel, a Cb channel and a Cr channel of the region to be processed;
the candidate white point obtaining submodule is used for obtaining candidate white points according to the mean value and the deviation of the H channel, the S channel, the Cb channel and the Cr channel of the area to be processed, and the calculation formula specifically comprises the following steps:
wherein, H (i, j), S (i, j), cb (i, j) and Cr (i, j) are respectively the i pixel and the j pixel of H channel, S channel, cb channel and Cr channel, avrH all 、AvrS all 、AvrCb all And AvrCr all Average of H channel, S channel, cb channel and Cr channel, devH all 、DevS all 、DevCb all And Devccr all Variances of an H channel, an S channel, a Cb channel and a Cr channel respectively;
the fusion submodule is used for selecting a reference white point from the candidate white points and fusing the reference white point with the brightness information of the HSV space and the YCbCr space, and the calculation formula is as follows:
wherein img represents a fused image obtained by fusing brightness information of a reference white point and HSV space and YCbCr space, imgR, imgG and imgB represent channel images corresponding to an R channel, a G channel and a B channel respectively, theta represents multi-color-domain fusion degree, and V is max And Y max Respectively, the maximum value of lightness and the maximum value of brightness, R avgw 、G avgw 、B avgw The average pixel gray scale values of the R channel, the G channel and the B channel are respectively.
Furthermore, the high-temperature industrial endoscope with the faint light also comprises a display module which sends the video stream of the image in the furnace with high brightness and clarity to an upper computer for displaying.
Compared with the prior art, the invention has the advantages that:
the invention provides a high-temperature industrial endoscope for faint light, which comprises a shell component, a faint light level optical imaging system and an image brightness processing module, wherein the faint light level optical imaging system and the image brightness processing module are installed in the shell component, and the high-temperature industrial endoscope for faint light comprises: the dim light level optical imaging system is used for acquiring image video stream in the furnace; the invention solves the technical problems of insufficient image imaging brightness, blurred details and unclear contours in an industrial furnace under the condition of extremely weak light, and utilizes the designed fidelity function, consistency function and regularization function to eliminate the ghost image appearing in the enhanced image while ensuring the enhanced image brightness, thereby effectively improving the image quality of the enhanced image.
The key technical points of the invention are as follows:
in the aspect of optical imaging system design, a brand new thought is adopted, and by utilizing a reverse long-focus objective lens group with a large view field and a large light inlet quantity, a Hopkins bar relay lens group without light loss and capable of offsetting transmission distortion and deviation and a focusing lens group capable of manually focusing to enable imaging to be clear, video image brightness is improved from a hardware level, video image quality is improved, and high-quality imaging in a dim environment is achieved.
In the aspect of a processing platform of an image processing algorithm, the algorithm is integrated in an FPGA chip, the hardware of the algorithm is realized, the dependence on an upper computer is eliminated, an optical imaging system and the image processing algorithm are integrated into a whole, the integrity and the independence of the functions of equipment are ensured, a fidelity function, a consistency function and a regularization function are designed, the strong correlation between an optimized image and an original image is utilized, the aim of eliminating ghosts in the enhanced image while the brightness of the enhanced image is ensured is fulfilled, and the image quality of the enhanced image is effectively improved.
Drawings
Fig. 1 is an overall structure diagram of a high-temperature industrial endoscope for deep light according to a first embodiment of the invention;
FIG. 2 is a schematic diagram of an optical imaging system of a high-temperature industrial endoscope according to a first embodiment of the present invention;
fig. 3 is a block diagram of an image brightness processing module of the high-temperature industrial endoscope according to the first embodiment of the present invention;
fig. 4 is a schematic diagram of a highlight video composition according to a first embodiment of the present invention;
FIG. 5 is a flowchart illustrating a white balance processing according to a first embodiment of the present invention;
fig. 6 is a flowchart of the operation of the image brightness processing module of the deep-light high-temperature industrial endoscope according to the first embodiment of the present invention.
Reference numerals:
u1, an air cooling circulation channel; u2, a water-cooling circulation flow channel; u3, CCD imaging chip; u4, an air-cooled air inlet; u5, an image brightness processing module; u6, a water-cooled water inlet; u7, a water-cooling water outlet; m1, an image capturing objective lens group; m2, a relay lens group; m3, hopkins rod lens; m4, a focusing lens group; m5, a CCD photosensitive chip; s1, an FPGA video stream image processing unit; s2, presetting a processing task module; s3, a signal processing task triggering module; s4, a MicroBlaze processing module; s5, a DDR SDRAM storage module; s6, CCD; s7, an input processing unit; s8, an output processing unit; s9, an upper computer.
Detailed Description
In order to facilitate an understanding of the invention, the invention will be described more fully and in detail below with reference to the accompanying drawings and preferred embodiments, but the scope of the invention is not limited to the specific embodiments below.
Example one
As shown in FIG. 1, the high-temperature industrial endoscope of the invention comprises a housing assembly, a cooling system, a light level optical imaging system and an image brightness processing module U5. The specific working process comprises the following steps: firstly, the normal work of equipment in a high-temperature environment in an industrial kiln furnace is ensured by means of a double-cooling system which simultaneously works by air cooling and water cooling, then, a focusing lens group M4 of a conoscopic optical imaging system is adjusted to ensure the image taking definition, image information shot by an endoscope is transmitted to a CCD photosensitive chip M5 without damage through an image taking objective lens group and a relay lens group M2, and then, the image is subjected to image preprocessing and brightness synthesis by an image brightness processing module U5 to obtain a high-brightness and clear internal video of the industrial kiln furnace. The following specifically describes each component of the apparatus one by one:
a housing assembly:
the shell of the embodiment of the invention takes high-temperature-resistant nickel-based high-temperature alloy as a main material, as shown in figure 1. The shell has a double-layer structure, wherein the outer layer is an air-cooling circulation channel U1, and the outer side of the inner layer is a double-spiral wound water-cooling circulation flow channel U2. An air cooling air inlet U4 and a water cooling water inlet and outlet are arranged at the rear end of the shell. The rear end cylindrical shell mainly comprises a CCD imaging chip U3, an imaging drive circuit, an image brightness processing module U5, a video signal output interface, a power supply and the like.
Cooling system
The embodiment of the invention adopts an air cooling-water cooling double-cooling system to ensure that the equipment stably works for a long time in a high-temperature environment in the furnace, as shown in figure 1. The cold air enters the air cooling channel of the inner layer of the shell from the air inlet, and the air circulates in the air cooling channel to form an air cooling effect. Water flow enters the spiral pipeline from the water-cooling water inlet U6 and is finally discharged from the water-cooling water outlet U7, and the discharged water flow can flow in again after being treated, so that the water-cooling effect is formed through circulation. According to the embodiment of the invention, the inlet and the outlet of the double cooling system are designed at the rear end of the equipment, so that the endoscope body can be cooled in a high-temperature environment, the imaging and image brightness processing module U5 at the rear end of the endoscope can also be cooled, and the working stability of the endoscope in a high-temperature severe environment is improved.
The imaging system of the image level of the ghost of the embodiment of the invention is shown in figure 2, and comprises:
an image capturing objective lens group M1, a relay lens group M2, a focusing lens group M4 and a CCD imaging chip U3. The parts are specifically introduced as follows:
(1) Image capturing objective lens assembly M1
The imaging of the reverse long-focus structure has the characteristic that an image plane is small when the field angle is large, under the same aperture, not only can a wider image be collected, but also the light inlet quantity of the lens can be increased, and support is provided for realizing large-field wide-view-angle ghost-level imaging of the endoscope. The image capturing objective lens group M1 based on the reverse telephoto structure mainly comprises a negative power lens group and a positive power lens group. The negative focal power lens group is responsible for scattering incident light, and uniformly scatters the collected optical information with a large field angle and a large light inlet quantity to the positive focal power lens group, so that initial conditions are provided for long-distance parallel transmission of the optical information. The positive power lens group gathers scattered light again to form light beams parallel to the optical axis, and the gathered parallel light beams prevent the divergence loss of partial light and avoid the brightness reduction of the optical image in the acquisition process.
(2) Relay lens group M2
Because the front end of the endoscope is in a high-temperature and high-corrosion environment in the furnace, the imaging chip is easy to burn down when being in the environment for a long time, and the imaging system can be ensured to operate stably for a long time when the rear end of the endoscope is outside the furnace, so the imaging chip is arranged at the rear end of the endoscope. Based on the above imaging chip installation scheme, a group of relay lens group M2 needs to be constructed, and an image acquired by the image capturing objective lens group M1 is transmitted to the rear end of the endoscope, so that the length of the optical imaging system is increased, and meanwhile, transmission of the acquired image without light loss is realized. Considering that the hopkins rod lens M3 is composed of two identical rod lenses, and each rod lens group is composed of one thick biconvex lens and two symmetrical concave lenses, the hopkins rod lens M3 has an entirely symmetrical structure. The structure can not only mutually offset the distortion and deviation generated in the transmission process, but also has extremely small light loss because light passes through the lens with larger refractive index in most time. Therefore, the relay lens group M2 composed of three hopkins rod lenses M3 is designed to realize the optical image transmission without optical loss from the front end to the rear end of the endoscope.
(3) Focusing lens group M4
As the condition in the industrial kiln is dynamically changed, a focusing mirror group M4 is designed for obtaining clear in-furnace imaging. When the CCD photosensitive chip M5 is imaged in a fuzzy manner, focusing can be performed again by adjusting the focusing mirror group M4, so that the imaging clarity is ensured.
(4) CCD imaging chip U3
The CCD imaging chip U3 converts the clear image transmitted by the focusing mirror group M4 into a digital image, and the acquired image is conveniently subjected to subsequent processing.
As shown in fig. 3, the image brightness processing module U5 includes an input processing unit S7, an FPGA video stream image processing unit S1, and an output processing unit S8.
(1) Input processing unit S7
After the conversion by the CCD S6, the video stream composed of the digital image frames needs to be decoded by a video decoding chip (e.g. TVP 5150), and the decoded data information is stored in the DDR SDRAM memory module S5.
(2) FPGA video stream image processing unit S1
The FPGA video stream image processing unit S1 is the core of an image brightness processing module U5, and has the main functions of processing task signal triggering, three-dimensional digital comb filtering, a video highlight synthesis algorithm based on time-space domain self-adaption and white balance based on a multi-color dynamic domain algorithm. The functions and algorithm are all embedded into the video stream image processing module based on the FPGA. The module is a hardware platform embedded system taking MicroBlaze as a core, is developed by using Verilog HDL hardware description language, and the process block diagram of the module is shown in FIG. 3. It comprises the following components:
(1) MicroBlaze processing module S4
MicroBlaze is a high-flexibility configurable soft core embedded processor, the instruction function of which includes read-write operation and operation, etc., the instruction operation is connected with DRR SDRAM through DRR interface, and reads and operates the relevant data, or writes the operation result into DDR SDRAM, and stores the data.
(2) Signal processing task triggering module S3
The signal processing task triggering module S3 is mainly responsible for acquiring image frame information stored in the DDR SDRAM when the FPGA video stream image processing unit S1 operates, and sending a task command to the MicroBlaze, so as to implement functions of task triggering, task configuration, task management, and the like. In the aspect of task triggering, image frame information collected by the CCD and stored by a decoder is judged whether the number of image frames meets the requirement of an S2 algorithm of a preset processing task module, and a related task command is sent to the MicroBlaze; in the aspect of task configuration, a storage space required by task operation, a task calculation mode and the like are preset, and a MicroBlaze operation stage is set according to the actual progress of a task; in the aspect of task management, before a task is loaded, a related operation instruction is configured for MicroBlaze, image frame data required by the task is obtained through a DDR SDRAM, a current algorithm task is read from a preset processing task module S2, and a related function is loaded. In the task loading process, the size of the storage space is allocated and adjusted in real time according to hardware resources required by the execution of the current task, the task execution efficiency is improved, and the stable operation of the algorithm and the function is ensured.
(3) Preset processing task module S2
The preset processing task module S2 is a memory in which an algorithm of a task to be processed and a command for executing a corresponding task are stored in advance. In the embodiment of the invention, the algorithm to be processed and the task command mainly comprise three-dimensional digital comb filtering, a video highlight synthesis algorithm based on time-space domain self-adaption and white balance based on a multi-color dynamic domain algorithm. The detailed steps of each algorithm are as follows,
step1: three-dimensional digital comb filtering:
because the received video signal is not subjected to any preprocessing, the brightness (L) and the chroma (C) in the original video signal are highly mixed, and the video image has the problems of brightness and color crosstalk, color overlapping, serious clutter interference and the like. Therefore, in order to retain image details and solve the above problems, a digital comb filter is used to filter an original video signal, and by designing a plurality of components such as a delay, an adder, a subtractor, and a band-pass filter of the digital comb filter, a video signal without cross color, dot noise, hanging point, and wider bandwidth of a bright signal is finally obtained, thereby providing a high-quality image basis for subsequent image brightness processing. Meanwhile, the three-dimensional comb filter has a good brightness and color separation effect on the static video, and the three-dimensional comb filter can be used for carrying out operation processing on two continuous frames of images of the furnace top charge surface video to obtain a clearer video image by considering that the descending of the furnace top charge surface of the blast furnace is a gradual change process, and the two continuous frames of images have small change and are relatively static.
Step2: video highlight synthesis algorithm based on time-space domain self-adaption
The image after the comb filtering process needs to be synthesized into a highlight video, and a schematic diagram of the embodiment for realizing the highlight video synthesis is shown in fig. 4. The method comprises the following specific steps:
(1) Original video image frame I t The contrast between light and shade is large, and in order to avoid the overexposure phenomenon in the superposition processing, the image frame of the video is subjected to nonlinear conversion to obtain I' t The tone value of the dark pixel in the original image is increased and the tone value of the bright portion is suppressed. The following transformations may be used:
P y and P x N is selected according to the gray scale distribution of the image, where n =2, and n is used to control the overall improvement degree of the luminance.
(2) And superposing the transformed images. Selecting I' t-1 、I′ t And I' t+1 Overlapping to obtain enhanced image Y t . Since three frames are selected for brightness enhancement, the image Y is enhanced t May appear ghosting, and therefore, optimization of the enhanced image is also required.
(3) For the superposed image Y t And (4) optimizing by adopting a maximum posterior model. The posterior probability is determined by the fidelity function ψ (-) and the consistency function Φ (-) and the regularization function L (-) together.
(4) The fidelity function psi (·, ·) measures the similarity between the optimized image and the original image, and adopts mahalanobis distance to calculate, the calculation formula is,
wherein psi (O) t ,Y j ) For fidelityDegree function for measuring optimized image O at time t t And j time instant enhanced image Y j The similarity of (a) to (b) is,
is O t And Y j B and D represent a blur matrix and a downsampled matrix, respectively, S represents a covariance matrix,
optimized image O representing time t t Enhanced image Y to time j j The motion compensation matrix of (2).
(5) The consistency function phi (·,) is used to control the temporal consistency of the composite frame, which is calculated as,
wherein,
the output optimization image at time t-1 is shown,
an optimized image O representing a consistency function and measuring the time t t And output optimized image at time t-1
The similarity between the two groups is similar to each other,
is O t And
the variance of the joint distribution of (a) is,
is the optimized image O at time t t Output optimized image to time t-1
The motion compensation matrix of (2).
(6) The regularization function L (-) is used to suppress image noise information, and is calculated as,
wherein, L (O) t ) For the regularization function, P is the size of the moving window, α is used to constrain the smoothness of the optimized image, H l And V j Indicating that the time t is optimized to the image O t The operators for the l and j pixels are shifted in the horizontal and vertical directions, respectively.
(7) The posterior probability of the composite video frame is:
wherein, p (O) t ) Is to optimize the image O t R is the video image frame number of the original video used by the output optimized image before the time t, and b is the video image frame number of the original video used by the output optimized image after the time t. The fidelity function, the consistency function and the regularization function designed by the embodiment of the invention utilize the strong correlation between the optimized image and the original image, achieve the purpose of eliminating ghost images appearing in the enhanced image while ensuring the brightness of the enhanced image, and effectively improve the image quality of the enhanced image.
(8) The output composite image frame is:
in conjunction with the above function, the output composite image frame may be represented as:
(9) According to the formula (7), the composite brightness enhanced image frame can be obtained by using the least square method
And processing the original video stream to obtain the highlight video stream.
Step3: white balance based on multi-color dynamic domain algorithm
The highlighted video image has color difference, and in order to improve the image quality, the image is subjected to white balance processing based on a multi-color dynamic domain algorithm to obtain an image closer to the real color temperature in the kiln, a flow chart for realizing the white balance processing of the embodiment is shown in fig. 5, and the specific steps of the white balance algorithm are as follows:
(1) The image is converted from RGB space to HSV space and YCbCr space, respectively, and the results are recorded as H, S, V, Y, cb, cr.
(2) Dividing the image into 8 blocks to enhance the robustness of the algorithm, and calculating the absolute deviation mean values of H, S, cb, cr of each region DevH, devS, devCb, devCr by the following formula:
where N is the number of pixels per block area.
(3) When one index of a region is small, the color distribution of the region is uniform, the white balance processing is not facilitated, and the region is selected to be ignored.
(4) The average AvrH of the entire image excluding the areas not contributing to the white balance processing in the previous step is calculated without considering the areas all ，AvrS all ，AvrCb all ，AvrCr all Sum variance DevH all ，DevS all ，DevCb all ，DevCr all 。
(5) Determining a candidate white point, the candidate white point satisfying the following requirements,
(6) Selecting a reference white point, wherein the selection method of the reference white point comprises the following steps: and arranging the brightness values of the candidate white point pixels from high to low, and respectively selecting the white point with the brightness value of the first 10 percent in the HSV space and the YCbCr space as a reference white point.
(7) The reference white point is fused with the brightness information of the HSV space and the YCbCr space to complete the white balance calculation, the calculation formula is as follows,
wherein, V max Is the maximum value of brightness of all images in an image, Y max Is the maximum value of all image brightness in the image, θ is the multi-color domain fusion degree, and θ =0.4 can be selected.
After the video white balance processing, the video is stored in a DDR SDRAM, and meanwhile, the decoding and the display are carried out through an output processing unit S8.
By the method, the in-furnace image can be obtained by the endoscope in an extremely low light environment. And then the image quality of the video stream is improved by three-position comb filtering. And then, performing highlight processing on the video stream in a multi-frame superposition mode, and optimizing the video stream. And then white balance processing is carried out on the highlight video stream to approach the real color temperature in the furnace, and a high-quality in-furnace image is obtained. And finally, storing the video stream obtained in the furnace, and sending the video stream to an upper computer S9 for displaying.
(4) DDR SDRAM memory module S5
The system is mainly used for storing the collected and decoded video stream image frame information, the image frame information processed by a preset task algorithm and relevant information in the task operation processing process, and providing a data exchange platform for other modules.
(5) Output processing unit S8
The highlighted video stream re-synthesized by the image processing task is output to the upper computer S9 through video coding (VENC) and on-screen display (OSD).
In this embodiment, based on the above steps, the image frame processing and the high-brightness video synthesis are performed on the captured video stream, and a specific workflow of the image brightness processing module U5 is as shown in fig. 6: firstly, decoding a CCD (charge coupled device) collected video stream and storing the decoded video stream in a DDR SDRAM; secondly, the signal processing triggering module judges the number of the image frames, sends an image preset task processing triggering signal to the MicroBlaze, and simultaneously performs advanced configuration on tasks and performs increasing and deleting management on data resources; and then, when the MicroBlaze receives a trigger signal, image frame data is taken out from the DDR SDRAM, three-dimensional digital comb filtering, video highlight synthesis based on time-space domain self-adaption and white balance based on a multi-color domain algorithm are carried out, and finally a highlight video stream is generated and output to be displayed.
The image of the high-temperature industrial endoscope with the faint light provided by the embodiment of the invention is obtained by acquiring a digital image in an industrial furnace, forming an original video stream, carrying out three-dimensional comb filtering on the original video stream, carrying out highlight video synthesis on the three-dimensional comb filtered video stream, obtaining a highlight video stream and carrying out white balance processing on image frames in the highlight video stream, thereby obtaining a high-quality furnace image, solving the technical problems of insufficient image imaging brightness, blurred details and unclear contours of the image in the industrial furnace under an extremely weak light condition, and utilizing the strong correlation between an optimized image and an original image by the designed fidelity function, consistency function and regularization function, thereby achieving the purposes of eliminating the ghost image appearing in the enhanced image while ensuring the enhanced image brightness, and effectively improving the image quality of the enhanced image.
Specifically, in this embodiment, the image quality of the video stream is first improved by three-dimensional comb filtering, then a method for synthesizing a high-brightness video stream based on multi-frame synthesis and a maximum posterior probability model is designed to perform high-brightness processing on the video stream, and finally, white balance processing is performed on the high-brightness video stream to approach the true color temperature in the furnace, so as to obtain a high-quality in-furnace image, thereby providing a feasible and reliable imaging device for image video stream information acquisition in an industrial kiln in an extremely low light environment (the illumination is lower than 0.0001 Lux).
Example two
After the second device is installed, the second device starts to acquire video information, and the second device comprises the following specific steps:
(1) In order to ensure that the faint light high-temperature endoscope can stably work in a high-temperature environment in the furnace, firstly, an air cooling-water cooling double cooling system is started, and cold air are respectively conveyed into the equipment through an air inlet and a water inlet to continuously cool the equipment.
(2) After the power is on, the equipment starts to stably acquire the image information inside the industrial kiln.
(3) And adjusting a focusing lens group M4 to focus the inside of the industrial kiln, so that the endoscope can clearly image.
(4) Original images in the industrial kiln are collected by a large-view-field and large-light-input-quantity reverse long-focus objective lens group, pass through a Hopkins bar lens M3 capable of offsetting transmission distortion and deviation, and are transmitted to a CCD imaging chip U3 without light loss, so that the aim of realizing the faint-level imaging of the industrial endoscope is fulfilled.
(5) The CCD imaging chip U3 converts the image signal into an electrical signal as an input to the image brightness processing module U5.
(6) And carrying out three-dimensional digital comb filtering processing on the input image frame.
(7) And a time-space domain self-adaptive video highlight synthesis algorithm is utilized to synthesize the video streams of the plurality of image frames, so that the video brightness is effectively improved.
(8) And carrying out white balance processing on the image frame based on white balance of a multi-color dynamic domain algorithm to realize high-quality faint-level highlight video stream output of the industrial endoscope.
(9) And the video stream passing through the video coding and screen display module is transmitted to an upper computer S9 through an output processing unit S8, so that the output and display of the high-brightness video stream of the dim level industrial endoscope are realized.
The present invention has been described in terms of the preferred embodiment, and it is not intended to be limited to the embodiment. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A pyloric high temperature industrial endoscope, comprising a housing assembly, a pyloric-level optical imaging system mounted within the housing assembly, and an image brightness processing module (U5), wherein:
the image capturing system comprises an imaging system, a light level optical imaging system and a light level optical imaging system, wherein the imaging system is used for acquiring an image video stream in a furnace;
and the image brightness processing module (U5) is connected with the faint light level optical imaging system and is used for receiving the in-furnace image video stream output by the faint light level optical imaging system and carrying out brightness processing on the in-furnace image video stream so as to obtain a high-brightness and clear in-furnace image video stream.
2. The pyloric high temperature industrial endoscope of claim 1, wherein the housing assembly includes a front end housing assembly and a back end housing assembly, wherein:
the front-end housing assembly is of a double-layer structure, an air-cooling circulation channel (U1) is arranged on the outer layer of the double-layer structure, and a double-spiral wound water-cooling circulation flow channel (U2) is arranged on the outer side of the inner layer;
the rear end shell component is used for installing a CCD imaging chip (U3), an imaging driving circuit and an image brightness processing module (U5), and an air cooling air inlet (U4), a water cooling water inlet (U6) and a water cooling water outlet (U7) are formed in the rear end shell component.
3. The pyloric high temperature industrial endoscope of claim 2, further comprising a cooling system, wherein:
the cooling system adopts an air cooling-water cooling double cooling system, cold air in the air cooling-water cooling double cooling system enters an air cooling circulation channel (U1) from an air cooling air inlet, and air circulates in the air cooling circulation channel (U1) to form an air cooling effect;
water flow in the air-cooling and water-cooling double-cooling system enters the water-cooling circulation flow channel (U2) from the water-cooling water inlet (U6) and is discharged from the water-cooling water outlet (U7), and the discharged water flow can flow in again after being treated, so that the water-cooling effect is formed by circulation.
4. The pyloric high-temperature industrial endoscope according to claim 1, wherein the pyloric optical imaging system comprises an image capturing objective lens group (M1), a relay lens group (M2), a focusing lens group (M4), and a CCD imaging chip (U3) connected in sequence, wherein:
the image capturing objective lens group (M1) comprises a negative power lens group and a positive power lens group, the negative power lens group is used for scattering incident light, the acquired optical information with a large field angle and a large light incoming amount is uniformly scattered to the positive power lens group, initial conditions are provided for long-distance parallel transmission of the optical information, and the positive power lens group is used for re-gathering the scattered light to form light beams parallel to an optical axis;
the relay lens group (M2) is used for transmitting the in-furnace image acquired by the image taking objective lens group (M1) to a CCD imaging chip (U3);
the focusing mirror group (M4) is used for adjusting the focal length when the CCD imaging chip (U3) is blurred, so that the imaging is clear;
and the CCD imaging chip (U3) is used for converting the clear image transmitted by the focusing mirror group (M4) into a digital image.
5. A claustrophobic high temperature industrial endoscope according to any one of claims 1-4, wherein the image brightness processing module (U5) comprises an input processing unit (S7) and an FPGA video stream image processing unit (S1) and an output processing unit (S8) connected in sequence with the input processing unit (S7), wherein:
the input processing unit (S7) is used for decoding the in-furnace image video stream output by the optical imaging system at the dim level and storing the decoded in-furnace image video stream into a DDR SDRAM storage module (S5);
the FPGA video stream image processing unit (S1) is used for carrying out image preprocessing and brightness synthesis on the decoded in-furnace image video stream, and specifically comprises a MicroBlaze processing module (S4), a signal processing task triggering module (S3), a preset processing task module (S2) and a DDR SDRAM storage module (S5), wherein the signal processing task triggering module (S3) is connected with the MicroBlaze processing module (S4), and the FPGA video stream image processing unit (S1) is used for carrying out image preprocessing and brightness synthesis on the decoded in-furnace image video stream, and comprises:
the signal processing task triggering module (S3) is used for judging whether the number of image frames in the furnace image video stream meets the algorithm requirement in the preset processing task module (S2) or not according to the furnace image video stream stored by the DDR SDRAM storage module (S5), and sending a task command to the MicroBlaze processing module (S4) to realize task triggering, task configuration and task management;
the preset processing task module (S2) is used for pre-storing an algorithm of an image preprocessing and brightness synthesis processing task;
the MicroBlaze processing module (S4) is used for receiving task triggering, task configuration and task management commands sent by the signal processing task triggering module (S3), and carrying out image preprocessing and brightness synthesis on the decoded in-furnace image video stream according to an algorithm of an image preprocessing and brightness synthesis processing task stored in the preset processing task module (S2) in advance;
the DDR SDRAM storage module (S5) is used for storing the in-furnace image video stream decoded by the input processing unit (S7) and performing data interaction with the signal processing task triggering module (S3) and the MicroBlaze processing module (S4);
and the output processing unit (S8) is used for outputting the in-furnace image video stream processed by the FPGA video stream image processing unit (S1).
6. The hyperspectral high-temperature industrial endoscope of claim 5, wherein the preset processing task module (S2) comprises a three-dimensional digital comb filtering algorithm sub-module, a video highlight synthesis algorithm sub-module based on time-space domain adaptation, and a white balance algorithm sub-module based on a multi-color dynamic domain algorithm, which are connected in sequence, wherein:
the three-dimensional digital comb filtering algorithm submodule is used for carrying out three-dimensional digital comb filtering on the decoded in-furnace video stream;
the video highlight synthesis algorithm submodule based on the time-space domain self-adaption is used for carrying out video highlight synthesis on the video stream in the furnace after the three-dimensional digital comb filtering;
and the white balance algorithm submodule based on the multi-color dynamic domain algorithm is used for carrying out white balance processing on the in-furnace video stream after the video brightness is synthesized.
7. The pyloric high-temperature industrial endoscope of claim 6, wherein the video highlight synthesis algorithm sub-module based on time-space domain adaptation comprises a video frame acquisition sub-module, a non-linear transformation sub-module, an overlay sub-module and a synthesized video frame acquisition sub-module which are connected in sequence, wherein:
the video frame acquisition submodule is used for acquiring video frames in the three-dimensional comb-shaped filtered video stream, wherein the video frames comprise a current video frame and a video frame adjacent to the current video frame;
the nonlinear transformation submodule is used for carrying out nonlinear transformation on the video frame;
the superposition submodule is used for superposing the video frames subjected to the nonlinear transformation to obtain an enhanced image;
and the composite video frame acquisition submodule is used for optimizing the enhanced image by adopting a maximum posterior model to obtain a composite video frame so as to obtain a highlight video stream.
8. The pyloric high-temperature industrial endoscope of claim 7, wherein the composite video frame acquisition sub-module comprises a fidelity function calculation sub-module, a consistency function calculation sub-module, a regularization function calculation sub-module, a posterior probability calculation sub-module, and a composite video frame calculation sub-module, which are connected in sequence, wherein:
the fidelity function calculating submodule is used for calculating the fidelity function, wherein the fidelity function has a calculation formula as follows:
wherein,ψ(O t ,Y j ) Is a fidelity function and is used for measuring the optimized image O at the time t t And j time instant enhanced image Y j The similarity of (a) to (b) is,
is O t And Y j B and D represent the blur matrix and the down-sampled matrix, respectively, S represents the covariance matrix,
optimized image O representing time t t Enhanced image Y to time j j The motion compensation matrix of (a);
the consistency function calculation submodule is used for calculating a consistency function, wherein the calculation formula of the consistency function is as follows:
wherein,
representing the output optimized image at time t-1,
an optimized image O representing a consistency function and measuring the t moment t And output optimized image at time t-1
The similarity between the two or more of the images,
is O t And
the variance of the joint distribution of (a),
is the optimized image O at time t t Output optimized image to time t-1
The motion compensation matrix of (a);
the regularization function calculation sub-module is used for calculating a regularization function, wherein a calculation formula of the regularization function is as follows:
wherein, L (O) t ) For the regularization function, P is the size of the moving window, α is used to constrain the smoothness of the optimized image, H l And V j Indicating that the time t is optimized to the image O t Operators that move the l and j pixels in the horizontal and vertical directions, respectively;
the posterior probability calculation submodule is used for obtaining an optimized image O according to the fidelity function, the consistency function and the regularization function t And the calculation formula of the posterior probability is as follows:
wherein, p (O) t ) Is to optimize the image O t R is the video image frame number of the original video used by the output optimized image before the time t, b is the video image frame number of the original video used by the output optimized image after the time t;
the composite video frame calculation submodule is used for obtaining a composite video frame according to the posterior probability of the output optimized image, and the specific calculation formula is as follows:
9. the deep light industrial endoscope of claim 8, wherein the white balance algorithm sub-module comprises a space conversion sub-module, an absolute deviation mean calculation sub-module, a region to be processed acquisition sub-module, a mean and deviation calculation sub-module, a candidate white point acquisition sub-module and a fusion sub-module, which are connected in sequence, wherein:
the space conversion sub-module is used for converting image frames in the highlight video stream from an RGB space to an HSV space and a YCbCr space respectively to obtain an H channel, an S channel, a V channel, a Y channel, a Cb channel and a Cr channel;
the absolute deviation mean value calculation submodule is used for dividing the image frame into a preset number of areas and calculating the absolute deviation mean values of the H channel, the S channel, the Cb channel and the Cr channel of each area respectively;
the to-be-processed area acquisition submodule is used for acquiring the to-be-processed area needing white balance processing according to the absolute deviation mean value of the H channel, the S channel, the Cb channel and the Cr channel;
the mean value and deviation calculation submodule is used for calculating the mean values and deviations of an H channel, an S channel, a Cb channel and a Cr channel of the region to be processed;
the candidate white point obtaining submodule is used for obtaining candidate white points according to the mean values and the deviations of the H channel, the S channel, the Cb channel and the Cr channel of the area to be processed, and the calculation formula specifically comprises the following steps:
wherein, H (i, j), S (i, j), cb (i, j) and Cr (i, j) are respectively the i pixel and the j pixel of the H channel, the S channel, the Cb channel and the Cr channel, and the AvrH all 、AvrS all 、AvrCb all And AvrCr all Mean values of H channel, S channel, cb channel and Cr channel, devH all 、DevS all 、DevCb all And DevcCr all Variances of an H channel, an S channel, a Cb channel and a Cr channel respectively;
the fusion submodule is used for selecting a reference white point from the candidate white points and fusing the reference white point with the brightness information of the HSV space and the YCbCr space, and the calculation formula specifically comprises the following steps:
wherein img represents a fused image obtained by fusing brightness information of a reference white point and HSV space and YCbCr space, imgR, imgG and imgB represent channel images corresponding to an R channel, a G channel and a B channel respectively, theta represents multi-color-domain fusion degree, and V is max And Y max Respectively, the maximum value of lightness and the maximum value of brightness, R avgw 、G avgw 、B avgw The average pixel gray scale values of the R channel, the G channel and the B channel are respectively.
10. The pyloric high-temperature industrial endoscope according to claim 9, further comprising a display module for transmitting the highlight clear in-furnace image video stream to an upper computer (S9) for display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210816693.6A CN115396570B (en) | 2022-07-12 | 2022-07-12 | Claustrophote high-temperature industrial endoscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210816693.6A CN115396570B (en) | 2022-07-12 | 2022-07-12 | Claustrophote high-temperature industrial endoscope |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115396570A true CN115396570A (en) | 2022-11-25 |
| CN115396570B CN115396570B (en) | 2023-12-12 |
Family
ID=84116422
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210816693.6A Active CN115396570B (en) | 2022-07-12 | 2022-07-12 | Claustrophote high-temperature industrial endoscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115396570B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05268638A (en) * | 1992-03-23 | 1993-10-15 | Aiwa Co Ltd | Video signal processing device |
| US20070035706A1 (en) * | 2005-06-20 | 2007-02-15 | Digital Display Innovations, Llc | Image and light source modulation for a digital display system |
| US20090278921A1 (en) * | 2008-05-12 | 2009-11-12 | Capso Vision, Inc. | Image Stabilization of Video Play Back |
| US7944508B1 (en) * | 2007-06-08 | 2011-05-17 | Kolorific, Inc. | Method and system for automatic detection and suppression of cross-luma and cross-color in a component video signal |
| CN109283663A (en) * | 2018-11-16 | 2019-01-29 | 中南大学 | A kind of big visual field, high-resolution industrial endoscope optical system |
| CN112243091A (en) * | 2020-10-16 | 2021-01-19 | 微创（上海）医疗机器人有限公司 | Three-dimensional endoscope system, control method, and storage medium |
| CN113814212A (en) * | 2021-09-06 | 2021-12-21 | 中南大学 | Front lens online intelligent self-cleaning device and method based on lens fuzzy detection |
| CN114415356A (en) * | 2021-12-20 | 2022-04-29 | 核工业西南物理研究院 | Large-view-field long-optical-path imaging system based on double telecentric relay optics |
| CN114513607A (en) * | 2022-01-27 | 2022-05-17 | 中南大学 | Method, device and system for self-adjusting field range of high-temperature industrial endoscope |
-
2022
- 2022-07-12 CN CN202210816693.6A patent/CN115396570B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05268638A (en) * | 1992-03-23 | 1993-10-15 | Aiwa Co Ltd | Video signal processing device |
| US20070035706A1 (en) * | 2005-06-20 | 2007-02-15 | Digital Display Innovations, Llc | Image and light source modulation for a digital display system |
| US7944508B1 (en) * | 2007-06-08 | 2011-05-17 | Kolorific, Inc. | Method and system for automatic detection and suppression of cross-luma and cross-color in a component video signal |
| US20090278921A1 (en) * | 2008-05-12 | 2009-11-12 | Capso Vision, Inc. | Image Stabilization of Video Play Back |
| CN109283663A (en) * | 2018-11-16 | 2019-01-29 | 中南大学 | A kind of big visual field, high-resolution industrial endoscope optical system |
| CN112243091A (en) * | 2020-10-16 | 2021-01-19 | 微创（上海）医疗机器人有限公司 | Three-dimensional endoscope system, control method, and storage medium |
| CN113814212A (en) * | 2021-09-06 | 2021-12-21 | 中南大学 | Front lens online intelligent self-cleaning device and method based on lens fuzzy detection |
| CN114415356A (en) * | 2021-12-20 | 2022-04-29 | 核工业西南物理研究院 | Large-view-field long-optical-path imaging system based on double telecentric relay optics |
| CN114513607A (en) * | 2022-01-27 | 2022-05-17 | 中南大学 | Method, device and system for self-adjusting field range of high-temperature industrial endoscope |
Non-Patent Citations (3)
| Title |
|---|
| ZHIPENG CHEN ET AL.: ""A Novel Device for Optical Imaging of Blast Furnace Burden Surface: Parallel Low-Light-Loss Backlight High-Temperature Industrial Endoscope"", 《IEEE SENSORS JOURNAL》, vol. 16, no. 17, pages 2 * |
| ZUNHUI YI ET AL.: ""A Novel 3-D High-Temperature Industrial Endoscope With Large Field Depth and Wide Field"", 《IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT》, vol. 69, no. 9, pages 2 * |
| 禹璐;程德文;周伟;王涌天;刘小华;金国藩;: "大景深高清硬性内窥镜光学系统的优化设计", 光学学报, no. 11 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115396570B (en) | 2023-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7920172B2 (en) | Method of controlling an action, such as a sharpness modification, using a colour digital image | |
| CN107959778B (en) | Imaging method and device based on dual camera | |
| CN108712608B (en) | Terminal equipment shooting method and device | |
| CN109005364A (en) | Image formation control method, device, electronic equipment and computer readable storage medium | |
| EP2111038B1 (en) | Image processing apparatus and image processing method for reducing color blur | |
| CN109005342A (en) | Panorama shooting method, device and imaging device | |
| CN107948519A (en) | Image processing method, device and equipment | |
| EP2074816B1 (en) | Imaging apparatus | |
| CN107948500A (en) | Image processing method and device | |
| CN103139476B (en) | The control method of image-pickup device and image-pickup device | |
| CN108055452A (en) | Image processing method, device and equipment | |
| CN114342351B (en) | Apparatus, method and storage medium for capturing image data | |
| CN107613216B (en) | Focusing method, device, computer readable storage medium and electronic equipment | |
| CN108024054A (en) | Image processing method, device and equipment | |
| EP0940978A3 (en) | High resolution camera | |
| CN109194855A (en) | Imaging method, device and electronic equipment | |
| CN110166706A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN101256764A (en) | Frame region filters | |
| CN108833802A (en) | Exposal control method, device and electronic equipment | |
| CN107197152A (en) | Focusing method, device, computer-readable recording medium and mobile terminal | |
| CN107277299A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
| CN115396570B (en) | Claustrophote high-temperature industrial endoscope | |
| CN110930440B (en) | Image alignment method, device, storage medium and electronic equipment | |
| CN114660782B (en) | Fisheye lens for underwater monitoring, imaging device and image correction method and system | |
| CN109035489A (en) | Automobile data recorder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |