CN116600209A - Image quality optimization method, device, equipment and storage medium - Google Patents
Image quality optimization method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116600209A CN116600209A CN202310857686.5A CN202310857686A CN116600209A CN 116600209 A CN116600209 A CN 116600209A CN 202310857686 A CN202310857686 A CN 202310857686A CN 116600209 A CN116600209 A CN 116600209A
- Authority
- CN
- China
- Prior art keywords
- image
- adjusted
- delay
- exposure
- energy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 89
- 238000005457 optimization Methods 0.000 title abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 25
- 230000000737 periodic effect Effects 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 13
- 230000004927 fusion Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 5
- 230000006872 improvement Effects 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000750 progressive effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000007620 mathematical function Methods 0.000 description 2
- 101001116668 Homo sapiens Prefoldin subunit 3 Proteins 0.000 description 1
- 102100024884 Prefoldin subunit 3 Human genes 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/745—Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination
Abstract
The application discloses an image quality optimization method, an image quality optimization device and a storage medium, wherein the image quality optimization method comprises the following steps: acquiring a first image and a second image based on the exposure delay; adjusting the number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image; determining a read delay based on the adjusted number of delay lines and/or the adjusted exposure delay such that the read delay is an odd multiple of half an energy period of the light source, the light source providing energy to an exposure process of the first image and the second image based on the energy period; and performing exposure based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image. By the aid of the scheme, image quality can be improved.
Description
Technical Field
The present application relates to the field of image acquisition technologies, and in particular, to an image quality optimization method, apparatus, device, and storage medium.
Background
Flicker (Flicker) is a phenomenon in which light in a scene flickers in an imaging picture to generate undesirable stroboscopic fringes due to interaction between a time-modulated light source (e.g., a pulse-width modulated LED lamp) and an image sensor, and is generally eliminated in a single frame image by adjusting an exposure time.
However, in the process of image acquisition of an object moving at a high speed, the exposure time of the camera is limited to a very small range to ensure that the object in the image is clear, for example, 4ms or even smaller, so that the Flicker phenomenon is difficult to eliminate by adjusting the exposure time when the image acquisition of the object moving at the high speed is performed, the shooting requirement under a moving scene is not met, and the image quality is reduced.
Disclosure of Invention
The application provides at least one image quality optimization method, device, equipment and computer readable storage medium.
The first aspect of the present application provides an image quality optimization method, the method comprising: acquiring a first image and a second image based on the exposure delay; adjusting a number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image; determining a read delay based on the adjusted number of delay lines and/or the adjusted exposure delay such that the read delay is an odd multiple of a half of an energy period of a light source that provides energy to an exposure process of the first and second images based on the energy period; and carrying out exposure based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image.
In an embodiment, the step of adjusting the number of delay lines between the first image and the second image includes: acquiring the energy period; determining an adjustment parameter according to the energy period, the delay line number and/or the exposure delay; and adjusting the delay line number and/or the exposure delay based on the adjustment parameters to obtain the adjusted delay line number and/or the adjusted exposure delay.
In an embodiment, the energy period comprises a period frequency and a period function, and the step of determining the adjustment parameter based on the energy period, the number of delay lines and/or the exposure delay comprises: determining a sum of energies received by the first and second images from the light source according to the periodic frequency and the periodic function; and acquiring a target parameter corresponding to the delay line number and/or the exposure delay when the energy sum is a fixed value, and taking the target parameter as the adjustment parameter.
In an embodiment, the step of exposing based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image includes: exposing based on the reading delay to obtain the adjusted first image and the adjusted second image respectively; respectively extracting features of the adjusted first image and the adjusted second image to obtain a first image feature and a second image feature; the optimized image is determined from a difference between the first image feature and the second image feature.
In an embodiment, the step of exposing based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image includes: exposing based on the reading delay to obtain the adjusted first image and the adjusted second image respectively; acquiring pixel differences between corresponding areas of the adjusted first image and the adjusted second image; and determining the optimized image according to the comparison result of the pixel difference and a preset threshold value.
In an embodiment, the step of determining the optimized image according to the comparison result of the pixel difference and a preset threshold value includes: if the pixel difference is greater than or equal to the preset threshold value, performing average value calculation based on the pixel information of the corresponding areas of the adjusted first image and the adjusted second image, and taking the average value calculation result as the pixel information of the corresponding areas in the optimized image; and if the pixel difference is smaller than the preset threshold value, taking the pixel information of the corresponding region of the adjusted first image or the adjusted second image as the pixel information of the corresponding region in the optimized image.
In an embodiment, the method further comprises: acquiring exposure time of pixels of the same row in a row-by-row exposure process of the first image and the second image; an exposure delay between the first image and the second image is determined based on the exposure time of the same row of pixels.
A second aspect of the present application provides an image quality optimizing apparatus comprising: an acquisition module that acquires a first image and a second image based on an exposure delay; an adjustment module for adjusting the number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image; a delay determination module for determining a read delay based on the adjusted number of delay lines and/or the adjusted exposure delay such that the read delay is an odd multiple of half an energy period of a light source that provides energy to an exposure process of the first image and the second image based on the energy period; and the exposure fusion module is used for carrying out exposure based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image.
A third aspect of the present application provides an electronic device comprising a memory and a processor for executing program instructions stored in the memory to implement the above-described image quality optimization method.
A fourth aspect of the present application provides a computer readable storage medium having stored thereon program instructions which, when executed by a processor, implement the above-described image quality optimization method.
According to the scheme, the reading delay between the first image and the second image is determined by adjusting the delay line number and/or the exposure delay of the first image and the second image, so that the reading delay is odd multiple of half an energy period of the light source, then the exposure is carried out according to the reading delay, the adjusted first image and the adjusted second image are obtained, the light source energy acquired by the adjusted first image and the adjusted second image through the energy period is stable, the adjusted first image and the adjusted second image are fused, and therefore the problems that the light energy received by each image is different during exposure, a flicker phenomenon occurs and the like can be avoided, and the improvement of the image quality is realized.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic view of an application scenario of an exemplary embodiment of an image quality optimization method of the present application;
FIG. 2 is a flow chart of another exemplary embodiment of an image quality optimization method of the present application;
FIG. 3 is a schematic illustration of an exposure process in the image quality optimization method of the present application;
FIG. 4 is a schematic diagram showing the variation of exposure light energy when exposing different pixel rows in an image according to the image quality optimization method of the present application;
FIG. 5 is a flow chart of determining adjustment parameters according to energy cycle, read delay and/or exposure delay in the image quality optimization method of the present application;
FIG. 6 is a block diagram of an image quality optimization apparatus shown in an exemplary embodiment of the present application;
FIG. 7 is a schematic diagram of an embodiment of an electronic device of the present application;
fig. 8 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application.
Detailed Description
The following describes embodiments of the present application in detail with reference to the drawings.
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship. Further, "a plurality" herein means two or more than two. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
At present, common image exposure modes in the image acquisition process comprise global exposure and roller shutter exposure, wherein the roller shutter exposure is also called progressive exposure, the sensor can perform progressive scanning and progressive exposure until the photosensitive assembly exposes all pixel points to the light, but if the light source intensities in the various line exposure processes are different, a Flicker phenomenon can occur in the image.
Therefore, referring to fig. 1, fig. 1 is a schematic view of an application scenario of an exemplary embodiment of an image quality optimization method according to the present application, in a line-by-line exposure process of an image, although exposure time of each line is the same, exposure energy obtained by each pixel line may be different due to a periodic variation of a light source following an alternating current frequency, so that a strobe stripe on the image is generated, resulting in a degradation of image quality.
Referring to fig. 2, fig. 2 is a flowchart illustrating another exemplary embodiment of the image quality optimization method of the present application. Specifically, the method may include the steps of:
step S210, acquiring a first image and a second image based on the exposure delay.
The first image and the second image are two adjacent frames of images acquired sequentially according to the exposure sequence, and for convenience of understanding and explanation, the following description will take the first image and the second image being exposed before the second image as an example, wherein the "first" and the "second" in the first image and the second image are only differences made in terms of names for distinguishing the two frames of images, other attributes of the two frames of images are not limited, for example, exposure time for acquiring the first image and exposure time for acquiring the second image may be the same or different, and a plurality of first images and a plurality of second images may exist in an application environment of the present application.
Wherein, referring to FIG. 3, FIG. 3 is a schematic diagram of the exposure process and exposure delay in the image quality optimization method of the present applicationRefers to the difference between the exposure start times of adjacent rows of pixels in a frame of an image, such as the difference between the exposure start times of adjacent two rows of exposed rows in a first image or the difference between the exposure start times of adjacent two rows of short exposed rows in a second image, when the images are exposed row by row.
Illustratively, the first image and the second image are acquired sequentially by performing line-by-line exposure based on a preset exposure delay.
Step S220, adjusting the number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image.
The number of delay lines is generally expressed in VBP, and is set based on the exposure delay (minimum time unit).
In step S230, the reading delay is determined based on the adjusted number of delay lines and/or the adjusted exposure delay such that the reading delay is an odd multiple of half the energy period of the light source, the light source providing energy to the exposure process of the first image and the second image based on the energy period.
Wherein the reading delay refers to the difference between the time of reading the first image and the time of reading the second image, and in an ideal state, the time of ending the image exposure corresponds to the time of reading the image, and the reading delay between the first image and the second image can be determined by the product of the delay line number and the exposure delay; for a read delay tdelay_1_2 between the first image and the second image, the read delay is typically in terms of exposure delay Setting for reference is exposure delay +.>Thus, in the present application, the read delay tdelay_1_2 is the delay line number VBP and the exposure delay +.>The corresponding time is known from the sensor manual that the correlation between the read delay and the number of delay lines and exposure delay is tdelay_1_2=vbp #The number of delay lines and/or the exposure delay are adjusted so that the read delay is adjusted.
It should be noted that, the number of delay lines and the exposure delay are both associated with the read delay, that is, the read delay can be determined by the number of delay lines and the exposure delay, and thus, adjusting the number of delay lines and/or adjusting the exposure delay can adjust the read delay.
The light source can be a light supplementing lamp of the image acquisition device or other light sources in an image acquisition scene, and the energy period of the light source is equivalent to the voltage period of alternating current, so that for grids of different countries, the energy period of the light source can be set based on the geographic position of the image acquisition device, for example, the alternating current frequency of European countries is 50hz, the alternating current frequency of America is 60hz, the alternating current frequency of China is 50hz, and the image quality optimization method can be more universal by corresponding adjustment according to different frequencies.
Illustratively, by adjusting the number of delay lines and/or adjusting the exposure delay, the reading delay determined based on the adjusted number of delay lines and/or the adjusted exposure delay is an odd multiple of half an energy period of the light source, so that the light source energy acquired by the adjusted first image and the adjusted second image, which are obtained by exposure based on the reading delay, can be stabilized, and the inconformity of the brightness of the first image and the second image and the appearance of a Flicker phenomenon are avoided.
Specifically, referring to fig. 4, fig. 4 is a schematic diagram of the change of exposure light energy when different pixel rows in an image are exposed in the image quality optimization method of the present application, the mathematical expression of the exposure light energy is obtained by performing integral calculation based on the energy period and the exposure time of each pixel row, and the delay row number and/or the exposure delay are adjusted to make the determined reading delay be the same as the odd multiple of half the energy period of the light source, so as to realize that the light energy received by each pixel row in the first image and the second image from the light source is stable in the corresponding image row-by-row exposure process following the change of the energy period of the light source, and avoid the difference of each pixel row in the first image and the second image due to the change of the energy period.
It should be noted that, in the current method capable of solving the flicker phenomenon, the image capturing frame rate is usually adjusted to match the image capturing frame rate with the energy period so as to eliminate the flicker phenomenon, but in some application scenarios, for example, shooting an object moving at a high speed, the image capturing frame rate cannot be adjusted at will, and adjusting the image capturing frame rate affects the quality of the image and the video, so that the method is not a preferred method for eliminating the flicker.
Step S240, exposure is performed based on the reading delay, an adjusted first image and an adjusted second image are obtained, and the adjusted first image and the adjusted second image are fused, so that an optimized image is obtained.
Specifically, performing exposure based on the reading delay is actually equivalent to performing exposure based on the adjusted delay line number and/or the adjusted exposure delay, so as to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image; the energy of the periodic light-compensating light sources received by adjacent lines of the fused image is the same without considering other influencing factors of the scene, so that the flicker phenomenon cannot occur.
It can be seen that the application determines the reading delay between the first image and the second image by adjusting the delay line number and/or the exposure delay of the first image and the second image, so that the reading delay is odd times of half the energy period of the light source, and then exposes according to the reading delay to obtain the adjusted first image and the adjusted second image, so that the light source energy acquired by the adjusted first image and the adjusted second image through the energy period is stable, and the adjusted first image and the adjusted second image are fused, thereby avoiding the problems of different received light energy, flickering phenomenon and the like of each image during exposure, and realizing the improvement of the image quality.
On the basis of the above embodiment, the present embodiment further describes one of the scenarios to which the image quality optimization method of the present application can be applied. Specifically, the method can be applied to a Digital overlay technology, the Digital overlay technology (DOL) is a popular sensor HDR (High Dynamic Range) technology or a sensor WDR (Wide Dynamic Rang) technology at present, an image acquisition device such as a camera continuously outputs images from underexposure to overexposure in time, namely, long exposure and short exposure are respectively carried out to obtain a long exposure image and a short exposure image, then the long exposure image and the short exposure image are subjected to image fusion to obtain an HDR image, so that dark parts and bright parts of the images can be represented in the same image, and further clear images are obtained, wherein the long exposure image and the short exposure image are equivalent to a first image and a second image in the application, and the first image and the second image in the subsequent embodiment can also be substituted into the long exposure image and the short exposure image to realize.
The long exposure image refers to a long frame exposure image in DOL based on WDR, and it should be noted that in DOL technology, long exposure and short exposure are controlled to be alternately performed, after each row of pixels completes long exposure and reads data corresponding to the long exposure, short exposure is correspondingly performed on the row of pixels, and the long, short exposure process and the reading process of each row of pixels do not affect the long, short exposure process and the affecting process of other rows of pixels.
The short exposure image refers to a short frame exposure image in DOL technology.
The reading delay between the long exposure image and the short exposure image can be determined according to the difference between the reading time of the long exposure image and the reading time of the short exposure image, and the specific data relationship and the calculation process can be similarly referred to the description of the first image and the second image in the embodiment of the present application, which is not repeated here.
On the basis of the above-described embodiments, the steps of adjusting the number of delay lines between the first image and the second image are exemplarily described in the embodiments of the present application. Specifically, the method of the embodiment comprises the following steps:
acquiring an energy period; determining an adjustment parameter according to the energy period, the delay line number and/or the exposure delay; and adjusting the delay line number and/or the exposure delay based on the adjustment parameters to obtain the adjusted delay line number and/or the adjusted exposure delay.
As described in connection with the foregoing embodiments, if the read delay determined by the delay line number and the exposure delay is made to be an odd multiple of the half energy period of the light source, the adjustment parameter for adjusting the delay line number and/or the exposure delay may be determined according to the energy period of the light source.
Specifically, according to the change condition (periodic function) of the energy period and the currently acquired delay line number and exposure delay, determining the adjustment parameters of the delay line number and/or the exposure delay; adjusting the delay line number and/or the exposure delay through adjusting parameters, so that the read delay determined by the adjusted delay line number and/or the adjusted exposure delay is an odd multiple of half an energy period of the light source; for example, the adjustment parameters may be obtained by calculating a mathematical expression of the energy period, adjusting the number of delay lines and/or the exposure delay with reference to the adjustment parameters, and determining the reading delay based on the adjusted number of delay lines and/or the adjusted exposure delay, where it is noted that one or more of the adjustment parameters may be used for three implementable modes of adjusting the number of delay lines, adjusting the exposure delay, adjusting the number of delay lines, and the exposure delay.
Based on the above embodiments, the steps of determining the adjustment parameters according to the energy period, the reading delay and/or the exposure delay are exemplarily described using a flowchart as shown in fig. 5, where the energy period includes a period frequency and a period function. Specifically, the method of the embodiment comprises the following steps:
step S510, determining the energy sum received by the first image and the second image from the light source according to the periodic frequency and the periodic function.
The periodic frequency refers to the frequency of the light source for providing light energy, and for the lighting equipment using alternating current, the periodic frequency corresponds to the frequency of the alternating current, for example, for a power grid of different countries, the alternating current frequency of european countries is 50hz, the alternating current frequency of united states is 60hz, the alternating current frequency of china is 50hz, and corresponding adjustment according to different periodic frequencies can make the image quality optimization method of the present application more universal, and for convenience of explanation, the periodic frequency is 50hz in the following description.
The periodic function refers to expressing the energy period in a mathematical function form, taking sine alternating current as an example, and the mathematical function of the alternating current is equivalent to a sine trigonometric function, namely, the mathematical expression of the alternating current is as follows:
Wherein, the liquid crystal display device comprises a liquid crystal display device,is an alternating voltage>Is a voltage effective value, < >>Is angular frequency, +.>，/>The power frequency is 50hz, the voltage period is the reciprocal of the power frequency, is +.>T is the alternating current power supply time, < >>For the purpose of easy subsequent calculation and description, the angle of the primary phase is +>And is 0.
From mathematical expression of powerIt can be seen that->The mathematical expression for the instantaneous power p (t) at time t is:
wherein, the liquid crystal display device comprises a liquid crystal display device,for effective power, +.>R is the resistance of the corresponding instantaneous voltage added at the two ends of the electric equipment.
Is provided with、/>F is the frequency of the light source energy, the mathematical expression of the light source brightness at time t is:
referring to fig. 4, based on the exposure time corresponding to each of the first image and the second image, performing integral calculation on L (t) to obtain the exposure light energy corresponding to each of the pixels in the ith row of the nth frameIs expressed as:
where expt is the exposure time of the i-th row of pixels, and expt is determined by the exposure start time t1 and the exposure end time t2 of the i-th row.
For the ith+jth row of pixels of the nth frame, it is different by j from the exposure start time t1 of the ith row of pixels of the nth frameJ +.>Thus, the exposure light energy of the i+j-th row pixels of the nth frame +. >Is expressed as:
in a summary of the present invention it is known that,and->A periodic function corresponding to the energy period of the light source.
Step S520, obtaining a target parameter corresponding to the delay line number and/or the exposure delay when the energy sum is a fixed value, and taking the target parameter as an adjustment parameter.
The target parameter is a parameter for adjusting the number of delay lines and/or exposure delay.
Specifically, as can be seen from the combination of step S510, if expt is made to be an integer multiple of the energy period, that isWherein k is a positive integer, the exposure process of the image is considered to be matched with the energy period, and the energy obtained by the image in the exposure process is a fixed value, namely +.>Therefore, the flicker phenomenon can not be generated; if expt is not an integer multiple of the energy frequency, then it is necessary toAnd fixing the light energy obtained in the exposure process of the first image and the second image by adjusting the delay line number and/or adjusting the exposure delay.
Further, the exposure problem when expt is not an integer multiple of the energy frequency is subdivided, the received energy sum of the first image and the second image is equivalent to the energy sum received after the first image is exposed and the second image is exposed on the same pixel row, if the sensor is adjusted to be operated in the wide dynamic mode, the exposure time of each pixel row of the adjusted first image and the adjusted second image can be fixed to expt, m1 is the reading time of the ith pixel row in the first image, m2 is the reading time of the ith pixel row in the second image, and the reading delay tdelay_1_2=m2-m 1 is obtained; in an ideal state, the exposure end time of the image is the reading time of the image, so if t1 is the exposure start time of the first image and t2 is the exposure end time of the first image, the exposure end time of the second image and the exposure end time of the first image differ by tdelay_1_2, the exposure end time of the second image is t2+tdelay_1_2, and similarly, the exposure start time of the second image is t1+tdelay_1_2; the sum of the energies acquired by the first and second images of the adjacent two frames is equivalent to the sum of the energies of the pixel rows i_1 and i_2 in the first and second images of the adjacent two frames based on the same exposure time, wherein i_1 represents the ith row of pixels in the first image, i_2 represents the ith row of pixels in the second image, and the sum of the energies acquired by the first and second images of the adjacent two frames in the ith pixel row The mathematical expression is:
from the foregoing embodiments, it is known that there is a timing difference between t1 and t2, and thereforeIs not 0, i.e. the sin function in the formula is not 0, < >>Corresponding to expt, i.e. expt is not 0, if it is necessary to let the energy of the ith row of the nth frame +.>Energy +.j of the ith row of the nth frame>Is equal to +.>The corresponding target parameter should be such thatIs 0, i.e. wherein->K may be a positive odd number and f is the frequency of the source energy. Thus, adjust->The energy sum of the first image and the second image is a value which is only related to the exposure time expt and is irrelevant to the exposure time, so that the influence of the periodical change of the light source on the light source due to the light source energy at different exposure times is avoided, and a flicker phenomenon is generated, wherein->Is composed of delay line number VBP and exposure delay +.>Determining, adjustingThe way of (a) may be to adjust the number of delay lines and/or the exposure delay.
On the basis of the above-described embodiments, the embodiments of the present application exemplarily explain the step of determining the read delay based on the adjusted delay line number and/or the adjusted exposure delay. Specifically, the method of the embodiment comprises the following steps:
as can be seen from the relevant data of the sensor, VBP1 is taken as an exampleThe number of delay lines before adjustment between the first image and the second image, the number of delay lines and/or the read delay before adjustment of the exposure delay The calculation method of (1) is that。
Taking a method of adjusting only the delay line number as an example, please refer to fig. 3, if the current exposure delayTo satisfy the multiple relationships mentioned in the preceding examples, for example k=1, then +.>(where f is 2 times the power frequency, i.e. 100 hz), the adjusted tdelay_1_2=5 ms, the adjusted delay line number +.>To validate parameters in registers during the next image exposure; in addition, the method for adjusting the reading delay may also be to adjust the number of delay lines and the exposure delay, or only adjust the exposure delay, which is not described herein, and the specific setting method may refer to different operation manuals for different sensors, which is not described herein.
On the basis of the above embodiment, the embodiment of the present application exemplarily describes the steps of performing exposure based on the reading delay to obtain the adjusted first image and the adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain the optimized image. Specifically, the method of the embodiment comprises the following steps:
exposing based on the reading delay to obtain an adjusted first image and an adjusted second image respectively; respectively extracting features of the adjusted first image and the adjusted second image to obtain a first image feature and a second image feature; an optimized image is determined based on the difference between the first image feature and the second image feature.
It should be noted that, the feature extraction is performed on the adjusted first image and the adjusted second image, and the difference between the first image feature and the second image feature of the adjusted first image and the adjusted second image is compared, so as to determine whether the adjusted first image and the adjusted second image have larger feature differences, so that a flicker phenomenon still exists after the adjusted first image and the adjusted second image are fused, and determine whether to continuously adjust the delay line number and/or the exposure delay; if the feature difference between the first image feature and the second image feature is larger than or equal to a preset feature difference threshold, fusing the currently acquired adjusted first image and the adjusted second image to obtain an optimized image.
On the basis of the above embodiment, the embodiment of the present application exemplarily describes the steps of performing exposure based on the reading delay to obtain the adjusted first image and the adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain the optimized image. Specifically, the method of the embodiment comprises the following steps:
exposing based on the reading delay to obtain an adjusted first image and an adjusted second image respectively; acquiring pixel differences between corresponding areas of the adjusted first image and the adjusted second image; and determining an optimized image according to the comparison result of the pixel difference and the preset threshold value.
The preset threshold is used for judging whether the pixel difference value between the corresponding areas of the adjusted first image and the adjusted second image is too large, and if the pixel difference value between the corresponding areas of the adjusted first image and the adjusted second image is larger than or equal to the preset threshold, a flicker phenomenon may occur after fusion.
Specifically, pixel information Img1 of the adjusted first image and pixel information Img2 of the adjusted second image are obtained, and a difference value between Img1 and Img2 is compared with a preset threshold deltaI to determine pixel information of an optimized image.
It should be noted that, in the acquired complete image (including the first image and the second image), all areas may be affected by the energy period of the light source, or some areas may be affected by the energy period of the light source, but the area where the light source is located in the first image and the area where the light source is located in the second image generally correspond to each other, and similarly, the area where the light source is located in the adjusted first image and the area where the light source is located in the adjusted second image also correspond to each other, so that whether a flicker phenomenon occurs may be determined by the pixel difference value and the preset threshold value between the adjusted first image and the adjusted corresponding area of the second image.
On the basis of the above embodiments, the steps of determining an optimized image according to the comparison result of the pixel difference and the preset threshold value are exemplarily described in the embodiments of the present application. Specifically, the method of the embodiment comprises the following steps:
if the pixel difference is greater than or equal to a preset threshold value, carrying out mean value calculation based on the pixel information of the corresponding areas of the adjusted first image and the adjusted second image, and taking the mean value calculation result as the pixel information of the corresponding areas in the optimized image; and if the pixel difference is smaller than the preset threshold value, taking the pixel information of the corresponding region of the adjusted first image or the adjusted second image as the pixel information of the corresponding region in the optimized image.
The corresponding region may include one pixel point or may include a plurality of pixel points.
Specifically, taking pixel-by-pixel calculation as an example, traversing each pixel in the adjusted first image and the adjusted second image, obtaining pixel information Img1 of each pixel in the adjusted first image and pixel information Img2 of each pixel in the adjusted second image, and optimizing pixel information img3= (img1+img2)/2 of each pixel in the adjusted first image and the adjusted second image corresponding to the pixel coordinate if img1-img2 is greater than or equal to a preset threshold deltaI, wherein the two pixels corresponding to the pixel coordinate in the adjusted first image and the adjusted second image are affected by the energy period of the light source, so that a fliker phenomenon may exist; if Img1-Img2 is smaller than the preset threshold deltaI, it means that two pixel points corresponding to the pixel coordinates in the adjusted first image and the adjusted second image are not affected by the energy period of the light source, and the pixel information img3=img1 or img3=img2 of the pixel point corresponding to the pixel coordinates in the optimized image.
Further, taking pixel-by-pixel region calculation as an example, the above method describes how to perform pixel-by-pixel difference calculation and threshold comparison based on pixel information of pixels with the same pixel coordinates in the adjusted first image and the adjusted second image, and determine pixel information of pixels with the same pixel coordinates in the optimized image; the method can be extended to how to calculate the difference value and compare the threshold value of each pixel region based on the pixel information of the same pixel region in the adjusted first image and the adjusted second image, and determine the pixel information of the same pixel region in the optimized image, wherein the pixel information of the pixel region is determined by the pixel information of all the pixel points in the pixel region.
On the basis of the above-described embodiments, the steps of acquiring exposure delay are exemplarily described in the embodiments of the present application. Specifically, the method of the embodiment comprises the following steps:
acquiring exposure time of pixels of the same row in a row-by-row exposure process of the first image and the second image; an exposure delay between the first image and the second image is determined based on the exposure time of the same row of pixels.
The exposure time includes an exposure start time, an exposure duration, and an exposure end time.
Illustratively, in a line-by-line exposure of an image in which n lines of pixels are present, the exposure start time and/or the exposure end time of each line of pixels in the image are different due to the line-by-line exposure, taking the exposure start time as an example, the exposure start time of the 1 st line of pixels and the exposure start time of the 2 nd line of pixels are differentThe exposure delay between the 1 st row and the 2 nd row is the same as the exposure delay of any two adjacent rows of pixels between n pixel rows in an image, i.e. for a single imageWhen the image is exposed line by line, it is +.>Typically unchanged.
It should be further noted that the main body of execution of the image quality optimization method may be an image quality optimization apparatus, for example, the image quality optimization method may be executed by a terminal device or a server or other processing device, where the terminal device may be a User Equipment (UE), a computer, a mobile device, a User terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like. In some possible implementations, the image quality optimization method may be implemented by way of a processor invoking computer readable instructions stored in a memory.
Fig. 6 is a block diagram of an image quality optimizing apparatus according to an exemplary embodiment of the present application. As shown in fig. 6, the exemplary image quality optimizing apparatus 600 includes: the system comprises an acquisition module 610, an adjustment module 620, a delay determination module 630 and an exposure fusion module 640. Specifically:
the acquisition module 610 acquires the first image and the second image based on the exposure delay.
An adjustment module 620 for adjusting the number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image.
The delay determining module 630 is configured to determine the reading delay based on the adjusted delay line number and/or the adjusted exposure delay such that the reading delay is an odd multiple of half an energy period of the light source, and the light source provides energy to the exposure process of the first image and the second image based on the energy period.
The exposure fusion module 640 is configured to perform exposure based on the read delay, obtain an adjusted first image and an adjusted second image, and fuse the adjusted first image and the adjusted second image to obtain an optimized image.
In the image quality optimizing device, the reading delay between the first image and the second image is determined by adjusting the delay line number and/or the exposure delay of the first image and the second image, so that the reading delay is odd times of half an energy period of a light source, then exposure is carried out according to the reading delay to obtain an adjusted first image and an adjusted second image, the light source energy acquired by the adjusted first image and the adjusted second image through the energy period is stable, and the adjusted first image and the adjusted second image are fused, so that the problems that the light energy received by each image is different during exposure, a flicker phenomenon occurs and the like can be avoided, and the image quality is improved.
The functions of each module may refer to an embodiment of an image quality optimization method, which is not described herein.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the application. The electronic device 700 comprises a memory 701 and a processor 702, the processor 702 being arranged to execute program instructions stored in the memory 701 to implement the steps of any of the above described image quality optimization method embodiments. In one particular implementation scenario, electronic device 700 may include, but is not limited to: the microcomputer and the server, and the electronic device 700 may also include a mobile device such as a notebook computer and a tablet computer, which is not limited herein.
In particular, the processor 702 is used to control itself and the memory 701 to implement the steps in any of the image quality optimization method embodiments described above. The processor 702 may also be referred to as a CPU (Central Processing Unit ). The processor 702 may be an integrated circuit chip with signal processing capabilities. The processor 702 may also be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 702 may be commonly implemented by an integrated circuit chip.
According to the scheme, the reading delay between the first image and the second image is determined by adjusting the delay line number and/or the exposure delay of the first image and the second image, so that the reading delay is odd multiple of half an energy period of the light source, then the exposure is carried out according to the reading delay, the adjusted first image and the adjusted second image are obtained, the light source energy acquired by the adjusted first image and the adjusted second image through the energy period is stable, the adjusted first image and the adjusted second image are fused, and therefore the problems that the light energy received by each image is different during exposure, a flicker phenomenon occurs and the like can be avoided, and the improvement of the image quality is realized.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an embodiment of a computer readable storage medium according to the present application. The computer readable storage medium 800 stores program instructions 801 that can be executed by a processor, the program instructions 801 being for implementing the steps in any of the image quality optimization method embodiments described above.
According to the scheme, the reading delay between the first image and the second image is determined by adjusting the delay line number and/or the exposure delay of the first image and the second image, so that the reading delay is odd multiple of half an energy period of the light source, then the exposure is carried out according to the reading delay, the adjusted first image and the adjusted second image are obtained, the light source energy acquired by the adjusted first image and the adjusted second image through the energy period is stable, the adjusted first image and the adjusted second image are fused, and therefore the problems that the light energy received by each image is different during exposure, a flicker phenomenon occurs and the like can be avoided, and the improvement of the image quality is realized.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Claims (10)
1. A method of optimizing image quality, the method comprising:
acquiring a first image and a second image based on the exposure delay;
adjusting a number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image;
determining a read delay based on the adjusted number of delay lines and/or the adjusted exposure delay such that the read delay is an odd multiple of a half of an energy period of a light source that provides energy to an exposure process of the first and second images based on the energy period;
and carrying out exposure based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image.
2. The method of claim 1, wherein the step of adjusting the number of delay lines between the first image and the second image comprises:
acquiring the energy period;
determining an adjustment parameter according to the energy period, the delay line number and/or the exposure delay;
and adjusting the delay line number and/or the exposure delay based on the adjustment parameters to obtain the adjusted delay line number and/or the adjusted exposure delay.
3. The method according to claim 2, wherein the energy period comprises a period frequency and a period function, the step of determining an adjustment parameter based on the energy period, the number of delay lines and/or the exposure delay comprising:
determining a sum of energies received by the first and second images from the light source according to the periodic frequency and the periodic function;
and acquiring a target parameter corresponding to the delay line number and/or the exposure delay when the energy sum is a fixed value, and taking the target parameter as the adjustment parameter.
4. The method of claim 1, wherein the exposing based on the read delay results in an adjusted first image and an adjusted second image, and the step of fusing the adjusted first image and the adjusted second image results in an optimized image, comprising:
exposing based on the reading delay to obtain the adjusted first image and the adjusted second image respectively;
respectively extracting features of the adjusted first image and the adjusted second image to obtain a first image feature and a second image feature;
The optimized image is determined from a difference between the first image feature and the second image feature.
5. The method of claim 1, wherein the exposing based on the read delay results in an adjusted first image and an adjusted second image, and the step of fusing the adjusted first image and the adjusted second image results in an optimized image, comprising:
exposing based on the reading delay to obtain the adjusted first image and the adjusted second image respectively;
acquiring pixel differences between corresponding areas of the adjusted first image and the adjusted second image;
and determining the optimized image according to the comparison result of the pixel difference and a preset threshold value.
6. The method of claim 5, wherein the step of determining the optimized image based on the comparison of the pixel differences and a preset threshold comprises:
if the pixel difference is greater than or equal to the preset threshold value, performing average value calculation based on the pixel information of the corresponding areas of the adjusted first image and the adjusted second image, and taking the average value calculation result as the pixel information of the corresponding areas in the optimized image;
And if the pixel difference is smaller than the preset threshold value, taking the pixel information of the corresponding region of the adjusted first image or the adjusted second image as the pixel information of the corresponding region in the optimized image.
7. The method according to claim 1, wherein the method further comprises:
acquiring exposure time of pixels of the same row in a row-by-row exposure process of the first image and the second image;
an exposure delay between the first image and the second image is determined based on the exposure time of the same row of pixels.
8. An image quality optimizing apparatus, comprising:
an acquisition module that acquires a first image and a second image based on an exposure delay;
an adjustment module for adjusting the number of delay lines between the first image and the second image, and/or adjusting the exposure delay of the first image and the second image;
a delay determination module for determining a read delay based on the adjusted number of delay lines and/or the adjusted exposure delay such that the read delay is an odd multiple of half an energy period of a light source that provides energy to an exposure process of the first image and the second image based on the energy period;
And the exposure fusion module is used for carrying out exposure based on the reading delay to obtain an adjusted first image and an adjusted second image, and fusing the adjusted first image and the adjusted second image to obtain an optimized image.
9. An electronic device comprising a memory and a processor for executing program instructions stored in the memory to implement the method of any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon program instructions, which when executed by a processor, implement the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310857686.5A CN116600209B (en) | 2023-07-13 | 2023-07-13 | Image quality optimization method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310857686.5A CN116600209B (en) | 2023-07-13 | 2023-07-13 | Image quality optimization method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116600209A true CN116600209A (en) | 2023-08-15 |
| CN116600209B CN116600209B (en) | 2023-09-29 |
Family
ID=87611956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310857686.5A Active CN116600209B (en) | 2023-07-13 | 2023-07-13 | Image quality optimization method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116600209B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003032551A (en) * | 2001-07-11 | 2003-01-31 | Sony Corp | Solid-state imaging device and its drive method, and camera system |
| CN102695004A (en) * | 2012-05-30 | 2012-09-26 | 昆山锐芯微电子有限公司 | CMOS image sensor and exposure control method thereof |
| JP2020010317A (en) * | 2018-06-29 | 2020-01-16 | キヤノン株式会社 | Imaging apparatus, control method of the same, and program |
| CN112543261A (en) * | 2020-12-08 | 2021-03-23 | 浙江大华技术股份有限公司 | Image quality improving method and device and computer readable storage medium |
| CN116091306A (en) * | 2023-03-17 | 2023-05-09 | 山东云海国创云计算装备产业创新中心有限公司 | Image scaling optimization method, system, storage medium and device |
-
2023
- 2023-07-13 CN CN202310857686.5A patent/CN116600209B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003032551A (en) * | 2001-07-11 | 2003-01-31 | Sony Corp | Solid-state imaging device and its drive method, and camera system |
| CN102695004A (en) * | 2012-05-30 | 2012-09-26 | 昆山锐芯微电子有限公司 | CMOS image sensor and exposure control method thereof |
| JP2020010317A (en) * | 2018-06-29 | 2020-01-16 | キヤノン株式会社 | Imaging apparatus, control method of the same, and program |
| CN112543261A (en) * | 2020-12-08 | 2021-03-23 | 浙江大华技术股份有限公司 | Image quality improving method and device and computer readable storage medium |
| CN116091306A (en) * | 2023-03-17 | 2023-05-09 | 山东云海国创云计算装备产业创新中心有限公司 | Image scaling optimization method, system, storage medium and device |
Non-Patent Citations (1)
| Title |
|---|
| 梁楠;贺强民;李博;李涛;: "高行频混合域TDI累加成像技术研究", 航天返回与遥感, no. 01 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116600209B (en) | 2023-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111028189B (en) | Image processing method, device, storage medium and electronic equipment | |
| CN108174118B (en) | Image processing method and device and electronic equipment | |
| WO2019085705A1 (en) | Method and device for obtaining exposure compensation value of high-dynamic-range image | |
| CN108737738B (en) | Panoramic camera and exposure method and device thereof | |
| CN106713755B (en) | Panoramic image processing method and device | |
| US10074165B2 (en) | Image composition device, image composition method, and recording medium | |
| US11431915B2 (en) | Image acquisition method, electronic device, and non-transitory computer readable storage medium | |
| EP3162047B1 (en) | Flash collision detection, compensation, and prevention | |
| WO2012170462A2 (en) | Automatic exposure correction of images | |
| US10609265B2 (en) | Methods and apparatus for synchronizing camera flash and sensor blanking | |
| US10721415B2 (en) | Image processing system with LED flicker mitigation | |
| US10972676B2 (en) | Image processing method and electronic device capable of optimizing hdr image by using depth information | |
| CN112383718A (en) | Image exposure method and device and electronic equipment | |
| CN116600209B (en) | Image quality optimization method, device, equipment and storage medium | |
| CN112714301A (en) | Dual-mode image signal processor and image sensor | |
| CN109120856B (en) | Camera shooting method and device | |
| WO2019244667A1 (en) | Information processing device, information processing method, and program | |
| US20190052803A1 (en) | Image processing system, imaging apparatus, image processing apparatus, control method, and storage medium | |
| EP3162046A1 (en) | Flash collision detection, compensation, and prevention | |
| CN116647760B (en) | Wide dynamic implementation method, device, camera and storage medium for target detection | |
| EP4055814B1 (en) | A system for performing image motion compensation | |
| WO2017096855A1 (en) | Method and apparatus for dynamically adjusting gamma parameter | |
| CN117278867B (en) | Image sensor chip imaging system, exposure method and device thereof and storage medium | |
| TWI768282B (en) | Method and system for establishing light source information prediction model | |
| JP2018133686A (en) | Imaging apparatus, image processing device, control method therefor and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |