CN114630106A - Strobe detection method and related device - Google Patents
Strobe detection method and related device Download PDFInfo
- Publication number
- CN114630106A CN114630106A CN202210122814.7A CN202210122814A CN114630106A CN 114630106 A CN114630106 A CN 114630106A CN 202210122814 A CN202210122814 A CN 202210122814A CN 114630106 A CN114630106 A CN 114630106A
- Authority
- CN
- China
- Prior art keywords
- value
- brightness
- frequency domain
- image
- strobe
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The application discloses a strobe detection method and a related device, wherein the method comprises the following steps: acquiring an image frame sequence; wherein the image frame sequence comprises a succession of long exposure images and at least two short exposure images; obtaining a ratio component by using the brightness of the long exposure image and the brightness of the short exposure image; obtaining a motion area of each pixel point in the short-exposure image and a brightness mean value column vector positioned in a non-motion area based on the ratio component; and performing strobe detection on the image frame sequence based on the brightness mean value column vector. Through the design mode, the ratio vector can be confirmed based on different exposures so as to determine the frequency of the flicker stripe, the condition of a static stroboscopic stripe and a slow stroboscopic stripe is avoided by utilizing multi-frame images for calculation, and meanwhile, the multi-frame images remove motion interference and carry out mutual comparison and judgment on the result so as to increase the accuracy.
Description
Technical Field
The present application relates to the field of signal processing technologies, and in particular, to a strobe detection method and a related apparatus.
Background
In video monitoring services, stroboscopic phenomena often occur in indoor or outdoor artificial light source scenes, which are caused by the fact that part of the radiation energy of the artificial light source changes with the period of alternating current. The existing monitoring equipment generally uses a CMOS (Complementary Metal Oxide Semiconductor) as a sensor, and the sensor adopts a line-by-line exposure strategy, and when the exposure time is not an integral multiple of the light energy change period of the artificial light source, the size of light energy received by each line of pixels is different, so that light and dark stripes exist in a video. Meanwhile, if the interval of the initial exposure time of the two adjacent frames of images is not integral multiple of the change period of the light energy of the artificial light source, the positions of the light and dark stripes of the two adjacent frames are different, and the stripes are seen to continuously roll. For this reason, the output time of each frame of image is fixed in the monitoring apparatus to be an integral multiple of the exposure time period. For example, the output time of each frame of image under a 100hz light source is 40ms, namely 25 frames per second, so that the phase difference of the exposure time of the same line of the two adjacent frames of images is eliminated, and the stripes do not roll. The current mainstream scheme of stroboscopic detection is based on rolling stripes, most background information of an image is removed through a difference image of front and back frame stripes, and simultaneously, stroboscopic characteristics of the image are kept, so that whether a stroboscopic phenomenon exists or not is judged, but the method is not suitable for the situation that the stripes roll at a low speed and even are not fixed. Therefore, a new strobe detection method is needed to solve the above problems.
Disclosure of Invention
The present application mainly solves the technical problem of providing a strobe detection method and a related apparatus, which can increase the credibility of the detection result by comparing two frequency domain information with each other.
In order to solve the technical problem, the application adopts a technical scheme that: there is provided a strobe detection method including: acquiring an image frame sequence; wherein the image frame sequence comprises consecutive long exposure images and at least two short exposure images; obtaining a ratio component by using the brightness of the long exposure image and the brightness of the short exposure image; obtaining a motion area of each pixel point in the short-exposure image and a brightness mean value column vector positioned in a non-motion area based on the ratio component; strobe detection is performed on the sequence of image frames based on the luma mean column vector.
The number of the short-exposure images in the image frame sequence is two, and the ratio component is the ratio of the brightness of the long-exposure image to the brightness of the two short-exposure images.
Wherein, the step of obtaining the motion region of each pixel point in the short-exposure image and the brightness mean value column vector in the non-motion region based on the ratio component comprises: obtaining a first difference value between the brightness of the same pixel point in the two short-exposure images and a square value of the first difference value, and obtaining a marking value of a motion area of the pixel point based on the relation between the square value and a motion threshold value; obtaining a second difference value between 1 and the mark value, obtaining a first ratio value between the sum of products of the ratio component and the second difference value and the sum of the second difference value, and taking the first ratio value as the brightness mean value column vector; wherein the luminance mean column vector of the non-motion region is not 0.
Wherein the step of obtaining a labeling value of the motion region of the pixel point based on the relationship between the square value and the motion threshold value includes: in response to the square value being greater than the motion threshold, setting a flag value of a motion region of the pixel point to 1; and/or, in response to the square value being less than or equal to the motion threshold, setting a flag value of a motion region of the pixel point to 0.
Wherein the step of strobing the sequence of image frames based on the luma mean column vector is preceded by: obtaining brightness frequency domain characteristics corresponding to the short-exposure image based on the brightness average value column vector; and obtaining a brightness frequency domain characteristic amplitude value and a brightness frequency domain characteristic phase corresponding to the short-exposure image by using the brightness frequency domain characteristic.
Wherein the luminance frequency domain features comprise real and imaginary features; the step of obtaining the brightness frequency domain characteristic amplitude and the brightness frequency domain characteristic phase corresponding to the short-exposure image by using the brightness frequency domain characteristic includes: obtaining a first product between half of the height of the luminance mean column vector and the sum of squares of the real and imaginary features, and obtaining a second ratio between the imaginary and real features, and taking the first product as the luminance frequency-domain feature amplitude, and the arctan function value of the second ratio as the luminance frequency-domain feature phase.
The brightness frequency domain characteristic amplitude comprises a plurality of position points from a first position point to a second position point; the step of strobe detecting the sequence of image frames based on the luma mean column vector comprises: obtaining two maximum values of the brightness frequency domain characteristic amplitude values and positions of the maximum values of the brightness frequency domain characteristic amplitude values from the first position point to the second position point; in response to the two positions being consistent, or in response to the absolute value of the difference between the maximum values of the two luminance frequency domain feature amplitudes being less than or equal to a first frequency domain threshold, determining whether the maximum values of the two luminance frequency domain feature amplitudes are both less than a second frequency domain threshold; if not, judging whether the absolute value of the difference value between the brightness frequency domain characteristic phases corresponding to the two positions is larger than a third frequency domain threshold value or not; if not, obtaining the predicted stroboscopic frequency of the image frame sequence based on the height of the brightness average value column vector, and carrying out stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency.
Wherein the step of obtaining a predicted strobe frequency of the sequence of image frames based on the height of the luma mean column vector and strobing the sequence of image frames according to the predicted strobe frequency comprises: obtaining a second product of the difference value of the initial exposure time of the upper line and the lower line of the sensor and the period of the stroboscopic stripe, and taking the reciprocal of the second product as the predicted stroboscopic frequency; the stroboscopic fringe period is the number of pixels from a stroboscopic fringe starting point to the next starting point, and the stroboscopic fringe period is a third ratio of the height of the brightness average value column vector to the position point; determining that a strobe exists in the sequence of image frames in response to the predicted strobe frequency being greater than a first preset value and less than a second preset value, or in response to the predicted strobe frequency being greater than a third preset value and less than a fourth preset value; wherein the first preset value, the second preset value, the third preset value and the fourth preset value are all related to a fourth frequency domain threshold.
Wherein the step of strobe detecting the sequence of image frames based on the luma mean column vector comprises: in response to a strobe being present in the sequence of image frames, adjusting exposure times and image gain values of the sequence of image frames.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided an electronic device comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is configured to execute the program instructions to implement the strobe detection method according to any of the above embodiments.
In order to solve the above technical problem, the present application adopts another technical solution: there is provided a computer-readable storage medium storing a computer program for implementing the strobe detection method mentioned in any one of the above embodiments.
Different from the prior art, the beneficial effects of the application are that: the strobe detection method provided by the application comprises the following steps: acquiring an image frame sequence; wherein the image frame sequence comprises a succession of long exposure images and at least two short exposure images; then, obtaining a ratio component by utilizing the brightness of the long exposure image and the brightness of the short exposure image; then obtaining a motion area of each pixel point in the short-exposure image and a brightness mean value column vector positioned in a non-motion area based on the ratio component; and finally, carrying out stroboscopic detection on the image frame sequence based on the brightness average value column vector. Through the design mode, the ratio vector can be confirmed based on different exposures so as to determine the frequency of the flicker stripe, the condition of a static stroboscopic stripe and a slow stroboscopic stripe is avoided by utilizing multi-frame images for calculation, and meanwhile, the multi-frame images remove motion interference and carry out mutual comparison and judgment on the result so as to increase the accuracy.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts. Wherein:
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a strobe detection method of the present application;
FIG. 2 is a schematic illustration of a long exposure image and a short exposure image;
FIG. 3 is a schematic flow chart illustrating an embodiment of step S3 in FIG. 1;
FIG. 4 is a schematic flow chart illustrating an embodiment of the method before step S1 in FIG. 1;
FIG. 5 is a schematic flow chart illustrating an embodiment of step S4 in FIG. 1;
FIG. 6 is a schematic flow chart illustrating one embodiment of step S37 in FIG. 5;
FIG. 7 is a schematic diagram of an embodiment of a strobe detection system according to the present application;
FIG. 8 is a block diagram of an embodiment of an electronic device of the present application;
FIG. 9 is a block diagram of an embodiment of a computer-readable storage medium of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The current stroboscopic detection methods mainly include the following methods: (1) calculating the average value of the line mean vector and the difference vector of the adjacent frame images to be used as the waveform floating center of the line difference vector of the current frame image, confirming the vector which is equal to the floating center in the difference vector as a position vector, determining the distance between adjacent points in the position vector to be compared with the corresponding distance in a preset flicker frequency set, and selecting the most appropriate preset flicker frequency according to the ratio of each preset flicker frequency in the preset flicker frequency set to the N frame image sequence; (2) counting the line sum of the brightness components of two adjacent frames of images in the video frame sequence, namely calculating the flicker characteristic component and the corresponding peak function and valley function, and judging whether the video frame flicker caused by the light source exists in the current frame of image; (3) calculating a brightness column vector and an intra-frame differential brightness column vector for a single image, and acquiring the zone bit information of the single image, namely determining a stroboscopic statistic value by calculating a filtering differential brightness column vector and a carrier wave, so as to determine whether the image has stroboscopic; (4) for the situation of slow rolling stroboflash, three frames of images are collected, a differential column vector of the images is obtained for two adjacent frames of images, the obtained differential column vector is used for determining a long-sequence stroboflash characteristic value, and whether stroboflash exists is respectively calculated. However, these methods have drawbacks, for example, in the methods (1) and (2), the flicker fringe frequency is determined based on the difference between the difference vectors of adjacent lines, and the methods cannot be used for scenes in which the fringes of adjacent frames are basically unchanged and the difference vectors cannot be distinguished; in the method (3), a single frame is used for calculation, and although the situation of a static stroboscopic stripe is considered, the stroboscopic stripe exists in various ways, so that calculation by using the single frame may cause calculation errors in a complex scene, thereby causing misjudgment; the method (4) uses three frames for calculation, and although the case of slow strobe stripes is considered, the difference vector of the three-frame image still cannot obtain effective strobe feature information assuming that the stripes are still.
Referring to fig. 1 and fig. 2 together, fig. 1 is a schematic flow chart of an embodiment of a strobe detection method of the present application, and fig. 2 is a schematic diagram of a long exposure image and a short exposure image. The strobe detection method comprises the following steps:
s1: a sequence of image frames is acquired.
Specifically, the image frame sequence includes a continuous long-exposure image and at least two short-exposure images. Specifically, in the present embodiment, the number of short-exposure images in the image frame sequence is two, and the image frame sequence of the long-exposure image, the first short-exposure image, and the second short-exposure image is acquired every lapse of a fixed time interval, specifically, the image frame sequence may be obtained from the image signal processing unit, and only the luminance channel thereof (i.e., the luminance of each image) needs to be acquired. For example, in a wide dynamic mode of the monitoring device, a long-exposure image and a multi-frame short-exposure image may be continuously acquired, where the acquired long-exposure image and the acquired short-exposure image are both bayer array images, a format of the bayer array image is a raw format, and a value of any one of R, Gr, Gb, or b values of each minimum unit of the bayer array image may be taken to form a luminance image of the long-and-short exposure; for another example, in the normal linear mode of the monitoring device, the exposure time may be adjusted first to obtain a 1-frame long exposure image, and the exposure time may be adjusted again to obtain the remaining two frames of short exposure images. The image format acquired at this time is the yuv format, and only the luminance channel (y channel) thereof needs to be taken as the luminance image. At the moment, the brightness of the acquired long exposure image is high, the brightness of two frames of short exposure images is low, and moving object displacement exists between the three frames of images. At present, the artificial light source mainly has two fixed periods, namely 50hz or 60hz alternating current period, and the light intensity frequency of the corresponding artificial light source is 100hz or 120 hz. The exposure time set for the long exposure image needs to be an integral multiple of the reciprocal of the light intensity frequency, and since the alternating current frequency of different countries has already been determined, the exposure time for the long exposure image can be determined, for example, 1/100hz is 0.01 s. The exposure time of the first short exposure image setting and the exposure time of the second short exposure image setting need to be equal and less than an integer multiple of the inverse of the light intensity frequency. There is no stroboscopic phenomenon in the duration exposure image, and there is stroboscopic phenomenon in the first short exposure image and the second short exposure image, as shown in fig. 2, a is the long exposure image without stroboscopic stripes, b is the first short exposure image with stroboscopic stripes, and c is the second short exposure image with stroboscopic stripes. Of course, in other embodiments, the number of short-exposure images in the image frame sequence may be more than two, and the application is not limited herein.
S2: the ratio component is obtained using the luminance of the long-exposure image and the luminance of the short-exposure image.
Specifically, the ratio component is obtained by using the luminance of the long-exposure image and the luminance of the short-exposure image, and since the number of the short-exposure images is two, two ratio components are correspondingly obtained. Specifically, in the present embodiment, the first ratio component D1 is obtained using the luminance L1 of the long-exposure image and the luminance S1 of the first short-exposure image, and the second ratio component D2 is obtained using the luminance L1 of the long-exposure image and the luminance S2 of the second short-exposure image. The ratio component is the ratio of the brightness of the long exposure image to the brightness of the two short exposure images, and the specific calculation formula is as follows:
D1＝L1/(S1+eps)
D2＝L1/(S2+eps)
where eps is a small value, which serves to prevent the divisor from being 0 and has no actual physical meaning. Because the first long exposure image has no stroboscopic stripes, the two short exposure images have stroboscopic stripes, and meanwhile, the three short exposure images have slightly different interframe motions, the two ratio images mainly contain stripe information and abnormal pixels caused by some motions. The ratio vector is confirmed based on different exposures, so that the flicker fringe frequency is determined, and the method can be used for scenes in which the fringes of adjacent frames of images are basically unchanged and the difference vector cannot be distinguished.
S3: and obtaining the motion area of each pixel point in the short-exposure image and the brightness mean value column vector positioned in the non-motion area based on the ratio component.
Specifically, in the present embodiment, the number of short-exposure images in the image frame sequence is two. Referring to fig. 3, fig. 3 is a schematic flowchart illustrating an embodiment of step S3 in fig. 1. Step S3 includes:
s10: and obtaining a first difference value between the brightness of the same pixel point in the two short-exposure images and a square value of the first difference value, and obtaining a marking value of a motion area of the pixel point based on the relation between the square value and the motion threshold value.
Specifically, the short-exposure image includes a plurality of pixels, a first difference between the brightness of the pixel (i, j) in the first short-exposure image being S1(i, j) and the brightness of the pixel S2(i, j) in the second short-exposure image being S1(i, j) -S2(i, j), and a square value thereof being (S1(i, j) -S2(i, j)) 2. Specifically, in this embodiment, the step of obtaining the label value of the motion region of the pixel point based on the relationship between the square value and the motion threshold value in step S10 includes using the result of the motion threshold value move _ th1 on the square valuePerforming image binarization operation, specifically, when the square value is (S1(i, j) -S2(i, j))2When the motion threshold value move _ th1 is larger than the motion threshold value move _ th1, it is indicated that motion exists at the position of the pixel point (i, j), and the mark value move (i, j) of the motion area of the pixel point is set to 1; when the square value is less than or equal to the motion threshold value move _ th1When the motion does not exist at the position of the pixel point (i, j), the mark value move (i, j) of the motion area of the pixel point is set to 0, which can be specifically expressed as:
move(i,j)＝1if(S1(i,j)-S2(i,j))2>move_th1
move(i,j)＝0if(S1(i,j)-S2(i,j))2≤move_th1
the move _ th1 is a motion threshold, which can be set manually according to actual conditions, and the move (i, j) is a mark value of a motion area of the pixel point (i, j). By the method, whether the position of one pixel point in the short-exposure image has motion or not can be obtained, and if the position has motion, the motion needs to be removed to prevent interference with the next operation.
S11: and obtaining a second difference value between 1 and the mark value, obtaining a first ratio value between the sum of the products of the ratio component and the second difference value and the sum of the second difference value, and taking the first ratio value as a luminance mean value column vector.
Specifically, the luminance average column vector of the non-motion area is not 0. Since there are two short-exposure images, the corresponding luminance mean vectors include a first luminance mean vector D1_ line (i) and a second luminance mean vector D2_ line (i), and the specific calculation formula is:
wherein D1(i, j) is the first ratio component of the pixel (i, j) in the first short-exposure image, D2(i, j) is the second ratio component of the pixel (i, j) in the second short-exposure image, D1_ line (i)And D2_ line (i) respectively represent the first luminance average column vector and the second luminance average column vector. In combination with the above, taking the first short-exposure image as an example, when there is motion at the position of the pixel point (i, j), the mark value move (i, j) of the motion region of the pixel point)D1_ line (i) is equal to 0, that is, the position of the pixel point in the first short-exposure image is removed, so as to prevent interference with the next operation; still using the first short exposure image, when there is no motion at the position of the pixel point (i, j), that is, the position of the pixel point is a non-motion region, the mark value move (i, j) of the motion region of the pixel point)If 0, D1_ line (i) ≠ 0. In the present embodiment, the luminance average column vector referred to herein refers to a luminance average column vector of a non-motion region. In addition, the second short-exposure image is similar to the first short-exposure image, and is not described herein again. And calculating by using multiple frames, taking the conditions of the static stroboscopic stripes and the slow stroboscopic stripes into consideration, removing motion interference by using the multiple frames, and comparing and judging the results to increase the accuracy.
S4: and performing strobe detection on the image frame sequence based on the brightness mean value column vector.
Specifically, in the present embodiment, please refer to fig. 4, where fig. 4 is a schematic flowchart of an embodiment before step S1 in fig. 1. Specifically, step S4 includes:
s20: and obtaining the brightness frequency domain characteristics corresponding to the short-exposure image based on the brightness average value column vector.
Specifically, fourier frequency domain features of the first luminance average column vector D1_ line (i) and the second luminance average column vector D2_ line (i) are calculated, and a specific calculation formula is as follows:
D1_f＝fft(D1_line(i))
D2_f＝fft(D2_line(i))
wherein fft is a fast fourier transform algorithm, which is a known algorithm and is not described herein again. D1_ f is the first luminance frequency domain feature, and D2_ f is the second luminance frequency domain feature.
S21: and obtaining the brightness frequency domain characteristic amplitude and the brightness frequency domain characteristic phase corresponding to the short-exposure image by utilizing the brightness frequency domain characteristic.
Specifically, the luminance frequency domain characteristic includes a real part characteristic and an imaginary part characteristic. In this embodiment, step S21 specifically includes: and obtaining a first product between half of the height of the brightness average value column vector and the square sum of the real part characteristic and the imaginary part characteristic, obtaining a second ratio between the imaginary part characteristic and the real part characteristic, taking the first product as the brightness frequency domain characteristic amplitude value, and taking the arc tangent function value of the second ratio as the brightness frequency domain characteristic phase. The specific calculation formula is as follows:
D1_f_abs(i)＝2/height*(D1_f_re(i)2+D1_f_im(i)2)
D2_f_abs(i)＝2/height*(D2_f_re(i)2+D2_f_im(i)2)
D1_f_angle(i)＝arctan(D1_f_im(i)/D1_f_re(i))
D2_f_angle(i)＝arctan(D2_f_im(i)/D2_f_re(i))
wherein D1_ f _ abs is a first luminance frequency domain feature amplitude, D2_ f _ abs is a second luminance frequency domain feature amplitude, D1_ f _ angle is a first luminance frequency domain feature phase, D2_ f _ angle is a second luminance frequency domain feature phase, height is a height of a column vector, D1_ f _ re and D1_ f _ im are respectively a real part feature and an imaginary part feature of the first luminance frequency domain feature D1_ f, D2_ f _ re and D2_ f _ im are respectively a real part feature and an imaginary part feature of the second luminance frequency domain feature D2_ f, and arctan is an arctangent function.
Specifically, in this embodiment, the luminance frequency domain feature amplitude includes a plurality of position points from a first position point to a second position point, where the first position point is the 2 nd point in the luminance frequency domain feature amplitude, and the second position point is the height/2 nd point in the luminance frequency domain feature amplitude. Referring to fig. 5, fig. 5 is a schematic flowchart illustrating an implementation manner of step S4 in fig. 1. Specifically, step S4 includes:
s30: and obtaining the maximum value of the two brightness frequency domain characteristic amplitudes from the first position point to the second position point and the position of the maximum value of the brightness frequency domain characteristic amplitude.
Specifically, the maximum value D1_ max of the first luminance frequency domain feature amplitude and the position P1 thereof are found from the 2 nd point to the height/2 nd point in the first luminance frequency domain feature amplitude, and the maximum value D2_ max of the second luminance frequency domain feature amplitude and the position P2 thereof are found from the 2 nd point to the height/2 nd point in the second luminance frequency domain feature amplitude. In the brightness frequency domain feature amplitude, the first point is a frequency domain direct current component, and the height/2+1 point to the height point are symmetrical frequency domain features which do not participate in subsequent calculation.
S31: and judging whether the two positions are consistent or not, or judging whether the absolute value of the difference value of the maximum values of the characteristic amplitudes of the two brightness frequency domains is smaller than or equal to a first frequency domain threshold value or not.
Specifically, it is determined whether the two positions P1 and P2 coincide, or whether the absolute value of D1_ max-D2_ max is less than or equal to the first frequency domain threshold f _ th 1.
S32: and if so, judging whether the maximum values of the two brightness frequency domain characteristic amplitudes are both smaller than a second frequency domain threshold value.
Specifically, if it is determined that the two positions P1 and P2 are identical, or the absolute value of D1_ max-D2_ max is less than or equal to the first frequency domain threshold value f _ th1, it is determined that there is no difference or a small difference between the results of the first luminance average value column vector D1_ line (i) and the second luminance average value column vector D2_ line (i), and at this time, it is determined whether the maximum value D1_ max of the first luminance frequency domain feature amplitude and the maximum value D2_ max of the second luminance frequency domain feature amplitude are both less than the second frequency domain threshold value f _ th 2. The first frequency domain threshold f _ th1 and the second frequency domain threshold f _ th2 may be set manually according to actual situations, and are not limited in this application.
S33: otherwise, it returns to step S1.
Specifically, if it is determined that the two positions P1 and P2 are not consistent, or the absolute value of D1_ max-D2_ max is greater than the first frequency domain threshold value f _ th1, it is determined that the results of the first luminance mean value column vector D1_ line (i) and the second luminance mean value column vector D2_ line (i) are greatly different, and it may be necessary to return to step S1 and re-measure the results due to a moving object or other external changes.
S34: if yes, the stroboscopic phenomenon is judged to be absent in the image frame sequence.
Specifically, if the maximum value D1_ max of the first luminance frequency domain feature amplitude and the maximum value D2_ max of the second luminance frequency domain feature amplitude are both smaller than the second frequency domain threshold value f _ th2, it is determined that the stroboscopic phenomenon does not exist in the scene, and the process is ended.
S35: if not, judging whether the absolute value of the difference value between the brightness frequency domain characteristic phases corresponding to the two positions is larger than a third frequency domain threshold value.
Specifically, if at least one of the maximum value D1_ max of the first luminance frequency domain feature amplitude and the maximum value D2_ max of the second luminance frequency domain feature amplitude is smaller than the second frequency domain threshold value f _ th2, the first luminance frequency domain feature phase D1_ f _ angle (P1) and the second luminance frequency domain feature phase D2_ f _ angle (P2) corresponding to the two positions P1 and P2 are searched, and whether the absolute value of the difference between the first luminance frequency domain feature phase D1_ f _ angle (P1) and the second luminance frequency domain feature phase D2_ f _ angle (P2) is larger than the third frequency domain threshold value f _ th3 is determined. The third frequency domain threshold value f _ th3 may be set manually according to actual conditions, and is not limited in this application.
S36: if yes, the process returns to step S1.
Specifically, if the absolute value of the difference between the two is greater than the third frequency-domain threshold value f _ th3, it indicates that the results of the first luminance average column vector D1_ line (i) and the second luminance average column vector D2_ line (i) are different greatly, and may be caused by a moving object or other external changes, and then it needs to return to step 1 and re-measure.
S37: if not, obtaining the predicted stroboscopic frequency of the image frame sequence based on the height of the brightness average value column vector, and carrying out stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency.
Specifically, if the absolute value of the difference between the first luminance mean column vector D1_ line (i) and the second luminance mean column vector D2_ line (i) is less than or equal to the third frequency domain threshold f _ th3, the result of the first luminance mean column vector D1_ line (i) and the result of the second luminance mean column vector D2_ line (i) have no difference or a small difference, the predicted strobe frequency of the image frame sequence is obtained based on the height of the luminance mean column vector, and the image frame sequence is strobed and detected according to the predicted strobe frequency. Referring to fig. 6, fig. 6 is a schematic flowchart illustrating an implementation manner of step S37 in fig. 5. Specifically, step S37 includes:
s370: and obtaining a second product of the difference value of the initial exposure time of the upper row and the lower row of the sensor and the period of the stroboscopic stripe, and taking the reciprocal of the second product as the predicted stroboscopic frequency.
Specifically, the strobe cycle is the number of pixels from the strobe start point to the next start point, and the strobe cycle is a third ratio of the height of the luminance average column vector to the position point. Specifically, in this embodiment, the calculation formula of the predicted strobe frequency is:
predict_f＝1/(line_time*height/P1)
the line _ time is a difference between two upper and lower lines of initial exposure time of the CMOS sensor, and the parameter is a preset parameter, which is not limited herein. height/P1 is the number of pixels from the strobe start point to the next start point, i.e., the strobe period. Of course, the sensor may be other sensors, and the present application is not limited thereto.
S371: and judging whether the predicted stroboscopic frequency is greater than a first preset value and less than a second preset value or whether the predicted stroboscopic frequency is greater than a third preset value and less than a fourth preset value.
Specifically, the first preset value, the second preset value, the third preset value and the fourth preset value are all related to the fourth frequency domain threshold value f _ th 4. The fourth frequency domain threshold value f _ th4 may be set manually according to actual conditions, and is not limited in this application.
S372: if yes, determining that stroboscopic exists in the image frame sequence.
S373: otherwise, it is determined that no strobes are present in the sequence of image frames.
In this embodiment, when the strobe frequency bodict _ f is predicted to approach the light intensity frequency of the artificial light source generated by two main ac currents, i.e. 100hz or 120hz, the following results are obtained:
100hz-f_th4<predict_f<100hz+f_th4
or 120hz-f_th4<predict_f<120hz+f_th4
the first preset value is 100hz-f _ th4, the second preset value is 100hz + f _ th4, the third preset value is 120hz-f _ th4, and the fourth preset value is 120hz + f _ th 4. When the predicted strobe frequency bodict _ f is close to 100hz, it is considered that there is a 100hz strobe in the current scene. When the predicted strobe frequency bodict _ f is close to 120hz, it is considered that 120hz strobe exists in the current scene. When the predicted strobe frequency bodict _ f is neither close to 100hz nor close to 120hz, it is considered that there is no strobe phenomenon in the current scene.
Specifically, in the present embodiment, step S4 is followed by: in response to the presence of a strobe in the sequence of image frames, the exposure time and image gain values for the sequence of image frames are adjusted. Specifically, when it is considered that there is a 100hz strobe, the luminance is adjusted to an integral multiple of the reciprocal of the 100hz strobe, for example, 10ms or the like; when the 120hz strobe is considered to exist, the brightness is adjusted to be an integral multiple of the reciprocal of the 120hz strobe, for example, 8.33 ms. In this case, the image brightness is increased by increasing the exposure time, and the gain value needs to be decreased to keep the image brightness substantially uniform. The formula is as follows:
gain_result＝gain*shutter/shutter_result
wherein, gain is the gain value of the image before the adjustment exposure, shutter is the exposure time value before the adjustment exposure, shutter _ result is the exposure time value after the adjustment exposure, and gain _ result is the new gain value after the adjustment exposure. The image brightness can be improved by adjusting the exposure time and the image gain value, and the probability of stroboflash in the image is reduced.
Through the design mode, the ratio vector can be confirmed based on different exposures so as to determine the frequency of the flicker stripe, the condition of a static stroboscopic stripe and a slow stroboscopic stripe is avoided by utilizing multi-frame images for calculation, and meanwhile, the multi-frame images remove motion interference and carry out mutual comparison and judgment on the result so as to increase the accuracy.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a strobe detection system according to the present application. This stroboscopic detection system specifically includes:
an obtaining module 10, configured to obtain a sequence of image frames; wherein the sequence of image frames comprises a succession of long exposure images and at least two short exposure images.
And a ratio component module 12, coupled to the obtaining module 10, for obtaining a ratio component by using the brightness of the long-exposure image and the brightness of the short-exposure image.
And a column vector module 14, coupled to the ratio component module 12, configured to obtain a motion region of each pixel in the short-exposure image and a luminance mean column vector located in a non-motion region based on the ratio component.
A detection module 16, coupled to the column vector module 14, is configured to perform strobe detection on the image frame sequence based on the luminance mean column vector.
Referring to fig. 8, fig. 8 is a schematic frame diagram of an embodiment of an electronic device according to the present application. The electronic device comprises a memory 20 and a processor 22 coupled to each other. Specifically, in the present embodiment, the memory 20 stores program instructions, and the processor 22 is configured to execute the program instructions to implement the strobe detection method in any of the above-mentioned embodiments.
Specifically, the processor 22 may also be referred to as a CPU (Central Processing Unit). The processor 22 may be an integrated circuit chip having signal processing capabilities. The Processor 22 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, processor 22 may be implemented collectively by a plurality of integrated circuit chips.
Referring to fig. 9, fig. 9 is a block diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure. The computer-readable storage medium 30 stores a computer program 300, which can be read by a computer, and the computer program 300 can be executed by a processor to implement the strobe detection method mentioned in any of the above embodiments. The computer program 300 may be stored in the computer-readable storage medium 30 in the form of a software product, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. The computer-readable storage medium 30 having a storage function may be various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or may be a terminal device, such as a computer, a server, a mobile phone, or a tablet.
In summary, unlike the prior art, the strobe detection method provided by the present application includes: acquiring an image frame sequence; wherein the image frame sequence comprises a succession of long exposure images and at least two short exposure images; then, obtaining a ratio component by utilizing the brightness of the long exposure image and the brightness of the short exposure image; then obtaining a motion area of each pixel point in the short-exposure image and a brightness mean value column vector positioned in a non-motion area based on the ratio component; and finally, carrying out stroboscopic detection on the image frame sequence based on the brightness average value column vector. Through the design mode, the ratio vector can be confirmed based on different exposures so as to determine the frequency of the flicker stripes, the condition of static flicker stripes and slow flicker stripes is avoided by utilizing multi-frame images for calculation, and meanwhile, the multi-frame images remove motion interference and carry out mutual comparison and judgment on the results so as to increase the accuracy.
The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application or are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (11)
1. A strobe detection method, comprising:
acquiring an image frame sequence; wherein the image frame sequence comprises consecutive long exposure images and at least two short exposure images;
obtaining a ratio component by using the brightness of the long exposure image and the brightness of the short exposure image;
obtaining a motion area of each pixel point in the short-exposure image and a brightness mean value column vector positioned in a non-motion area based on the ratio component;
strobe detection is performed on the sequence of image frames based on the luminance mean column vector.
2. The strobe detection method of claim 1, wherein the number of short-exposure images in the image frame sequence is two, and the ratio component is a ratio of the brightness of the long-exposure image to the brightness of the two short-exposure images.
3. The strobe detection method as claimed in claim 1 or 2, wherein said step of obtaining a motion region and a luminance mean column vector in a non-motion region of each pixel point in the short-exposure image based on the ratio component comprises:
obtaining a first difference value between the brightness of the same pixel point in the two short-exposure images and a square value of the first difference value, and obtaining a marking value of a motion area of the pixel point based on the relation between the square value and a motion threshold value;
obtaining a second difference value between 1 and the mark value, obtaining a first ratio value between the sum of products of the ratio component and the second difference value and the sum of the second difference value, and taking the first ratio value as the brightness mean value column vector; wherein the luminance mean column vector of the non-motion region is not 0.
4. The strobe detection method as claimed in claim 3, wherein said step of obtaining a mark value of a motion region of said pixel point based on a relation between said squared value and a motion threshold value comprises:
in response to the square value being greater than the motion threshold, setting a flag value of a motion region of the pixel point to 1; and/or the presence of a gas in the gas,
setting a flag value of a motion region of the pixel point to 0 in response to the squared value being less than or equal to the motion threshold.
5. The strobe detection method of claim 1, wherein the step of strobe detecting the sequence of image frames based on the luminance mean column vector is preceded by:
obtaining brightness frequency domain characteristics corresponding to the short-exposure image based on the brightness average value column vector;
and obtaining the brightness frequency domain characteristic amplitude and the brightness frequency domain characteristic phase corresponding to the short-exposure image by using the brightness frequency domain characteristic.
6. The strobe detection method as claimed in claim 5, wherein said luminance frequency domain features comprise real and imaginary features; the step of obtaining the brightness frequency domain characteristic amplitude and the brightness frequency domain characteristic phase corresponding to the short-exposure image by using the brightness frequency domain characteristic includes:
obtaining a first product between half of the height of the luminance mean column vector and the sum of squares of the real and imaginary features, and obtaining a second ratio between the imaginary and real features, and taking the first product as the luminance frequency-domain feature amplitude, and the arctan function value of the second ratio as the luminance frequency-domain feature phase.
7. The strobe detection method of claim 6, wherein the luminance frequency domain feature amplitude comprises a plurality of location points from a first location point to a second location point; the step of strobe detecting the sequence of image frames based on the luma mean column vector comprises:
obtaining two maximum values of the brightness frequency domain characteristic amplitude values and positions of the maximum values of the brightness frequency domain characteristic amplitude values from the first position point to the second position point;
in response to the two positions being consistent, or in response to the absolute value of the difference between the maximum values of the two luminance frequency domain feature amplitudes being less than or equal to a first frequency domain threshold, determining whether the maximum values of the two luminance frequency domain feature amplitudes are both less than a second frequency domain threshold;
if not, judging whether the absolute value of the difference value between the brightness frequency domain characteristic phases corresponding to the two positions is larger than a third frequency domain threshold value or not;
if not, obtaining the predicted stroboscopic frequency of the image frame sequence based on the height of the brightness mean value column vector, and carrying out stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency.
8. The strobe detection method of claim 7, wherein the step of obtaining a predicted strobe frequency of the sequence of image frames based on the height of the luminance mean column vector and strobing the sequence of image frames according to the predicted strobe frequency comprises:
obtaining a second product of the difference value of the initial exposure time of the upper line and the lower line of the sensor and the period of the stroboscopic stripe, and taking the reciprocal of the second product as the predicted stroboscopic frequency; the stroboscopic fringe period is the number of pixels from a stroboscopic fringe starting point to the next starting point, and the stroboscopic fringe period is a third ratio of the height of the brightness average value column vector to the position point;
determining that a strobe exists in the sequence of image frames in response to the predicted strobe frequency being greater than a first preset value and less than a second preset value, or in response to the predicted strobe frequency being greater than a third preset value and less than a fourth preset value; wherein the first preset value, the second preset value, the third preset value and the fourth preset value are all related to a fourth frequency domain threshold.
9. The strobe detection method of claim 1, wherein the step of strobe-detecting the sequence of image frames based on the luminance mean column vector is followed by:
in response to a presence of a strobe in the sequence of image frames, adjusting exposure times and image gain values of the sequence of image frames.
10. An electronic device comprising a memory and a processor coupled to each other, the memory having stored therein program instructions, the processor being configured to execute the program instructions to implement the strobe detection method of any one of claims 1 to 9.
11. A computer-readable storage medium, characterized in that a computer program is stored for implementing the strobe detection method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210122814.7A CN114630106A (en) | 2022-02-09 | 2022-02-09 | Strobe detection method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210122814.7A CN114630106A (en) | 2022-02-09 | 2022-02-09 | Strobe detection method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114630106A true CN114630106A (en) | 2022-06-14 |
Family
ID=81898665
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210122814.7A Pending CN114630106A (en) | 2022-02-09 | 2022-02-09 | Strobe detection method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114630106A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115514901A (en) * | 2022-09-09 | 2022-12-23 | 维沃移动通信有限公司 | Exposure time adjusting method and circuit thereof |
-
2022
- 2022-02-09 CN CN202210122814.7A patent/CN114630106A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115514901A (en) * | 2022-09-09 | 2022-12-23 | 维沃移动通信有限公司 | Exposure time adjusting method and circuit thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101659058B1 (en) | Flicker detection circuit for imaging sensors that employ rolling shutters | |
| US8040393B2 (en) | Image-pickup signal processor and method of detecting flicker | |
| WO2019148912A1 (en) | Image processing method, apparatus, electronic device and storage medium | |
| WO2014114098A1 (en) | Method and device for terminal side time domain video quality evaluation | |
| US8494267B2 (en) | Image processing device, image processing method, and program for normalizing a histogram | |
| US9569688B2 (en) | Apparatus and method of detecting motion mask | |
| JP2003198932A (en) | Flicker correction device, flicker correction method, and recording medium with flicker correction program recorded | |
| CN110766621A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US20120081569A1 (en) | Detection of flicker within captured imaged frames | |
| US11678064B2 (en) | Method for detecting and reducing video flicker and image processing device | |
| US8768160B2 (en) | Flicker band automated detection system and method | |
| CN114630106A (en) | Strobe detection method and related device | |
| CN111741290B (en) | Image stroboscopic detection method and device, storage medium and terminal | |
| CN101690170A (en) | Imaging apparatus and method of improving sensitivity of the same | |
| JP2003134385A (en) | Image synthesizing device | |
| JP2022548668A (en) | Reduce flickering of multiple light sources in images | |
| US8189068B2 (en) | Apparatus and method for detecting flicker noise and computer readable medium stored thereon computer executable instructions for performing the method | |
| JP2007201933A (en) | Monitoring device using image | |
| KR20210049682A (en) | Method and system for maintaining peak intensity frames as preview frames in slow motion mode of electronic device | |
| CN113163194B (en) | Detection method and device for flicker of shooting scene, shooting equipment and medium | |
| KR100949137B1 (en) | Apparatus for video interpolation, method thereof and computer recordable medium storing the method | |
| CN110933288B (en) | Image generation method and electronic device | |
| WO2023001373A1 (en) | Device and method for processing image data | |
| KR20120045812A (en) | Method for detecting flicker using motion compensation and apparatus for performing the same | |
| US20120045148A1 (en) | Image processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |