CN116916116A - Frame rate adjustment method, storage medium and electronic device - Google Patents
Frame rate adjustment method, storage medium and electronic device Download PDFInfo
- Publication number
- CN116916116A CN116916116A CN202310660902.7A CN202310660902A CN116916116A CN 116916116 A CN116916116 A CN 116916116A CN 202310660902 A CN202310660902 A CN 202310660902A CN 116916116 A CN116916116 A CN 116916116A
- Authority
- CN
- China
- Prior art keywords
- video
- video frame
- frame
- image block
- reflection intensity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000001502 supplementing effect Effects 0.000 claims abstract description 44
- 238000012545 processing Methods 0.000 claims description 12
- 230000000630 rising effect Effects 0.000 claims description 12
- 230000003827 upregulation Effects 0.000 claims description 4
- 230000000153 supplemental effect Effects 0.000 claims description 2
- 238000007619 statistical method Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 9
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000017525 heat dissipation Effects 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The application discloses a frame rate adjusting method and electronic equipment, wherein the frame rate adjusting method is used for acquiring a video frame sequence acquired by a video acquisition device in an infrared light supplementing state, wherein the video frame sequence comprises at least two adjacent video frames, calculating the infrared light reflecting intensity corresponding to each video frame in the video frame sequence, calculating the intensity fluctuation corresponding to the video frame sequence based on the infrared light reflecting intensity corresponding to each video frame, and adjusting the video acquisition frame rate of the video acquisition device in the infrared light supplementing state based on the intensity fluctuation, so that the video frame in a video stream is not required to be supported by complex hardware and chip performance, and the intensity fluctuation of the infrared light reflecting intensity of an acquisition picture is judged only through statistical analysis, so that the frame rate self-adaptive adjustment is performed, and the quality of video acquisition is improved.
Description
Technical Field
The present application relates to the field of image video processing technologies, and in particular, to a frame rate adjustment method, a storage medium, and an electronic device.
Background
Under the low illumination environment, the camera can be automatically switched to a night mode, namely, the optical filter can be switched to a night piece transmitting infrared light, and meanwhile, the infrared lamp is started to supplement light, so that a shooting scene is bright, and a bright picture is obtained.
However, due to the limitations of the cost of the camera, the heat dissipation of related devices, the service life and the like, the power of the infrared lamp cannot be maximized, and the light supplementing effect is not ideal, so that the improvement of the low-light image effect of the infrared camera becomes a research point of the industry.
Disclosure of Invention
The application provides at least a frame rate adjustment method, a storage medium and an electronic device.
The first aspect of the present application provides a frame rate adjustment method, including: acquiring a video frame sequence acquired by a video acquisition device in an infrared light supplementing state, wherein the video frame sequence comprises at least two adjacent video frames; calculating the infrared reflection intensity corresponding to each video frame in the video frame sequence; calculating intensity fluctuation corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame; based on the intensity fluctuation, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted.
In one embodiment, calculating the infrared reflection intensity corresponding to each video frame in the sequence of video frames includes: dividing the video frame to obtain a plurality of image blocks; calculating a characteristic value corresponding to each image block based on pixel information of each image block; and obtaining the infrared reflection intensity corresponding to the video frame based on the characteristic value corresponding to each image block.
In an embodiment, calculating a feature value corresponding to each image block based on pixel information of each image block includes: extracting a plurality of channel pixel values of pixel points in an image block; based on the plurality of channel pixel values, a corresponding feature value for each image block is calculated.
In an embodiment, obtaining the infrared reflection intensity corresponding to the video frame based on the feature value corresponding to each image block includes: acquiring a preset reference characteristic value; calculating the similarity between the characteristic value corresponding to each image block and a preset reference characteristic value to obtain the similarity of the image blocks corresponding to each image block; and obtaining the infrared reflection intensity corresponding to the video frame based on the similarity of the image blocks corresponding to each image block.
In an embodiment, obtaining the infrared reflection intensity corresponding to the video frame based on the similarity of the image blocks corresponding to each image block includes: acquiring a preset similarity threshold; comparing the similarity of the image blocks corresponding to each image block with a preset similarity threshold value, and classifying each image block based on a comparison result; and calculating the infrared reflection intensity corresponding to the video frame based on the number of the image blocks of each category in the classification result.
In one embodiment, calculating intensity fluctuations corresponding to a sequence of video frames based on the infrared reflection intensity corresponding to each video frame includes: calculating the variation of the infrared reflection intensity between adjacent video frames in the video frame sequence; and counting the variation quantity between each pair of adjacent video frames in the video frame sequence to obtain the intensity fluctuation corresponding to the video frame sequence.
In an embodiment, adjusting a video capture frame rate of a video capture device in an infrared light supplementing state based on intensity fluctuations includes: if the intensity fluctuation is smaller than a preset frame dropping fluctuation threshold value, performing downward adjustment processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state; if the intensity fluctuation is larger than a preset frame rising fluctuation threshold, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted upwards.
In an embodiment, if the intensity fluctuation is greater than a preset frame rising fluctuation threshold, performing up-regulation processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state, including: acquiring a preset reflection intensity threshold value; and obtaining a target reflection intensity corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame in the video frame sequence; if the target reflection intensity is larger than a preset reflection intensity threshold value and the intensity fluctuation is larger than a preset frame rising fluctuation threshold value, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted upwards.
A second aspect of the present application provides a frame rate adjustment apparatus, comprising: the acquisition module is used for acquiring a video frame sequence acquired by the video acquisition device in an infrared light supplementing state, wherein the video frame sequence comprises at least two adjacent video frames; the reflection intensity calculation module is used for calculating the infrared reflection intensity corresponding to each video frame in the video frame sequence; the intensity fluctuation calculation module is used for calculating intensity fluctuation corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame; and the frame rate adjusting module is used for adjusting the video acquisition frame rate of the video acquisition device in the infrared light supplementing state based on the intensity fluctuation.
A third aspect of the present application provides an electronic device, including a memory and a processor for executing program instructions stored in the memory to implement the frame rate adjustment method described above.
A fourth aspect of the present application provides a computer readable storage medium having stored thereon program instructions which, when executed by a processor, implement the above-described frame rate adjustment method.
According to the scheme, the video frame sequence acquired by the video acquisition device in the infrared light supplementing state is acquired, the video frame sequence contains at least two adjacent video frames, the infrared light reflecting intensity corresponding to each video frame in the video frame sequence is calculated, the intensity fluctuation corresponding to the video frame sequence is calculated based on the infrared light reflecting intensity corresponding to each video frame, and the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted based on the intensity fluctuation, so that complex hardware and chip performance support are not needed, the video frames in a video stream are only analyzed through statistics, the intensity fluctuation of the infrared light reflecting intensity of an acquired picture is judged in the infrared light supplementing environment, and the frame rate self-adaptive adjustment is performed, so that the quality of video acquisition is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 is a flow chart of an exemplary embodiment of a frame rate adjustment method of the present application;
fig. 2 is a schematic view of a scene of a frame rate adjustment method according to an exemplary embodiment of the present application;
FIG. 3 is a schematic diagram of video frame segmentation shown in accordance with an exemplary embodiment of the present application;
FIG. 4 is a schematic diagram illustrating the calculation of feature value similarity according to an exemplary embodiment of the present application;
fig. 5 is a flowchart illustrating a frame rate adjustment method according to another exemplary embodiment of the present application;
fig. 6 is a block diagram of a frame rate adjustment apparatus according to an exemplary embodiment of the present application;
FIG. 7 is a schematic diagram of an electronic device shown in an exemplary embodiment of the application;
fig. 8 is a schematic diagram of a structure of a computer-readable storage medium according to an exemplary embodiment of the present application.
Detailed Description
The following describes embodiments of the present application in detail with reference to the drawings.
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship. Further, "a plurality" herein means two or more than two. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
As shown in fig. 1, in an exemplary embodiment, the frame rate adjustment method at least includes steps S110 to S140, which are described in detail below:
step S110: and acquiring a video frame sequence acquired by the video acquisition device in an infrared light supplementing state, wherein the video frame sequence comprises at least two adjacent video frames.
The video acquisition device has a video acquisition function, and a video frame sequence acquired by the video acquisition device is acquired in an infrared light supplementing state.
For example, an infrared lamp provided on the video capturing device may generate infrared rays by using a Light-Emitting Diode (LED) lamp or other Light source, and the video capturing device may enter an infrared Light supplementing state by turning on the infrared lamp provided on the video capturing device, so as to obtain a video frame sequence captured by the video capturing device in the infrared Light supplementing state.
In some embodiments, the adjacent video frames may be video frames adjacent to each other with a generation timestamp corresponding to the video frames, for example, a video stream of the video acquisition device includes video frame 1 acquired at time T and video frame 2 acquired at time t+1, where time T is adjacent to time t+1, and then video frame 1 is adjacent to video frame 2.
In some embodiments, the adjacent video frames may also be video frames whose generation time interval corresponding to the video frames meets a preset requirement, for example, the video stream of the video acquisition device contains a video frame 1 acquired at a time T, the video frame 1 is taken as a reference video frame, a target time period is obtained based on the generation time corresponding to the reference video frame, for example, the target time period is obtained to be t+1 to t+5, then the video frames in the video stream within the target time period are taken as candidate video frames, and the video frames meeting the preset condition in the candidate video frames are selected as the adjacent video frames of the reference video frame. The adjacent video frames may be video frames whose video frame quality satisfies a preset condition.
Step S120: and calculating the infrared reflection intensity corresponding to each video frame in the video frame sequence.
The infrared reflection intensity refers to the degree of infrared reflection in a collection picture corresponding to the video collection device.
The application reflects the distance between the acquisition object and the video acquisition device by the infrared reflection intensity. The collection object may be a pedestrian, an animal, a vehicle, a building, or the like, which is not limited in the present application.
Step S130: and calculating intensity fluctuation corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame.
Because the distance between the acquisition object and the video acquisition device can be reflected by the infrared reflection intensity, the motion state of the acquisition object and the video acquisition device can be reflected by the intensity fluctuation between each video frame in the video frame sequence obtained through calculation.
Step S140: based on the intensity fluctuation, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted.
The frame rate refers to the number of frames taken per second by the video capture device.
And judging whether a moving acquisition object exists in an acquisition scene corresponding to the video acquisition device or not through the acquired intensity fluctuation, so as to adjust the video acquisition frame rate of the video acquisition device in the infrared light supplementing state according to different judgment results.
In some embodiments, referring to fig. 2, fig. 2 is a schematic view of a frame rate adjustment method according to an exemplary embodiment of the present application, as shown in fig. 2, a video capturing device turns on an infrared lamp in a low-illumination environment, enters an infrared light supplementing state, and performs video capturing in real time in the infrared light supplementing state to obtain a video stream. And acquiring a video frame sequence corresponding to the current moment from the video stream, if the video frame sequence acquisition time period corresponding to the current moment is determined, extracting video frames in the video frame sequence acquisition time period in the video stream, and obtaining the video frame sequence. And carrying out infrared reflection intensity calculation on each video frame in the video frame sequence to obtain the infrared reflection intensity corresponding to each video frame, further obtaining the intensity fluctuation of the infrared reflection intensity between each video frame in the video frame sequence, and further adjusting the video acquisition frame rate of the video acquisition device in the infrared light supplementing state according to the intensity fluctuation.
Next, a process of calculating the infrared reflection intensity corresponding to the video frame will be described in detail:
in some embodiments, calculating an infrared reflection intensity for each video frame in a sequence of video frames includes: dividing the video frame to obtain a plurality of image blocks; calculating a characteristic value corresponding to each image block based on pixel information of each image block; and obtaining the infrared reflection intensity corresponding to the video frame based on the characteristic value corresponding to each image block.
The image block segmentation size corresponding to the segmentation of the video frame can be preset or flexibly calculated, for example, the image block segmentation size corresponding to the segmentation of the video frame is determined according to the resolution of the video acquisition device, the type of the acquisition scene corresponding to the video acquisition device and the like, so that the accuracy of segmenting the video frame is improved.
In other embodiments, the dividing area in the video frame may be determined first, and image blocks of the image content in the dividing area may be divided to obtain a plurality of image blocks, so as to calculate the important image content in the video frame, and improve the accuracy of calculating the subsequent infrared reflection intensity. The dividing area may be preset or may be flexibly calculated. For example, according to historical video stream data of the video acquisition device, judging an active area in a picture range corresponding to an acquisition environment of the video acquisition device, if video frames with moving objects in the historical video stream data are extracted to obtain active video frames, acquiring the motion ranges of the moving objects in the active video frames, assigning a value to each pixel point in the picture corresponding to the acquisition environment based on each motion range corresponding to the active video frames, taking the pixel point with an assignment result larger than a preset value as an active pixel point, and obtaining the active area based on each active pixel point. The assignment process may be: mapping the positions of the pixel points into each active video frame, if the positions of the pixel points are in the motion range of the active video frame, determining that the pixel points are in association with the active video frame, and counting the number of the active video frames with association of each pixel point to obtain a value assignment result.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating video frame segmentation according to an exemplary embodiment of the present application, and as shown in fig. 3, a segmentation area in a video frame is determined, and image block division is performed on the segmentation area in the video frame based on an image block segmentation size to obtain a plurality of image blocks.
Then, based on the pixel information of each image block, a feature value corresponding to each image block is calculated.
In some embodiments, calculating the feature value corresponding to each image block based on the pixel information of each image block includes: extracting a plurality of channel pixel values of pixel points in an image block; based on the plurality of channel pixel values, a corresponding feature value for each image block is calculated.
The pixel point contains a plurality of channel pixel values, such as channel pixel values of three color channels of red (red, R), green (G), blue (B).
Taking a channel pixel value with a pixel point containing a R, G, B color channel as an example, the calculation process of the characteristic value is described as follows:
and calculating the average channel pixel value of each color channel of each pixel point in the image block, for example, obtaining the average channel pixel values of each color channel corresponding to the image block i as Ri, gi and Bi, and then taking Ri/Gi and Bi/Gi as the characteristic values corresponding to the image block i to represent the infrared ray component contained in the image block i.
Further, in some embodiments, obtaining the infrared reflection intensity corresponding to the video frame based on the feature value corresponding to each image block includes: acquiring a preset reference characteristic value; calculating the similarity between the characteristic value corresponding to each image block and a preset reference characteristic value to obtain the similarity of the image blocks corresponding to each image block; and obtaining the infrared reflection intensity corresponding to the video frame based on the similarity of the image blocks corresponding to each image block.
The preset reference characteristic value can be a characteristic value of a video frame acquired by the video acquisition device under the environment of pre-calibrating pure infrared light, the environment of calibrating the pure infrared light can be in a sealed small space, and the specified infrared light supplementing lamp is used for irradiating an image acquisition sensor corresponding to the video acquisition device. The characteristic value of the video frame acquired by the video acquisition device under the environment of pre-calibrating pure infrared light can be calculated by adopting the same calculation mode as the characteristic value corresponding to the calculated image block. And because the infrared light supplementing light source is uniform, sufficient and single in the environment of the pure infrared light calibration in advance, the characteristic values of all the image blocks obtained in the environment of the pure infrared light calibration are relatively close, so that the image blocks are taken as the pure infrared image blocks, the characteristic values of all the pure infrared image blocks are accumulated and then averaged, and the preset reference characteristic values Rir/Gir and Bir/Gir are obtained.
For example, referring to fig. 4, fig. 4 is a schematic diagram showing similarity of calculated feature values according to an exemplary embodiment of the present application, as shown in fig. 4, the feature value corresponding to each image block and the preset reference feature value are mapped to a designated coordinate system, and distances between coordinates (Ri/Gi, bi/Gi) of the feature value corresponding to the image block i and coordinates (Rir/Gir, bir/Gir) of the preset reference feature value are calculated, so as to obtain a similarity calculation result corresponding to the image block i. The smaller the distance is, the more similar the feature value corresponding to the image block is to the preset reference feature value, otherwise, the larger the distance is, the more dissimilar the feature value corresponding to the image block is to the preset reference feature value.
And then, obtaining the infrared reflection intensity corresponding to the video frame based on the similarity of the image blocks corresponding to each image block. In some embodiments, the method specifically may include: acquiring a preset similarity threshold; comparing the similarity of the image blocks corresponding to each image block with a preset similarity threshold value, and classifying each image block based on a comparison result; and calculating the infrared reflection intensity corresponding to the video frame based on the number of the image blocks of each category in the classification result.
For example, taking the distance L as a preset similarity threshold, as shown in a graph of fig. 4, in a two-dimensional coordinate system, the calibrated preset reference feature value is a circle center Xir (Rir/Gir, bir/Gir), if the coordinates of the feature value of the image block are located in a circle with Xir as the circle center, that is, if the distance between the coordinates is smaller than L, the image block is divided into similar image blocks, otherwise, the image block with the coordinates of the feature value not located in the circle with Xir as the circle center is divided into dissimilar image blocks. And then counting the number of the similar image blocks, and calculating the infrared reflection intensity corresponding to the video frame based on the number of the similar image blocks.
The calculation formula corresponding to the infrared reflection intensity (ratio) may be:
ration＝m/M(0<ration<1)
where M is the number of similar image blocks, and M is the total number of image blocks segmented in the video frame.
In other embodiments, the infrared reflection intensity corresponding to the video frame may be calculated in other manners, for example, the gain value gain and the target brightness value ev of the video acquisition device are obtained, and the infrared reflection intensity corresponding to the video frame is calculated based on the gain value gain and the target brightness value ev. Under the automatic exposure of the video capture device, if the image range corresponding to the capture environment of the video capture device is darker, the gain value of the automatic exposure is larger, the average brightness value is smaller, the target brightness value set by video frame capture is smaller, otherwise, if the image range corresponding to the capture environment of the video capture device is brighter, the gain value of the automatic exposure is smaller, the average brightness value is larger, the target brightness value set by video frame capture is larger, and different target brightness values, such as (0 db, decibel), 100, (6 db, 90), (12 db, 80), (18 db, 70) (24 db, 60) (30 db, 50), are set under different gain values, so that gainevration=k is defined. When no object is reflected in the acquisition environment of the video acquisition device in the low-illumination environment, the darker the picture range is, the larger the GainEvRation is, and when the object is reflected, the brighter the picture range is, and the smaller the GainEvRation is. The calculation formula of the infrared reflection intensity can be as follows:
ration＝(GainEvRationTargt/GainEvRationCur)
the GainEvrationTargt is a GainEvration value when a specified distance exists between the acquisition object and the video acquisition device, and the picture range of the video acquisition device is considered to be in the brightest state when the specified distance exists; the GainEvrationCur is a GainEvration value of the current video acquisition device and is used for reflecting the reflection state of the picture range of the current video acquisition device, and the infrared reflection intensity corresponding to the video frame acquired by the current video acquisition device is obtained through the ratio between GainEvrationTargt and GainEvrationCur.
Next, a process of calculating intensity fluctuation corresponding to the video frame will be described in detail:
based on the infrared reflection intensity corresponding to each video frame, calculating intensity fluctuation corresponding to the video frame sequence, comprising: calculating the variation of the infrared reflection intensity between adjacent video frames in the video frame sequence; and counting the variation quantity between each pair of adjacent video frames in the video frame sequence to obtain the intensity fluctuation corresponding to the video frame sequence.
Extracting adjacent video frames in the video frame sequence, calculating the variation of the infrared reflection intensity between the adjacent video frames, and then calculating the average value of the variation between each pair of adjacent video frames in the video frame sequence to obtain an average variation delta t, wherein the average variation delta t is used as the intensity fluctuation corresponding to the video frame sequence.
Next, a detailed description is given of an adjustment process of the video acquisition frame rate:
in some embodiments, adjusting the video capture frame rate of the video capture device in the infrared light supplemental state based on the intensity fluctuations comprises: if the intensity fluctuation is smaller than a preset frame dropping fluctuation threshold value, performing downward adjustment processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state; if the intensity fluctuation is larger than a preset frame rising fluctuation threshold, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted upwards.
The preset frame fluctuation threshold and the preset frame fluctuation threshold may be preset or flexibly acquired, for example, the preset frame fluctuation threshold and the preset frame fluctuation threshold are flexibly calculated based on video quality requirements input by a user, environmental brightness in a corresponding acquisition environment of a current video acquisition device, and the like.
The preset falling frame fluctuation threshold is less than or equal to the preset rising frame fluctuation threshold.
If the intensity fluctuation is smaller than a preset frame dropping fluctuation threshold value, the condition that a moving acquisition object does not exist in an acquisition scene corresponding to the video acquisition device is considered, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state can be adjusted down, the picture quality is improved, and the video storage pressure is optimized; if the intensity fluctuation is larger than a preset frame rising fluctuation threshold, the acquisition object in a moving state exists in an acquisition scene corresponding to the video acquisition device, so that the video acquisition frame rate of the video acquisition device in an infrared light supplementing state can be adjusted upwards, and the fluency of the moving object in the acquired video is ensured.
For example, if the intensity fluctuation is greater than a preset frame rising fluctuation threshold, performing up-regulation processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state, including: acquiring a preset reflection intensity threshold value; and obtaining a target reflection intensity corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame in the video frame sequence; if the target reflection intensity is larger than a preset reflection intensity threshold value and the intensity fluctuation is larger than a preset frame rising fluctuation threshold value, the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted upwards.
The infrared reflection intensity with the largest value in each video frame can be used as the target reflection intensity corresponding to the video frame sequence; or the latest generated video frame in the video frame sequence is taken as a reference video frame, and the infrared reflection intensity corresponding to the reference video frame is taken as the target reflection intensity corresponding to the video frame sequence; the method can also be used for carrying out average calculation on the infrared reflection intensity corresponding to each video frame, and taking the average infrared reflection intensity as the target reflection intensity corresponding to the video frame sequence.
And comparing the target reflection intensity with a preset reflection intensity threshold value.
The preset reflection intensity threshold may be preset or flexibly acquired, for example, the preset reflection intensity threshold may be flexibly calculated based on a video quality requirement input by a user, an environmental brightness in a corresponding acquisition environment of the current video acquisition device, and the like.
If the target reflection intensity is larger than a preset reflection intensity threshold value and the intensity fluctuation is larger than a preset frame rising fluctuation threshold value, the acquisition object in a moving state is considered to exist in the acquisition scene corresponding to the video acquisition device, the distance between the acquisition object and the video acquisition device reaches a preset acquisition condition, and the video acquisition frame rate of the video acquisition device in the infrared light supplementing state can be adjusted upwards.
In addition, in the image control system of the image capturing apparatus, the relationship of the frame rate (frame rate), the system clock (clksys), the vertical total line number (Vertical total Size, VTS), and the horizontal total column number (Horizontal Total Size, HTS) is expressed as follows: clksys=frame rate×vts×hts×α, where α is a constant, and in this expression, since clksys, HTS, and α are all constant values, frame rate is inversely proportional to VTS, and frame rate can be respectively increased and decreased by changing VTS, thereby changing the adjustable range of exposure. Meanwhile, the relation between the frame rate and the maximum exposure time per frame (shartmaxe) satisfies: the shift maxe=1/frame, for example, the maximum exposure time is 40ms when the frame rate is 25 frames, and 80ms when the frame rate is reduced to 12.5 frames. Therefore, in the frame rate adjustment process, the maximum exposure time is also subjected to frame rate adjustment synchronization adjustment.
For example, referring to fig. 5, fig. 5 is a flowchart of a frame rate adjustment method according to another exemplary embodiment of the present application, as shown in fig. 5, an infrared reflection intensity corresponding to each video frame in a video frame sequence is calculated, so as to obtain intensity fluctuation Δt corresponding to the video frame sequence, and whether Δt is smaller than a preset frame fluctuation threshold t1 or whether Δt is larger than a preset frame fluctuation threshold t2 is determined.
If Δt is smaller than a preset frame-dropping fluctuation threshold t1, the video-capturing frame rate is reduced to fps=std_fps/a, std_fps is a standard output frame rate, a is a frame-dropping parameter, the exposure maximum time is a (1/std_fps), at this time, the video-capturing device is in a frame-dropping state, the video-capturing frame rate is 1/a of the standard output frame rate, and the maximum exposure time is a times. The frame dropping parameter a may be a preset fixed value or a flexibly calculated value, for example, the frame dropping parameter a is calculated according to the intensity fluctuation Δt.
If Δt is greater than the preset frame-dropping fluctuation threshold t2, then judging whether the ratio is greater than the preset reflection intensity threshold thr1, if the ratio is greater than the preset reflection intensity threshold thr1, gradually increasing the video acquisition frame rate fps to std_fps, and gradually decreasing the maximum exposure time to the maximum exposure time corresponding to the standard output frame rate.
The influence of frame rate adjustment on video stream data is reduced by gradually increasing the frame rate or reducing the frame rate, so that the fluency of video frame pictures is ensured.
According to the frame rate adjusting method provided by the application, the video frame sequence acquired by the video acquisition device in the infrared light supplementing state is acquired, the video frame sequence contains at least two adjacent video frames, the infrared light reflecting intensity corresponding to each video frame in the video frame sequence is calculated, the intensity fluctuation corresponding to the video frame sequence is calculated based on the infrared light reflecting intensity corresponding to each video frame, and the video acquisition frame rate of the video acquisition device in the infrared light supplementing state is adjusted based on the intensity fluctuation, so that complex hardware and chip performance support are not needed, the video frame in the video stream is only analyzed through statistics, the intensity fluctuation of the infrared light reflecting intensity of the acquired picture is judged in the infrared light supplementing environment, and the frame rate self-adaption adjustment is performed, so that the quality of video acquisition is improved.
Fig. 6 is a block diagram of a frame rate adjustment apparatus according to an exemplary embodiment of the present application. As shown in fig. 6, the exemplary frame rate adjustment apparatus 600 includes: an acquisition module 610, a reflection intensity calculation module 620, an intensity fluctuation calculation module 630, and a frame rate adjustment module 640. Specifically:
an acquisition module 610, configured to acquire a video frame sequence acquired by a video acquisition device in an infrared light supplementing state, where the video frame sequence contains at least two adjacent video frames;
the reflection intensity calculating module 620 is configured to calculate an infrared reflection intensity corresponding to each video frame in the video frame sequence;
an intensity fluctuation calculating module 630, configured to calculate intensity fluctuation corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame;
and the frame rate adjusting module 640 is configured to adjust a video acquisition frame rate of the video acquisition device in the infrared light supplementing state based on the intensity fluctuation.
In the above-mentioned exemplary frame rate adjustment device, complex hardware and chip performance support are not required, and only by statistically analyzing video frames in a video stream, in an infrared light supplementing environment, intensity fluctuation of infrared reflection intensity of an acquisition picture is judged, frame rate self-adaptive adjustment is performed, and quality of video acquisition is improved.
The functions of each module may be described in the frame rate adjustment method embodiments, and are not described herein.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the application. The electronic device 700 comprises a memory 701 and a processor 702, the processor 702 being arranged to execute program instructions stored in the memory 701 to implement the steps of any of the frame rate adjustment method embodiments described above. In one particular implementation scenario, electronic device 700 may include, but is not limited to: the microcomputer and the server, and the electronic device 700 may also include a mobile device such as a notebook computer and a tablet computer, which is not limited herein.
Specifically, the processor 702 is configured to control itself and the memory 701 to implement the steps in any of the frame rate adjustment method embodiments described above. The processor 702 may also be referred to as a central processing unit (Central Processing Unit, CPU). The processor 702 may be an integrated circuit chip with signal processing capabilities. The processor 702 may also be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 702 may be commonly implemented by an integrated circuit chip.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an embodiment of a computer readable storage medium according to the present application. The computer readable storage medium 800 stores program instructions 810 that can be executed by a processor, the program instructions 810 being configured to implement the steps in any of the above-described frame rate adjustment method embodiments.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Claims (10)
1. A method of frame rate adjustment, the method comprising:
acquiring a video frame sequence acquired by a video acquisition device in an infrared light supplementing state, wherein the video frame sequence comprises at least two adjacent video frames;
calculating the infrared reflection intensity corresponding to each video frame in the video frame sequence;
calculating intensity fluctuation corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame;
and adjusting the video acquisition frame rate of the video acquisition device in the infrared light supplementing state based on the intensity fluctuation.
2. The method of claim 1, wherein said calculating an infrared reflection intensity for each video frame in said sequence of video frames comprises:
dividing the video frame to obtain a plurality of image blocks;
calculating a characteristic value corresponding to each image block based on pixel information of each image block;
and obtaining the infrared reflection intensity corresponding to the video frame based on the characteristic value corresponding to each image block.
3. The method according to claim 2, wherein calculating the feature value corresponding to each image block based on the pixel information of each image block includes:
extracting a plurality of channel pixel values of pixel points in the image block;
and calculating the characteristic value corresponding to each image block based on the channel pixel values.
4. The method according to claim 2, wherein the obtaining the infrared reflection intensity corresponding to the video frame based on the feature value corresponding to each image block includes:
acquiring a preset reference characteristic value;
calculating the similarity between the characteristic value corresponding to each image block and the preset reference characteristic value to obtain the image block similarity corresponding to each image block;
and obtaining the infrared reflection intensity corresponding to the video frame based on the image block similarity corresponding to each image block.
5. The method of claim 4, wherein the obtaining the infrared reflection intensity corresponding to the video frame based on the image block similarity corresponding to each image block comprises:
acquiring a preset similarity threshold;
comparing the similarity of the image blocks corresponding to each image block with the preset similarity threshold value, and classifying each image block based on a comparison result;
and calculating the infrared reflection intensity corresponding to the video frame based on the number of the image blocks of each category in the classification result.
6. The method of claim 1, wherein calculating intensity fluctuations corresponding to the sequence of video frames based on the infrared reflection intensity corresponding to each video frame comprises:
calculating the variation of the infrared reflection intensity between adjacent video frames in the video frame sequence;
and counting the variation between each pair of adjacent video frames in the video frame sequence to obtain the intensity fluctuation corresponding to the video frame sequence.
7. The method of claim 1, wherein adjusting the video capture frame rate of the video capture device in the infrared light supplemental state based on the intensity fluctuations comprises:
if the intensity fluctuation is smaller than a preset frame dropping fluctuation threshold value, performing downward adjustment processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state;
and if the intensity fluctuation is larger than a preset frame rising fluctuation threshold, performing up-regulation processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state.
8. The method according to claim 7, wherein if the intensity fluctuation is greater than a preset frame rising fluctuation threshold, the step of up-regulating the video capturing frame rate of the video capturing apparatus in the infrared light filling state includes:
acquiring a preset reflection intensity threshold value; and obtaining a target reflection intensity corresponding to the video frame sequence based on the infrared reflection intensity corresponding to each video frame in the video frame sequence;
and if the target reflection intensity is greater than a preset reflection intensity threshold and the intensity fluctuation is greater than a preset frame rising fluctuation threshold, performing up-regulation processing on the video acquisition frame rate of the video acquisition device in the infrared light supplementing state.
9. An electronic device comprising a memory and a processor for executing program instructions stored in the memory to implement the steps of the method according to any of claims 1-8.
10. A computer readable storage medium storing program instructions executable by a processor to perform the steps of the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310660902.7A CN116916116A (en) | 2023-06-05 | 2023-06-05 | Frame rate adjustment method, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310660902.7A CN116916116A (en) | 2023-06-05 | 2023-06-05 | Frame rate adjustment method, storage medium and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116916116A true CN116916116A (en) | 2023-10-20 |
Family
ID=88367449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310660902.7A Pending CN116916116A (en) | 2023-06-05 | 2023-06-05 | Frame rate adjustment method, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116916116A (en) |
-
2023
- 2023-06-05 CN CN202310660902.7A patent/CN116916116A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10070053B2 (en) | Method and camera for determining an image adjustment parameter | |
| JP6816097B2 (en) | Methods and equipment for determining depth maps for images | |
| CN108702437B (en) | Method, system, device and storage medium for calculating depth map | |
| CN108322646B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US8605185B2 (en) | Capture of video with motion-speed determination and variable capture rate | |
| CN110022469B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN109361910B (en) | Self-adaptive white balance correction method and device | |
| WO2021109620A1 (en) | Exposure parameter adjustment method and apparatus | |
| US9357127B2 (en) | System for auto-HDR capture decision making | |
| US11558554B2 (en) | Optical distance measurement system and imaging system with dynamic exposure time | |
| KR102015086B1 (en) | Auto white balance using infrared and / or ultraviolet signals | |
| JP2018503325A (en) | System and method for performing operations on pixel data | |
| CN108322651B (en) | Photographing method and device, electronic equipment and computer readable storage medium | |
| KR20170030933A (en) | Image processing device and auto white balancing metohd thereof | |
| US8798369B2 (en) | Apparatus and method for estimating the number of objects included in an image | |
| Hertel et al. | Image quality standards in automotive vision applications | |
| CN110930340B (en) | Image processing method and device | |
| CN116916116A (en) | Frame rate adjustment method, storage medium and electronic device | |
| CN113228621B (en) | Imaging control method, system and non-transitory computer readable medium | |
| US11743444B2 (en) | Electronic device and method for temporal synchronization of videos | |
| CN108337448B (en) | High dynamic range image acquisition method and device, terminal equipment and storage medium | |
| EP4090006A2 (en) | Image signal processing based on virtual superimposition | |
| CN114513611B (en) | Method and electronic device for increased dynamic range of images | |
| CN114630095B (en) | Automatic white balance method and device for target scene image and terminal | |
| US11417151B2 (en) | Adaptive rolling shutter image sensor and IR emitter control for facial recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |