WO2023025010A1 - 频闪条带信息识别方法、装置和电子设备 - Google Patents

频闪条带信息识别方法、装置和电子设备 Download PDF

Info

Publication number
WO2023025010A1
WO2023025010A1 PCT/CN2022/113178 CN2022113178W WO2023025010A1 WO 2023025010 A1 WO2023025010 A1 WO 2023025010A1 CN 2022113178 W CN2022113178 W CN 2022113178W WO 2023025010 A1 WO2023025010 A1 WO 2023025010A1
Authority
WO
WIPO (PCT)
Prior art keywords
vector
information
image
stroboscopic
preview image
Prior art date
Application number
PCT/CN2022/113178
Other languages
English (en)
French (fr)
Inventor
程林
Original Assignee
维沃移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 维沃移动通信有限公司 filed Critical 维沃移动通信有限公司
Publication of WO2023025010A1 publication Critical patent/WO2023025010A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters

Definitions

  • the application belongs to the technical field of image recognition, and in particular relates to a method, device and electronic equipment for stroboscopic strip information recognition.
  • the electronic rolling shutter is a shooting device on electronic equipment.
  • the electronic rolling shutter controls the image sensor so that different parts have different sensitivities to light at different times, and exposes row by row until all pixels are covered. exposure. However, due to the different exposure time of each line, the obtained light energy may also be different; if the shutter receives different light energy on different photosensitive surfaces, it will produce stroboscopic bands on the image, that is, banding phenomenon, which affects The imaging effect of electronic equipment shooting.
  • the recoil band value is a device parameter used to counteract the flickering frequency of the light source when the electronic device is shooting, and the recoil is automatically determined by the electronic device when shooting. With value, determine whether banding occurs when shooting.
  • different light sources in different environments have certain complexity, and the accuracy of the recoil zone value determined automatically is poor.
  • the electronic device cannot accurately identify whether there are stroboscopic bands during imaging, it will cause the device to complete the shooting even if banding occurs, and output images with stroboscopic bands, making the shooting rate low and affecting users' shooting experience.
  • the purpose of the embodiments of the present application is to provide a method, device and electronic device for identifying stroboscopic band information, capable of accurately detecting stroboscopic band information in images captured by the electronic device.
  • the embodiment of the present application provides a method for identifying stroboscopic band information, the method comprising:
  • the mean value coded vector information is fused with the image feature information extracted from the shooting preview image to obtain the position and/or intensity information of the stroboscopic band.
  • a stroboscopic strip identification device which includes:
  • the first determination module determines the pixel mean value vector corresponding to the preset imaging direction according to the shooting preview image
  • the first extraction module is used to extract vector feature information according to the pixel mean value vector to obtain mean value coded vector information
  • the fusion module is used to fuse the mean coding vector information with the image feature information extracted from the shooting preview image to obtain the position and /or strength information.
  • an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is The processor implements the steps of the method described in the first aspect when executed.
  • an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .
  • the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.
  • the embodiment of the present application it is possible to detect the shooting preview image, determine the pixel mean vector corresponding to the preset imaging direction according to the shooting preview image, extract the vector feature information according to the pixel mean value vector, obtain the mean value encoding vector information, and according to the mean value Encoding vector information determines whether there is a stroboscopic band. Since the generation of stroboscopic bands is related to the imaging direction of the image, the embodiment of the present application extracts feature information based on the pixel mean vector corresponding to the imaging direction, and can obtain more accurate stroboscopic band identification results for each frame of preview image.
  • the mean value coded vector information is fused with the image feature information extracted from the shot preview image to obtain the position and intensity information of the stroboscopic band; the embodiment of the present application combines the stroboscopic band
  • the identification of the band information is accurate to the position and intensity, which is beneficial to remove the stroboscopic band based on this information to obtain an image without banding phenomenon, and then improve the filming rate of electronic equipment.
  • FIG. 1 is a schematic flow diagram of a method for identifying stroboscopic band information provided in an embodiment of the present application
  • FIG. 2 is a schematic diagram of cutting a shooting preview image into multiple sub-images in a specific example of the present application
  • Fig. 3 is a schematic diagram of vector feature extraction through the second model in a specific example of the present application.
  • Fig. 4 is a schematic flow chart of step S103 in a specific embodiment of the present application.
  • Fig. 5 is a schematic diagram of image feature extraction and banding information identification through the first model in a specific example of the present application
  • Fig. 6 is a schematic structural diagram of a stroboscopic strip information identification device provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a hardware structure of an electronic device provided by another embodiment of the present application.
  • the banding of the image refers to the fact that the light of the light source flickers at a fixed frequency, so that the brightness of the light changes continuously, resulting in a banding effect after imaging.
  • the Chinese household electricity standard is 220V 50Hz
  • the fluctuation of light intensity is 100Hz
  • the cycle is 10ms.
  • electronic devices such as cameras or mobile phones
  • the energy obtained may also be different; then the light energy received on different photosensitive surfaces is different, resulting in stroboscopic fringes on the image.
  • one solution is to control the shutter speed within a certain threshold range, so that a lower shutter speed can avoid banding, but at the same time limit the imaging quality, resulting in partial Fuzzy, low image quality.
  • Another solution is to judge the flickering frequency of the light source through hardware, so as to control the shutter speed and avoid banding.
  • the accuracy of the hardware in judging the flicker frequency of the light source is low, and it cannot adapt to light sources of different frequencies, or even complex light sources. Therefore, due to poor applicability, the occurrence of banding cannot be completely avoided during imaging.
  • the recoil zone value automatically determined by the electronic device is used to determine whether banding occurs during shooting.
  • the automatically determined recoil band value has low adaptability, so the accuracy of judging whether banding occurs in this way is also low.
  • embodiments of the present application provide a method, device, electronic device, and readable storage medium for identifying stroboscopic band information.
  • a mobile electronic device with a shooting function may be a digital camera (Digital Still Camera, DSC, referred to as: Digital Camera, DC), a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a wearable device, a super Mobile personal computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (personal digital assistant, PDA), etc.
  • non-mobile electronic devices with shooting functions can be personal computer (personal computer, PC), television (television , TV), teller machines or self-service machines, etc., are not specifically limited in this embodiment of the present application.
  • FIG. 1 shows a schematic flowchart of a method for identifying stroboscopic band information provided by an embodiment of the present application. As shown in Figure 1, the method may include steps S101-S104:
  • S101 Acquire a shooting preview image.
  • the shooting preview image may be a preview image when the user shoots through the electronic device.
  • S102 Determine a pixel mean value vector corresponding to a preset imaging direction according to the shooting preview image.
  • the preset imaging direction may be the scanning direction of an image sensor (sensor) of the shutter when the electronic device is shooting. Since the shutter scanning direction of each electronic device is fixed, such as scanning from top to bottom to expose pixels for imaging row by row, the imaging direction of each frame of captured image is the same.
  • the inventors of the present application found that since the stroboscopic banding is related to the scanning direction of the image sensor of the electronic device, the banding of the image will only appear as a band when imaging, and the banding of the banding still exists even if it is imaged by a complex light source .
  • the shooting preview image is subjected to data processing corresponding to its preset imaging direction, and the pixel mean value vector corresponding to its preset imaging direction is calculated, so that through subsequent feature extraction processing on the pixel mean value vector, and then Quickly get the judgment result of whether it is in banding.
  • Feature extraction is performed on the pixel mean value vector, and the vector feature of the pixel mean value vector corresponding to the imaging direction is extracted to obtain mean value coded vector information.
  • the vector feature information is used to identify whether banding occurs in the image, which can better capture the banding information in each frame of the image, and improve the accuracy and efficiency of banding identification in the image.
  • the mean coding vector information is fused with the image feature information extracted from the shooting preview image to obtain the position and/or intensity of the stroboscopic band information.
  • the mean value coded vector information is fused with the image feature information extracted from the shot preview image, and the pixel mean vector feature is fused with the image feature information, The position and/or intensity information of the stroboscopic bands are identified through the fused data with richer information features.
  • the embodiment of the present application it is possible to detect the shooting preview image, determine the pixel mean vector corresponding to the preset imaging direction according to the shooting preview image, extract the vector feature information according to the pixel mean value vector, obtain the mean value encoding vector information, and according to the mean value Encoding vector information determines whether there is a stroboscopic band. Since the generation of stroboscopic bands is related to the imaging direction of the image, the embodiment of the present application extracts feature information based on the pixel mean vector corresponding to the imaging direction, and can obtain more accurate stroboscopic band identification results for each frame of preview image.
  • the mean value coded vector information is fused with the image feature information extracted from the shot preview image to obtain the position and intensity information of the stroboscopic band; the embodiment of the present application combines the stroboscopic band
  • the identification of the band information is accurate to the position and intensity, which is beneficial to remove the stroboscopic band based on this information to obtain an image without banding phenomenon, and then improve the filming rate of electronic equipment.
  • step S101 in the embodiment of the present application, when the data processing of the shooting preview image is performed through step S102, the following steps S201-S204 may be included :
  • the shooting preview image is grayscaled, that is, the RGB (R: Red, G: Green, B: Blue) three components of each pixel of the image are unified into the same value, which simplifies the difficulty of image processing.
  • S202 Obtain a first pixel mean value vector corresponding to a preset imaging direction of the shooting preview image.
  • the shooting preview image is corresponding to its preset imaging direction to count the average value of the pixels in the image to obtain the first average value vector of the pixels.
  • S203 Obtain a second pixel mean value vector corresponding to a preset imaging direction of the sub-image, where there are multiple sub-images, and the multiple sub-images are obtained by segmenting the shooting preview image according to the preset imaging direction.
  • the light source Due to the flickering characteristics of the light source, in general, the light source presents periodic brightness changes. This brightness change may be reflected in the banding generated by the image to a certain extent, and in order to reduce the impact of the image itself (such as the image background) on the subsequent detection results. Therefore, in this step, the shooting preview image is cut according to the preset imaging direction to obtain multiple sub-images. As shown in FIG. 2 , the shooting preview image 201 is cut into five sub-images 202 according to the scanning direction from top to bottom.
  • the size of each sub-image is equal during cutting, where the unit of size is pixel.
  • the number of cuts of the shooting preview image may not be limited, and the sizes of the cut sub-images may also be unequal.
  • the pixel mean values of the divided sub-images are respectively calculated according to the preset imaging directions to obtain the pixel mean value vectors of each sub-image in the preset imaging direction, that is, the vector pixel mean values of multiple sub-images are obtained.
  • the pixel mean values in the first pixel mean value vector and the second pixel mean value vector are normalized so that each pixel mean value is within a suitable preset threshold range, which can speed up the efficiency of the subsequent processing process.
  • vector feature extraction can be performed through step S103.
  • a preset neural network is used to extract and identify features.
  • the neural network may include a first model and a second model, wherein the first model may be used for image feature extraction and identification of specific banding information, and the second model may be used for pixel mean vector feature extraction and identification of whether banding exists.
  • the second model may include a fully connected layer (Fully Connected Layer) and a classifier, then by step S103 extracting the vector feature information according to the pixel mean value vector, and obtaining the mean value encoding vector information, specifically may include Steps S301-S302:
  • one first pixel mean value vector containing global pixel information and five second pixel mean value vectors containing local pixel information can be obtained.
  • the above-mentioned six pixel mean value vectors 301 are input into the second model, and in the second model, the vector feature information is extracted and integrated through the fully connected layer 302, and the vector feature connection of each pixel mean value vector can be obtained.
  • the means together encode vector information 303 .
  • the process of obtaining the mean encoding vector information is a process of encoding and unifying the information of the image mean vector, which can better represent the strip data information of the image.
  • the six vector data can be mapped into fixed-length feature vectors (ie, average value coded vectors), and the shooting preview image is retained in the average value coded vector information
  • the key information of interest in the image is ignored, and irrelevant information such as location is ignored, which is conducive to quickly calculating the characteristics of the shooting preview image and determining whether there is banding.
  • the method for identifying stroboscopic band information further includes:
  • S303 Classify the mean coded vector information by a binary classification prediction algorithm, and determine whether there is a stroboscopic band in the shooting preview image according to the classification result.
  • step S303 may be performed through a second model, wherein the classifier included in the second model may be a binary classifier implemented by a binary classification prediction algorithm, and the binary classification prediction algorithm may be a softmax function (that is, a normalized index function).
  • the classifier included in the second model may be a binary classifier implemented by a binary classification prediction algorithm, and the binary classification prediction algorithm may be a softmax function (that is, a normalized index function).
  • the mean coded vector information 303 obtained in step S302 is connected to the fully connected layer 304 and the softmax function, wherein the softmax function can be used to perform logistic regression processing on the data to obtain binary classification output results.
  • the mean coded vector information is processed by the softmax function, and the output classification results include two probability values p1 and p2, p1 represents the probability of banding in the image, p2 represents the probability of no banding in the image, and the sum of p1 and p2 is 1.
  • the probability value of p1 is greater than p2, it means that there is banding in the image, and proceed to step S104; otherwise, if the probability of p1 is less than p2, it means that there is no banding in the image, and the shooting preview image can be directly output to obtain the final imaging. In this way, it is possible to quickly and accurately determine whether there is banding in the shooting preview image, thereby avoiding the output of images with stroboscopic banding, and avoiding unnecessary subsequent banding removal operations by electronic equipment when the captured image has no banding, thereby ensuring shooting efficiency.
  • the shooting preview image can be canceled to continue to output the captured image, or the stroboscopic banding can be eliminated by means of banding removal, and then the captured image can be output, thereby improving the shooting rate of the electronic device.
  • step S104 may be performed based on the first model to identify the position and/or of the stroboscopic banding in the image or intensity. Specifically, as shown in FIG. 4, step S104 may include steps S401-S403:
  • the first model may be a neural network, and the neural network includes a first network layer, and the first network layer includes at least a convolution layer, and may also include a pooling layer.
  • the convolution layer includes a plurality of sequentially connected strip convolution kernels, each strip convolution kernel includes two asymmetric convolution blocks, and the strip convolution kernels have different sizes.
  • Step S401 extracts the image feature information of the shooting preview image and the sub-image through the first model, which may specifically include S4011-S4015:
  • the size of the images is unified, so as to reduce the complexity of image processing by the subsequent neural network.
  • the sizes of one shooting preview image and five sub-images are unified into a size of 512 ⁇ 512 pixels.
  • each image is processed by multiple banded convolution kernels in the convolution layer in the first network layer, and then pooled layer to obtain the two-dimensional information corresponding to the image features extracted, and then map each two-dimensional information into a fixed-length vector to obtain image feature encoding information.
  • each image at the first network layer includes:
  • a global convolution operation is performed on the image 501 input to the first network layer.
  • the convolution kernel can use a standard square convolution kernel, such as a size of 3 ⁇ 3, and the first matrix is obtained after the global convolution (not shown in the figure). marked).
  • the first matrix is then subjected to a second convolution operation by the banded convolution kernel.
  • the banded convolution kernel may consist of two asymmetric convolution blocks of 7 ⁇ 1 and 1 ⁇ 7.
  • the banded convolution kernel can better learn the banded banding information of the image, and the two asymmetric convolution blocks can be equivalent to the standard square convolution kernel of the same size, it will not increase the time overhead, and at the same time compared to The standard square convolution kernel can enhance the feature extraction accuracy, so in this embodiment, the features of the image are extracted through the strip convolution kernel.
  • the first matrix respectively passes through the two asymmetric convolution blocks corresponding to the first strip convolution kernel 502 (such as 7 ⁇ 1 and 1 ⁇ 7 convolution blocks), and using the additivity of convolution, it can be obtained with 7 ⁇ 7 square kernel equivalent of the second matrix.
  • the second matrix passes through the two asymmetric convolution blocks corresponding to the second strip convolution kernel 503 (such as 5 ⁇ 1 and 1 ⁇ 5 convolution blocks), and the second matrix equivalent to the 5 ⁇ 5 square kernel can be obtained.
  • Second matrix passes through the two asymmetric convolution blocks (such as 3 ⁇ 1 and 1 ⁇ 3 convolution blocks) corresponding to the third strip convolution kernel 504 respectively, and the third matrix equivalent to the 3 ⁇ 3 square kernel can be obtained.
  • the above-mentioned image feature extraction is performed on the shooting preview image and each sub-image respectively, and the image features of each image are respectively obtained, and the image feature of each image is two-dimensional information. Map the two-dimensional information corresponding to each image into a fixed-length feature vector, that is, image feature coding information 506, and determine the image feature coding information as image feature information to perform subsequent steps S402-S403.
  • the first model may include, in addition to the above-mentioned first network layer for image feature extraction, a second network layer for identifying banding specific information, and the second network layer includes at least a fully connected layer and a logistic regression function.
  • the images corresponding to different areas of the preview image have different receptive field information, and the fusion of feature information in multiple areas can assist the neural network to make more accurate judgments. Therefore, in this example, the image characteristic encoding information 506 including the global image feature and each local image feature is used to calculate the banding information, and the mean value encoding vector information 507 of the image is used to provide certain banding information.
  • image feature coding information 506 and mean value coding vector 507 are fused through the fully connected layer of the second network layer to obtain fusion data 508 of image features and pixel mean value features.
  • the regression prediction may be to perform Logistic regression on the fused data 508 to obtain a W ⁇ 1 vector 509, wherein W in the W ⁇ 1 vector 509 represents the width of the corresponding image, and the vector is a one-dimensional vector.
  • W in the W ⁇ 1 vector 509 represents the width of the corresponding image
  • the vector is a one-dimensional vector.
  • the broad side of the global image and the sub-image are consistent.
  • the data contained in the W ⁇ 1 vector corresponds to different image positions, and the data range corresponding to each position is between 0-1. The closer the value of each image area position is to 1, the stronger the banding strength is, and conversely, the closer the value of each image area position is to 0, the weaker the banding intensity is.
  • the method in the embodiment of the present application may further include steps S105-S107:
  • the neural network is used to learn the position and intensity of the stroboscopic bands in the image, and the banding is removed to obtain a target image without the stroboscopic bands, which is displayed on the screen of the electronic device.
  • the shooting parameters (such as shutter speed) of the electronic device may be adjusted through the determined position and intensity of the stroboscopic banding to reacquire the target image without stroboscopic banding.
  • the stroboscopic band information identification method of this embodiment can quickly and accurately determine whether there is banding in the image on a single preview image. On the one hand, it can prevent electronic equipment from performing unnecessary operations when there is no banding in the captured image, and on the other hand On the one hand, according to the banding detection results, it can assist in removing the banding in the image, so as to present the user with an imaging effect without banding.
  • step S201 may specifically include:
  • S501 Obtain a first pixel mean value vector and first pixel mean value vector interpolation of the shooting preview image corresponding to a preset imaging direction.
  • Calculating the first pixel mean value vector corresponding to the preset imaging direction of the shot preview image and the interpolation of the pixel mean value vector is equivalent to calculating the pixel mean value of the current frame shot preview image, and obtaining the pixel mean value of the next frame interpolated image of the frame shot preview image .
  • the captured image of the next frame of the preview image captured in the current frame is simulated, thereby forming continuous frame images.
  • the pixel mean vector is a vector of continuous frame images, and the vector is followed by the following steps S103-S104 can identify whether there is banding and the position and/or intensity of banding in the continuous frame images, and then simulate the change process of banding in the shooting process, which can be used to predict the banding situation in subsequent image shooting.
  • steps S103 and S104 based on continuous frame images is the same as the processing of steps S103 and S104 based on a single shot preview image, and will not be repeated here to avoid repetition.
  • the execution subject may be a stroboscopic band information identification device, or the stroboscopic band information identification device used to execute the stroboscopic band information A control module for the identification method.
  • the method for identifying stroboscopic band information performed by the stroboscopic band information identifying device is taken as an example to illustrate the stroboscopic band information identifying device provided in the embodiment of the present application.
  • Figure 6 shows a device for identifying stroboscopic band information provided by an embodiment of the present application.
  • the device includes:
  • An acquisition module 601 configured to acquire a shooting preview image
  • the first determination module 602 is configured to determine the pixel mean value vector corresponding to the preset imaging direction according to the shooting preview image;
  • the first extraction module 603 is configured to extract vector feature information according to the pixel mean value vector to obtain mean value encoded vector information
  • a fusion module 604 configured to fuse the mean coding vector information with the image feature information extracted from the shooting preview image when it is determined according to the mean coding vector information that there is a stroboscopic band in the shooting preview image , to obtain the position and/or intensity information of the stroboscopic strip.
  • the embodiment of the present application it is possible to detect the shooting preview image, determine the pixel mean vector corresponding to the preset imaging direction according to the shooting preview image, extract the vector feature information according to the pixel mean value vector, obtain the mean value encoding vector information, and according to the mean value Encoding vector information determines whether there is a stroboscopic band. Since the generation of stroboscopic bands is related to the imaging direction of the image, the embodiment of the present application extracts feature information based on the pixel mean vector corresponding to the imaging direction, and can obtain more accurate stroboscopic band identification results for each frame of preview image.
  • the mean value coded vector information is fused with the image feature information extracted from the shot preview image to obtain the position and intensity information of the stroboscopic band; the embodiment of the present application combines the stroboscopic band
  • the identification of the band information is accurate to the position and intensity, which is beneficial to remove the stroboscopic band based on this information to obtain an image without banding phenomenon, and then improve the filming rate of electronic equipment.
  • the first determining module 602 may specifically include:
  • the first obtaining sub-module 6021 is configured to obtain the first pixel mean value vector corresponding to the preset imaging direction of the shooting preview image;
  • the second acquisition sub-module 6022 is used to acquire the second pixel mean value vector corresponding to the preset imaging direction of the sub-image, where there are multiple sub-images, and the multiple sub-images are the shooting preview according to the preset imaging direction Image segmentation is obtained.
  • a preset neural network is used to extract and identify features.
  • the neural network may include a first model and a second model, wherein the first model may be used for image feature extraction and identification of specific banding information, and the second model may be used for pixel mean vector feature extraction and identification of whether banding exists.
  • the second model can include a fully connected layer (Fully Connected Layer) and a classifier
  • the classifier included in the second model can be a binary classifier realized by a binary classification prediction algorithm
  • the binary classification prediction algorithm can be softmax function (i.e. normalized exponential function).
  • the first extraction module 603 may include:
  • the first extraction sub-module 6031 is configured to extract vector feature information of the first pixel mean vector and the second pixel mean vector;
  • the first connection sub-module 6032 is configured to connect the vector feature information of the first pixel mean vector with the vector feature information of the second pixel mean vector to obtain the mean coded vector information.
  • the first determination sub-module 6033 is configured to classify the mean coded vector information through a binary classification prediction algorithm, and determine whether there is a stroboscopic band in the shooting preview image according to the classification result.
  • This example can quickly and accurately determine whether there is banding in the shooting preview image, thereby avoiding the output of images with stroboscopic banding, and avoiding unnecessary subsequent banding removal operations by electronic devices when the captured image has no banding, ensuring shooting efficiency.
  • the captured image can be canceled to continue outputting the captured image, or the captured image can be output after eliminating the stroboscopic banding by means of banding removal, thereby improving the filming rate of the electronic device.
  • the position and/or intensity of the stroboscopic band in the image may be identified through the first fusion module 604 based on the first model.
  • the first model may be a neural network, and the neural network includes a first network layer, and the first network layer includes at least a convolution layer, and may also include a pooling layer.
  • the convolution layer includes a plurality of sequentially connected strip convolution kernels, each strip convolution kernel includes two asymmetric convolution blocks, and the strip convolution kernels have different sizes.
  • the first model may also include a second network layer for identifying banding specific information, and the second network layer includes at least a fully connected layer and a logistic regression function.
  • the first fusion module 604 may include:
  • the second extraction sub-module 6041 is configured to extract the image feature information of the shooting preview image and sub-images through the first model when it is determined that there is a stroboscopic band in the shooting preview image;
  • the fusion sub-module 6042 is used to fuse the mean coded vector information with the image feature information to obtain the fusion feature information;
  • the prediction sub-module 6043 is configured to perform regression prediction according to the fused feature information to obtain the position and/or intensity information of the stroboscopic band in the shooting preview image.
  • the device may also include:
  • the elimination module 605 is configured to eliminate the stroboscopic band according to the position information when the position information of the stroboscopic band in the shooting preview image is obtained; In the case of the intensity information in the shooting preview image, eliminating the stroboscopic band according to the intensity information;
  • a display module 606, configured to display the target image where the stroboscopic band is eliminated.
  • the stroboscopic band information recognition device of this embodiment can quickly and accurately determine whether there is banding phenomenon in the single preview image, on the one hand, it can avoid unnecessary operations of electronic equipment when there is no banding in the captured image, and on the other hand On the one hand, according to the banding detection results, it can assist in removing the banding in the image, so as to present the user with an imaging effect without banding.
  • the device for identifying stroboscopic band information in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal.
  • the device may be a mobile electronic device or a non-mobile electronic device.
  • the mobile electronic device may be a digital camera, a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc.
  • the non-mobile electronic device may be a personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., which are not specifically limited in this embodiment of the present application.
  • the device for identifying stroboscopic band information in the embodiment of the present application may be a device with an operating system.
  • the operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.
  • the stroboscopic band information identification device provided in the embodiment of the present application can realize various processes realized by the method embodiments in FIG. 1 to FIG. 5 , and details are not repeated here to avoid repetition.
  • the embodiment of the present application further provides an electronic device 700, including a processor 701, a memory 702, and programs or instructions stored in the memory 702 and operable on the processor 701,
  • the program or instruction is executed by the processor 701
  • the processes of the above-mentioned embodiment of the method for identifying stroboscopic band information can be realized, and the same technical effect can be achieved. In order to avoid repetition, details are not repeated here.
  • the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 8 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
  • the electronic device 800 includes, but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810, etc. part.
  • the electronic device 800 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 810 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions.
  • a power supply such as a battery
  • the structure of the electronic device shown in FIG. 8 does not constitute a limitation to the electronic device.
  • the electronic device may include more or fewer components than shown in the figure, or combine some components, or arrange different components, and details will not be repeated here. .
  • the input unit 804 is used to obtain a shooting preview image
  • the processor 810 is configured to determine a pixel mean value vector corresponding to a preset imaging direction according to the shot preview image.
  • the mean value coded vector information is fused with the image feature information extracted from the shooting preview image to obtain the position and/or intensity information of the stroboscopic band.
  • the electronic device in this embodiment can detect the shooting preview image, determine the pixel mean value vector corresponding to the preset imaging direction according to the shooting preview image, extract vector feature information according to the pixel mean value vector, obtain the mean value encoding vector information, and Encoding vector information determines whether there is a stroboscopic band. Since the generation of stroboscopic bands is related to the imaging direction of the image, the embodiment of the present application extracts feature information based on the pixel mean vector corresponding to the imaging direction, and can obtain more accurate stroboscopic band identification results for each frame of preview image.
  • the mean value coded vector information is fused with the image feature information extracted from the shot preview image to obtain the position and intensity information of the stroboscopic band; the embodiment of the present application combines the stroboscopic band
  • the identification of the band information is accurate to the position and intensity, which is beneficial to remove the stroboscopic band based on this information to obtain an image without banding phenomenon, and then improve the filming rate of electronic equipment.
  • the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 is used for the image capture device (such as the image data of the still picture or video obtained by the camera) for processing.
  • the display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072 .
  • the touch panel 8071 is also called a touch screen.
  • the touch panel 8071 may include two parts, a touch detection device and a touch controller.
  • Other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here.
  • the memory 809 can be used to store software programs and various data, and the memory 809 can mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area can store an operating system, at least one function required applications or instructions (such as sound playback function, image playback function, etc.) and so on.
  • memory 809 may include volatile memory or nonvolatile memory, or, memory 809 may include both volatile and nonvolatile memory.
  • the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash.
  • ROM Read-Only Memory
  • PROM programmable read-only memory
  • Erasable PROM Erasable PROM
  • EPROM erasable programmable read-only memory
  • Electrical EPROM Electrical EPROM
  • EEPROM electronically programmable Erase Programmable Read-Only Memory
  • Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synch link DRAM , SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM).
  • RAM Random Access Memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM double data rate synchronous dynamic random access memory
  • Double Data Rate SDRAM Double Data Rate SDRAM
  • DDRSDRAM double data rate synchronous dynamic random access memory
  • Enhanced SDRAM, ESDRAM enhanced synchronous dynamic random access memory
  • Synch link DRAM , SLDRAM
  • Direct Memory Bus Random Access Memory Direct Rambus
  • the processor 810 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations of the operating system, user interface, and application programs, and modulates
  • the demodulation processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the foregoing modem processor may not be integrated into the processor 810 .
  • the embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or an instruction, and when the program or instruction is executed by the processor, each process of the above-mentioned embodiment of the method for identifying stroboscopic band information is implemented, And can achieve the same technical effect, in order to avoid repetition, no more details here.
  • the processor is the processor in the electronic device described in the above embodiments.
  • the readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and the like.
  • the embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to realize the above-mentioned strobe band information identification
  • the chip includes a processor and a communication interface
  • the communication interface is coupled to the processor
  • the processor is used to run programs or instructions to realize the above-mentioned strobe band information identification
  • chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.
  • the term “comprising”, “comprising” or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase “comprising a " does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.
  • the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
  • the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation.
  • the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to enable a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in various embodiments of the present application.
  • a terminal which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Abstract

本申请公开了一种频闪条带信息识别方法、装置和电子设备，属于图像识别技术领域。在本申请实施例中，能够通过对拍摄预览图像进行检测，根据拍摄预览图像确定其对应预设成像方向的像素均值向量，根据像素均值向量提取向量特征信息，得到均值编码向量信息，并根据均值编码向量信息判断是否存在频闪条带。基于对应成像方向的像素均值向量提取特征信息，能够针对每一帧预览图像得到更为准确的频闪条带识别结果。并且在判断存在频闪条带的情况下，将均值编码向量信息与拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和强度信息。

Description

频闪条带信息识别方法、装置和电子设备
相关申请的交叉引用
本申请主张2021年08月25日在中国提交的中国专利申请号202110982554.6的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于图像识别技术领域，具体涉及一种频闪条带信息识别方法、装置和电子设备。
背景技术
电子卷帘快门是电子设备上的一种拍摄装置，电子卷帘快门是通过控制图像传感器，使其不同部分在不同时间下对光的敏感度不同，逐行进行曝光，直到所有的像素点被曝光。但由于每一行曝光时间不同，获得的光能量也可能不同；如果快门在不同的感光面接收到的光能量不一样，则会产生图像上的频闪条带，也即产生banding现象，从而影响电子设备拍摄的成像效果。
为此，现有技术中有种通过反冲带值确定图像存在banding的方法，反冲带值是电子设备拍摄时用于对抗光源闪烁频率的设备参数，通过电子设备拍摄时自动确定的反冲带值，判定拍摄时是否产生了banding。但不同环境下的不同光源具有一定的复杂性，自动确定的反冲带值准确性较差。而如果电子设备不能准确识别出成像时是否产了频闪条带，会导致即使产生banding现象设备也会完成拍摄，并输出存在频闪带条的图像，使得拍摄成片率低，影响用户拍摄体验。
发明内容
本申请实施例的目的是提供一种频闪条带信息识别方法、装置和电子设备，能够准确检测电子设备拍摄图像中的频闪条带信息。
第一方面，本申请实施例提供了一种频闪条带信息识别方法，该方法包括：
获取拍摄预览图像；
根据拍摄预览图像，确定其对应预设成像方向的像素均值向量；
根据像素均值向量提取向量特征信息，得到均值编码向量信息；
在根据均值编码向量信息确定拍摄预览图像中存在频闪条带的情况下，将均值编码向量信息与对拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和/或强度信息。
第二方面，本申请实施例提供了一种频闪条带识别装置，装置包括：
获取模块，获取拍摄预览图像；
第一确定模块，根据拍摄预览图像，确定其对应预设成像方向的像素均值向量；
第一提取模块，用于根据像素均值向量提取向量特征信息，得到均值编码向量信息；
融合模块，用于在根据均值编码向量信息确定拍摄预览图像中存在频闪条带的情况下，将均值编码向量信息与对拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和/或强度信息。
第三方面，本申请实施例提供了一种电子设备，该电子设备包括处理器、存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。
第四方面，本申请实施例提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤。
第五方面，本申请实施例提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法。
在本申请实施例中，能够通过对拍摄预览图像进行检测，根据拍摄预览图像确定其对应预设成像方向的像素均值向量，根据像素均值向量提取向量特征信息，得到均值编码向量信息，并根据均值编码向量信息判断是否存在频闪条带。由于频闪条带的产生与图像成像方向有关，因此本申请实施例基于对应成像方向的像素均值向量提取特征信息，能够针对每一帧预览图像得到更为准确的频闪条带识别结果。并且本申请实施例在判断存在频闪条带的情况下，将均值编码向量信息与拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和强度信息；本申请实施例将频闪条带信息的识别精确到位置和强度，利于基于这些信息去除频闪条带得到无banding现象的图像，继而提高电子设备拍摄的成片率。
附图说明
图1是本申请实施例提供的一种频闪条带信息识别方法的流程示意图；
图2是本申请一个具体示例中，将拍摄预览图像切割为多个子图像的示意图；
图3是本申请一个具体示例中，通过第二模型进行向量特征提取的示意图；
图4是本申请一个具体实施例中步骤S103的流程示意图；
图5是本申请一个具体示例中，通过第一模型进行图像特征提取和banding信息识别的示意图；
图6是本申请实施例提供的一种频闪条带信息识别装置的结构示意图；
图7是本申请实施例提供的一种电子设备的硬件结构示意图；
图8是本申请另一实施例提供电子设备的硬件结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”等所区分的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。
图像的banding，是指由于光源的光线按固定的频率闪烁，使得光线亮度不停变化导致成像后产生条带效应。一般的，室内有人造光源影响时，例如中国家用电标准是220V 50Hz，光强的波动是100Hz，周期为10ms，这种情况下如果电子设备(如相机或手机)使用例如电子卷帘式曝 光方式时，由于每一行曝光时间不同，获得的能量也可能不同；那么在不同的感光面接收到的光能量不一样，从而产生了图像上的频闪条纹。
在普通场景下，电子设备快门的速度较低，一般不会出现banding现象。在拍摄运动物体时，为了捕捉到清晰的图像，需要提高快门速度，但是当快门速度过高后，往往会出现频闪问题，即拍摄出的图像会出现频闪banding(即频闪条带)。
相关技术中为了解决图像banding问题，一种解决方式是将快门速度控制在一定的阈值范围内，这样较低的快门速度可以避免banding，但同时限制了成像的质量，导致拍摄额的图像出现部分模糊，影像成像质量低。还有一种解决方式是，通过硬件判断光源闪烁频率，从而控制快门速度，避免banding出现。但这种方式中硬件判断光源闪烁频率的准确性较低，并且无法适应不同频率的光源，甚至复杂光源，因此由于适用性差导致不能在成像时完全避免banding的出现。
相关技术中由于很难直接从硬件控制方面避免出现banding现象，因此还有一些方案中会通过电子设备自动确定的反冲带值判别拍摄是否发生banding。但由于光源的复杂性，自动确定的反冲带值适应性较低，因此通过这种方式判别拍摄是否产生banding的准确性也较低。
针对上述相关技术中存在的至少一个问题，本申请实施例提供了一种频闪条带信息识别方法、装置、电子设备和可读存储介质。
需要说明的是，本申请实施例中的频闪条带信息识别方法的步骤可以通过具有拍摄功能的电子设备执行，该电子设备可以是移动电子设备，也可以为非移动电子设备。示例性的，具有拍摄功能的移动电子设备可以为数码相机(Digital Still Camera，DSC，简称：Digital Camera，DC)、手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本或 者个人数字助理(personal digital assistant，PDA)等，具有拍摄功能的非移动电子设备可以为个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
下面结合附图，通过具体的实施例及其应用场景对本申请实施例提供的频闪条带信息识别方法进行详细地说明。
图1示出的是本申请实施例提供的频闪条带信息识别方法的流程示意图。如图1所示，方法可以包括步骤S101～S104：
S101：获取拍摄预览图像。
拍摄预览图像可以是用户通过电子设备拍摄时的预览图像。
S102：根据拍摄预览图像，确定其对应预设成像方向的像素均值向量。
预设成像方向可以为电子设备拍摄时快门的图像传感器(sensor)的扫描方向。由于每个电子设备的快门扫描方向固定，如从上到下扫描以逐行曝光像素成像，所以每帧拍摄图像的成像方向相同。
本申请的发明人发现，由于频闪banding与电子设备的图像传感器扫描方向有关，因此在成像时，图像的banding只会出现带状的情况，即使受复杂光源影像，banding的带状性依旧存在。
基于上述发现，本步骤中，对拍摄预览图像进行对应其预设成像方向的数据处理，计算得到对应其预设成像方向上的像素均值向量，以通过后续对像素均值向量的特征提取处理，进而快速得到是否在banding的判断结果。
S103：根据像素均值向量提取向量特征信息，得到均值编码向量信息。
对像素均值向量进行特征提取，提取出该对应成像方向的像素均值向量的向量特征，得到均值编码向量信息。
由于像素均值向量是按照成像方向计算得到的，而拍摄预览图像出现的banding具有带状性的特征，所以进行向量特征信息提取后，可以一定程度上去掉无关特征的影响，得到相关性较高的向量特征信息，用于识别图像是否产生banding时，可以更好的捕捉对应每帧图像中的banding带状信息，提高对图像中banding识别的准确性和效率。
S104.在根据均值编码向量信息确定拍摄预览图像中存在频闪条带的情况下，将均值编码向量信息与对拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和/或强度信息。
如果根据均值编码向量信息进行判断后，确定拍摄预览图像中存在频闪条带，则将均值编码向量信息与对拍摄预览图像提取的图像特征信息融合，通过像素均值向量特征与图像特征信息融合，通过信息特征更为丰富的融合数据，识别出频闪条带的位置和/或强度信息。
在本申请实施例中，能够通过对拍摄预览图像进行检测，根据拍摄预览图像确定其对应预设成像方向的像素均值向量，根据像素均值向量提取向量特征信息，得到均值编码向量信息，并根据均值编码向量信息判断是否存在频闪条带。由于频闪条带的产生与图像成像方向有关，因此本申请实施例基于对应成像方向的像素均值向量提取特征信息，能够针对每一帧预览图像得到更为准确的频闪条带识别结果。并且本申请实施例在判断存在频闪条带的情况下，将均值编码向量信息与拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和强度信息；本申请实施例将频闪条带信息的识别精确到位置和强度，利于基于这些信息去除频闪条带得到无banding现象的图像，继而提高电子设备拍摄的成片率。
为提高对拍摄预览图像频闪条带信息识别的准确性，本申请实施例通过步骤S101得到拍摄预览图像之后，在通过步骤S102进行对拍摄预览图像的数据处理时，可以包括以下步骤S201～S204：
S201:将拍摄预览图像灰度化。
拍摄预览图像灰度化，即将图像每个像素点的RGB(R：Red，G：Green，B：Blue)三个分量统一成同一个值，简化图像的处理难度。
S202：获取拍摄预览图像对应其预设成像方向的第一像素均值向量。
拍摄预览图像对应其预设成像方向统计图像中的像素均值，得到第一像素均值向量。
S203：获取子图像对应预设成像方向的第二像素均值向量，子图像为多个，多个子图像为按照预设成像方向对拍摄预览图像分割得到。
由于光源的闪烁特性，一般情况下光源呈现周期性的亮度变化，这种亮度变化可能会一定程度的反映在图像产生的banding中，并且为了降低图像本身因素(如图像背景)对后续检测结果造成的影响，故而本步骤中，将拍摄预览图像按照预设成像方向进行切割，得到多个子图像。如图2所示，将拍摄预览图像201按照从上到下的扫描方向，切割为5个子图像202。
示例性的，为降低计算复杂度，切割时各子图像的尺寸大小相等，其中尺寸大小的单位为像素。在其他示例中，拍摄预览图像的切割数量可以不限，且切割的子图像大小也可以不相等。
对切割后的子图像分别按照预设成像方向计算像素均值，得到各个子图像在预设成像方向上的像素均值向量，即得到多个子图像的向量像素均值。
S204：将第一像素均值向量和第二像素均值向量进行归一化处理。
将第一像素均值向量和第二像素均值向量中的像素均值进行归一化处理，使得每个像素均值处于适合的预设阈值范围，可以加快后续处理过程的效率。
得到上述能够反映拍摄预览图像全局信息的第一像素均值向量和反映各局部信息的第二像素均值向量后，可以通过步骤S103进行向量特征提取。
为提高像素均值向量特征提取的效率和对向量特征分析的准确性，本申请实施例中通过预设的神经网络进行特征的提取和识别。
示例性的，该神经网络可以包括第一模型和第二模型，其中第一模型可以用于图像特征提取和banding具体信息的识别，第二模型用于像素均值向量特征提取和识别是否存在banding。
示例性的，参考图3所示，第二模型可以包括全连接层(Fully Connected Layer)和分类器，则通过步骤S103根据像素均值向量提取向量特征信息，得到均值编码向量信息中，具体可以包括步骤S301～S302：
S301：提取所述第一像素均值向量和所述第二像素均值向量的向量特征信息；
S302：将第一像素均值向量的向量特征信息和第二像素均值向量的向量特征信息连接，得到均值编码向量信息。
结合图2所示的例子进行说明。通过步骤S102对图2所示的拍摄预览图像切割处理后，可以得到一个包含全局像素信息的第一像素均值向量和五个包含局部像素信息的第二像素均值向量。再结合图3所示，将上述的六个像素均值向量301输入第二模型，在第二模型中经过全连接层302提取向量特征信息并整合，可以得到将每个像素均值向量的向量特征连接在一起的均值编码向量信息303。
本步骤中，得到均值编码向量信息的过程，是对于图像均值向量进行编码和信息统一的过程，可以更好的表示出图像的带状数据信息。
并且本步骤中，上述的六个像素均值向量经过全连接层302后，能够将六个向量数据映射为固定长度的特征向量(即均值编码向量)，该均值 编码向量信息中保留了拍摄预览图像中感兴趣的关键信息，忽略位置等无关信息，进而利于快速计算拍摄预览图像的特征，判定是否存在banding。
为提高判定图像是否存在banding的效率，示例性的，在上述步骤S302后，该频闪条带信息识别方法还包括：
S303.通过二分类预测算法对均值编码向量信息分类，根据分类结果确定拍摄预览图像中是否存在频闪条带。
示例性的，可以通过第二模型执行步骤S303，其中，第二模型中包括的分类器可以为通过二分类预测算法实现的二分类器，二分类预测算法可以为softmax函数(即归一化指数函数)。
参考图3，将步骤S302得到的均值编码向量信息303再接入全连接层304和softmax函数，其中，softmax函数可以用于对数据进行逻辑回归处理，得到二分类的类别输出结果。本示例中，将均值编码向量信息经过softmax函数处理，输出的分类结果包括两个概率值p1和p2，p1表示图像中存在banding的概率，p2表示图像中没有banding的概率，p1和p2之和为1。
若p1的概率值大于p2，则表示图像存在banding，执行进行步骤S104；反之，若p1的概率小于p2，则表示图像不存在banding，可以直接输出拍摄预览图像，得到最终的成像。这样可以快速准确判断出拍摄预览图像是否存在banding，进而可以避免输出具有频闪banding的图像，还可以避免电子设备在拍摄图像没有banding时进行后续不必要的banding去除操作，保障拍摄效率。
并且，如果识别出拍摄预览图像存在频闪banding，可以取消继续输出拍摄图像，也可以通过banding去除手段消除频闪banding后，再输出拍摄图像，从而提高电子设备的拍摄成片率。
为了在确定拍摄预览图像存在频闪banding的情况下，有针对性的消除图像中的频闪banding，本申请实施例中可以基于第一模型执行步骤S104来识别图像中频闪条带的位置和/或强度。具体的，如图4所示，步骤S104可以包括步骤S401～S403：
S401.在确定拍摄预览图像中存在频闪条带的情况下，通过第一模型提取拍摄预览图像和子图像的图像特征信息。
示例性的，第一模型可以为神经网络，神经网络包括第一网络层，第一网络层至少包括卷积层，还可以包括池化层。其中，卷积层包括多个依次连接的带状卷积核，每个带状卷积核包括两个非对称卷积块，多个带状卷积核大小不同。
步骤S401通过第一模型提取所述拍摄预览图像和所述子图像的图像特征信息中，具体可以包括S4011～S4015：
S4011：统一拍摄预览图像和对应的多个子图像的尺寸大小。
示例性的，对于输入的拍摄预览图像以及多个不同图像区域的子图像，统一图像尺寸大小，以降低后续神经网络对图像的处理复杂度。例如将一张拍摄预览图像和五张子图像的尺寸大小，统一为512×512像素大小。
S4012.将拍摄预览图像输入第一网络层，通过多个带状卷积核提取图像特征，得到拍摄预览图像的图像特征；以及
S4013.将所述子图像输入所述第一网络层，通过所述多个带状卷积核提取图像特征，得到所述子图像的图像特征；
S4014.将所述拍摄预览图像的图像特征和子图像的图像特征连接，得到图像特征编码信息，
S4015.将所述图像特征编码信息确定为所述图像特征信息。
将拍摄预览图像和多个子图像一一输入第一网络层，通过步骤S4012～S4012使每个图像均分别通过第一网络层中卷积层的多个带状卷积核处理，然后经过池化层，得到提取出对应图像特征的二维信息，然后将各个二维信息映射为固定长度的向量，得到图像特征编码信息。
以上述一张拍摄预览图像和五张对应子图像的例子进行说明，每个图像在第一网络层的处理过程包括：
参考图5，首先对输入第一网络层的图像501进行全局的卷积操作，卷积核可以采用标准方形卷积核，如3×3大小，全局卷积后得到第一矩阵(图中未标示)。第一矩阵再被带状卷积核进行二次卷积操作，例如该带状卷积核可以由一个7×1和1×7的两个非对称卷积块组成。由于带状的卷积核可以更好的学习到图像的带状banding信息，且两个非对称卷积块可以等效于相同尺寸的标准方形卷积核，不会增加时间开销，同时相对于标准方形卷积核可以增强特征提取精度，因此本实施例中，通过带状卷积核提取图像的特征。
第一矩阵分别经过第一带状卷积核502对应的两个非对称卷积块(如7×1和1×7的卷积块)，利用卷积的可加性，则可以得到与7×7方形核等效的第二矩阵。之后第二矩阵分别经过第二带状卷积核503对应的两个非对称卷积块(如5×1和1×5的卷积块)，可以得到与5×5方形核等效的第二矩阵。第二矩阵分别经过第三带状卷积核504对应的两个非对称卷积块(如3×1和1×3的卷积块)，可以得到与3×3方形核等效的第三矩阵，最后通过池化层505进行全局平均池化，提取出图像的二维信息(也即图像特征第四矩阵)。
对拍摄预览图像和各个子图像分别进行上述的图像特征提取，分别得到各个图像的图像特征，每个图像的图像特征为二维信息。将各个图像对应的二维信息映射为一个固定长度的特征向量，即图像特征编码信息 506，将该图像特征编码信息确定为图像特征信息进行后续S402～S403的步骤。
S402.将所述均值编码向量信息与所述图像特征信息融合，得到融合特征信息。
示例性的，第一模型除了包括上述进行图像特征提取的第一网络层外，还可以包括进行banding具体信息的识别第二网络层，该第二网络层至少包括全连接层和逻辑回归函数。
拍摄预览图像的不同区域对应的图像(包括全局图像和局部的子图像)，有不同的感受野信息，多区域的特征信息融合能够辅助神经网络进行更准确的判断。因此本示例中，利用包含全局图像特征和各局部图像特征的图像特性编码信息506进行banding信息的计算，并且利用图像的均值编码向量信息507提供一定的banding信息。
本步骤中，通过第二网络层的全连接层融合图像特征编码信息506和均值编码向量507，得到图像特征和像素均值特征的融合数据508。
S403.根据融合特征信息回归预测，得到频闪条带在所述拍摄预览图像中的位置和/或强度信息。
示例性的，回归预测可以是对融合数据508进行Logistic回归，得到W×1的向量509，其中，W×1的向量509中W表示对应图像的宽度，该向量为一维向量。全局图像和子图像中的宽边一致，经过对融合数据的回归预测后，W×1的向量中包含的数据对应不同的图像位置，每个位置对应的数据范围在0-1之间。每个图像区域位置的数值越接近1，说明banding的强度越强，反之，每个图像区域位置的数值越接近0，则banding的强度越弱。
通过上述步骤得到反映拍摄预览图像的banding位置和banding强度的向量后，本申请实施例的方法还可以包括步骤S105～S107：
S105.在得到频闪条带在所述拍摄预览图像中的位置信息的情况下，根据位置信息消除频闪条带；
S106.在得到频闪条带在拍摄预览图像中的强度信息的情况下，根据强度信息消除频闪条带；
S107.显示消除频闪条带的目标图像。
示例性的，通过神经网络学习图像中存在频闪条带的位置和强度，去除banding，得到消除频闪条带的目标图像，显示在电子设备屏幕上。或者可以通过确定的频闪条带的位置和强度调整电子设备的拍摄参数(如快门速度)重新获取不存在频闪banding的目标图像。
本实施例的频闪条带信息识别方法，可以在单张预览图像上快速准确的判断出图像是否存在banding现象，一方面可以避免电子设备在拍摄图像中没有banding时进行不必要的操作，另一方面可以根据banding检测结果，辅助去除图像中的banding，从而给用户呈现出无banding的成像效果。
在其他实施例中，还可以通过步骤S101～S104进行多帧图像的预测。示例性的，在步骤S101获取拍摄预览图像也后，步骤S201具体可以包括：
S501：获取拍摄预览图像对应预设成像方向的第一像素均值向量和第一像素均值向量插值。
计算拍摄预览图像对应预设成像方向的第一像素均值向量和该像素均值向量的插值，相当于计算当前帧拍摄预览图像像素均值，并得到该帧拍摄预览图像的下一帧插值图像的像素均值。
S502.获取子图像对应预设成像方向的第二像素均值向量和第二像素均值向量插值，子图像为多个，多个子图像为按照预设成像方向对拍摄预览图像分割得到。
对拍摄预览图像按照预设成像方向进行分割，得到多个子图像，每个子图像计算第二像素均值向量，并计算该第二像素均值向量的差值，相当于计算当前帧各子图像像素均值，并得到该帧子图像的下一帧插值子图像的像素均值。
通过各像素均值向量的差值，模拟当前帧拍摄预览图像的下一帧拍摄图像，进而形成连续帧图像。
S503.将第一像素均值向量、第一像素均值向量插值、第二像素均值向量以及第二像素均值向量插值确定为像素均值向量。
将第一像素均值向量、第一像素均值向量插值、第二像素均值向量以及第二像素均值向量插值确定为像素均值向量后，该像素均值向量为连续帧图像的向量，将该向量执行后续步骤S103～S104，可以识别出连续帧图像中是否存在banding以及banding的位置和/或强度，进而模拟出拍摄过程中banding的变化过程，可用于预测后续图像拍摄时的banding情况。
应理解，本实施例中，基于连续帧图像进行上述步骤S103和S104的处理，与上述基于单张拍摄预览图像进行步骤S103和S104的处理的过程相同，为避免重复，这里不再赘述。
需要说明的是，本申请实施例提供的频闪条带信息识别方法，执行主体可以为频闪条带信息识别装置，或者该频闪条带信息识别装置中的用于执行频闪条带信息识别方法的控制模块。本申请实施例中以频闪条带信息识别装置执行频闪条带信息识别方法为例，说明本申请实施例提供的频闪条带信息识别装置。
图6示出了本申请实施例提供的一种频闪条带信息识别装置，装置包括：
获取模块601，用于获取拍摄预览图像；
第一确定模块602，用于根据所述拍摄预览图像，确定其对应预设成像方向的像素均值向量；
第一提取模块603，用于根据所述像素均值向量提取向量特征信息，得到均值编码向量信息；
融合模块604，用于在根据所述均值编码向量信息确定所述拍摄预览图像中存在频闪条带的情况下，将所述均值编码向量信息与对所述拍摄预览图像提取的图像特征信息融合，得到所述频闪条带的位置和/或强度信息。
在本申请实施例中，能够通过对拍摄预览图像进行检测，根据拍摄预览图像确定其对应预设成像方向的像素均值向量，根据像素均值向量提取向量特征信息，得到均值编码向量信息，并根据均值编码向量信息判断是否存在频闪条带。由于频闪条带的产生与图像成像方向有关，因此本申请实施例基于对应成像方向的像素均值向量提取特征信息，能够针对每一帧预览图像得到更为准确的频闪条带识别结果。并且本申请实施例在判断存在频闪条带的情况下，将均值编码向量信息与拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和强度信息；本申请实施例将频闪条带信息的识别精确到位置和强度，利于基于这些信息去除频闪条带得到无banding现象的图像，继而提高电子设备拍摄的成片率。
可选的，为提高对拍摄预览图像频闪条带信息识别的准确性，第一确定模块602具体可以包括：
第一获取子模块6021，用于获取所述拍摄预览图像对应预设成像方向的所述第一像素均值向量；以及，
第二获取子模块6022，用于获取子图像对应预设成像方向的所述第二像素均值向量，所述子图像为多个，多个子图像为按照所述预设成像方向对所述拍摄预览图像分割得到。
为提高像素均值向量特征提取的效率和对向量特征分析的准确性，本申请实施例中通过预设的神经网络进行特征的提取和识别。
示例性的，该神经网络可以包括第一模型和第二模型，其中第一模型可以用于图像特征提取和banding具体信息的识别，第二模型用于像素均值向量特征提取和识别是否存在banding。
参考图3所示，第二模型可以包括全连接层(Fully Connected Layer)和分类器，第二模型中包括的分类器可以为通过二分类预测算法实现的二分类器，二分类预测算法可以为softmax函数(即归一化指数函数)。对应的，第一提取模块603可以包括：
第一提取子模块6031，用于提取所述第一像素均值向量和所述第二像素均值向量的向量特征信息；
第一连接子模块6032，用于将第一像素均值向量的向量特征信息和第二像素均值向量的向量特征信息连接，得到均值编码向量信息。
第一确定子模块6033，用于通过二分类预测算法对所述均值编码向量信息分类，根据分类结果确定所述拍摄预览图像中是否存在频闪条带。
本示例可以快速准确判断出拍摄预览图像是否存在banding，进而可以避免输出具有频闪banding的图像，还可以避免电子设备在拍摄图像没有banding时进行后续不必要的banding去除操作，保障拍摄效率。
并且，如果识别出图像存在频闪banding，可以取消继续输出拍摄图像，也可以通过banding去除手段消除频闪banding后，再输出拍摄图像，从而提高电子设备的拍摄成片率。
为了有针对性的消除图像中的频闪banding，本申请实施例中可以基于第一模型通过第一融合模块604来识别图像中频闪条带的位置和/或强度。
可选的，第一模型可以为神经网络，神经网络包括第一网络层，第一网络层至少包括卷积层，还可以包括池化层。其中，卷积层包括多个依次连接的带状卷积核，每个带状卷积核包括两个非对称卷积块，多个带状卷积核大小不同。
可选的，第一模型除了包括上述进行图像特征提取的第一网络层外，还可以包括进行banding具体信息的识别第二网络层，该第二网络层至少包括全连接层和逻辑回归函数。
对应的，第一融合模块604可以包括：
第二提取子模块6041，用于在确定拍摄预览图像中存在频闪条带的情况下，通过第一模型提取拍摄预览图像和子图像的图像特征信息；
融合子模块6042，用于将均值编码向量信息与图像特征信息融合，得到融合特征信息；
预测子模块6043，用于根据融合特征信息回归预测，得到频闪条带在所述拍摄预览图像中的位置和/或强度信息。
可选的，装置还可以包括：
消除模块605，用于在得到所述频闪条带在所述拍摄预览图像中的位置信息的情况下，根据所述位置信息消除所述频闪条带；在得到所述频闪条带在所述拍摄预览图像中的强度信息的情况下，根据所述强度信息消除所述频闪条带；
显示模块606，用于显示消除所述频闪条带的目标图像。
本实施例的频闪条带信息识别装置，可以在单张预览图像上快速准确的判断出图像是否存在banding现象，一方面可以避免电子设备在拍摄图像中没有banding时进行不必要的操作，另一方面可以根据banding检测结果，辅助去除图像中的banding，从而给用户呈现出无banding的成像效果。
本申请实施例中的频闪条带信息识别装置可以是装置，也可以是终端中的部件、集成电路、或芯片。该装置可以是移动电子设备，也可以为非移动电子设备。示例性的，移动电子设备可以为数码相机、手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本或者个人数字助理(personal digital assistant，PDA)等，非移动电子设备可以为个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
本申请实施例中的频闪条带信息识别装置可以为具有操作系统的装置。该操作系统可以为安卓(Android)操作系统，可以为ios操作系统，还可以为其他可能的操作系统，本申请实施例不作具体限定。
本申请实施例提供的频闪条带信息识别装置能够实现图1至图5的方法实施例实现的各个过程，为避免重复，这里不再赘述。
可选的，如图7所示，本申请实施例还提供一种电子设备700，包括处理器701，存储器702，存储在存储器702上并可在所述处理器701上运行的程序或指令，该程序或指令被处理器701执行时实现上述频闪条带信息识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，本申请实施例中的电子设备包括上述所述的移动电子设备和非移动电子设备。
图8为实现本申请实施例的一种电子设备的硬件结构示意图。
该电子设备800包括但不限于：射频单元801、网络模块802、音频输出单元803、输入单元804、传感器805、显示单元806、用户输入单元807、接口单元808、存储器809、以及处理器810等部件。
本领域技术人员可以理解，电子设备800还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器810逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。图8中示出的电子设备结构并不构成对电子设备的限定，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
其中，输入单元804，用于获取拍摄预览图像；
处理器810，用于根据拍摄预览图像，确定其对应预设成像方向的像素均值向量；以及
根据像素均值向量提取向量特征信息，得到均值编码向量信息；
在根据均值编码向量信息确定拍摄预览图像中存在频闪条带的情况下，将均值编码向量信息与对拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和/或强度信息。
本实施例的电子设备，能够通过对拍摄预览图像进行检测，根据拍摄预览图像确定其对应预设成像方向的像素均值向量，根据像素均值向量提取向量特征信息，得到均值编码向量信息，并根据均值编码向量信息判断是否存在频闪条带。由于频闪条带的产生与图像成像方向有关，因此本申请实施例基于对应成像方向的像素均值向量提取特征信息，能够针对每一帧预览图像得到更为准确的频闪条带识别结果。并且本申请实施例在判断存在频闪条带的情况下，将均值编码向量信息与拍摄预览图像提取的图像特征信息融合，得到频闪条带的位置和强度信息；本申请实施例将频闪条带信息的识别精确到位置和强度，利于基于这些信息去除频闪条带得到无banding现象的图像，继而提高电子设备拍摄的成片率。
应理解的是，本申请实施例中，输入单元804可以包括图形处理器(Graphics Processing Unit，GPU)8041和麦克风8042，图形处理器8041 对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元806可包括显示面板8061，可以采用液晶显示器、有机发光二极管等形式来配置显示面板8061。用户输入单元807包括触控面板8071以及其他输入设备8072中的至少一种。触控面板8071，也称为触摸屏。触控面板8071可包括触摸检测装置和触摸控制器两个部分。其他输入设备8072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。存储器809可用于存储软件程序以及各种数据，存储器809可主要包括存储程序或指令的第一存储区和存储数据的第二存储区，其中，第一存储区可存储操作系统、至少一个功能所需的应用程序或指令(比如声音播放功能、图像播放功能等)等。此外，存储器809可以包括易失性存储器或非易失性存储器，或者，存储器809可以包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请实施例中的存储器809包括但不限于这些和任意其它适合类型的存储器。处理器810可包括一个或多个处理单元；可选的，处理器110集成应用处理器和调制解调处理器，其中，应用处理器 主要处理操作系统、用户界面和应用程序等的操作，调制解调处理器主要处理无线通信信号，如基带处理器。可以理解的是，上述调制解调处理器也可以不集成到处理器810中。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述频闪条带信息识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的电子设备中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器ROM、随机存取存储器RAM、磁碟或者光盘等。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现上述频闪条带信息识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片、系统芯片、芯片系统或片上系统芯片等。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方 法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (15)

  1. 一种频闪条带信息识别方法，包括：
    获取拍摄预览图像；
    根据所述拍摄预览图像，确定其对应预设成像方向的像素均值向量；
    根据所述像素均值向量提取向量特征信息，得到均值编码向量信息；
    在根据所述均值编码向量信息确定所述拍摄预览图像中存在频闪条带的情况下，将所述均值编码向量信息与对所述拍摄预览图像提取的图像特征信息融合，得到所述频闪条带的位置和/或强度信息。
  2. 根据权利要求1所述的方法，其中，所述像素均值向量包括第一像素均值向量和第二像素均值向量，所述根据所述拍摄预览图像，确定其对应预设成像方向的像素均值向量，包括：
    获取所述拍摄预览图像对应预设成像方向的所述第一像素均值向量；以及，
    获取子图像对应预设成像方向的所述第二像素均值向量，所述子图像为多个，多个子图像为按照所述预设成像方向对所述拍摄预览图像分割得到。
  3. 根据权利要求1所述的方法，其中，所述根据所述拍摄预览图像，确定其对应预设成像方向的像素均值向量，包括：
    获取所述拍摄预览图像对应预设成像方向的第一像素均值向量和第一像素均值向量插值；
    获取子图像对应预设成像方向的第二像素均值向量和第二像素均值向量插值，所述子图像为多个，多个子图像为按照所述预设成像方向对所述拍摄预览图像分割得到；
    将所述第一像素均值向量、第一像素均值向量插值、第二像素均值向量以及第二像素均值向量插值确定为所述像素均值向量。
  4. 根据权利要求2或3所述的方法，其中，所述根据所述像素均值向量提取向量特征信息，得到均值编码向量信息，包括：
    提取所述第一像素均值向量和所述第二像素均值向量的向量特征信息；
    将第一像素均值向量的向量特征信息和第二像素均值向量的向量特征信息连接，得到均值编码向量信息。
  5. 根据权利要求1所述的方法，其中，在所述根据所述像素均值向量提取向量特征信息，得到均值编码向量信息之后，所述方法还包括：
    通过二分类预测算法对所述均值编码向量信息分类，根据分类结果确定所述拍摄预览图像中是否存在频闪条带。
  6. 根据权利要求2或3所述的方法，其中，所述在根据所述均值编码向量信息确定所述拍摄预览图像中存在频闪条带的情况下，将所述均值编码向量信息与对所述拍摄预览图像提取的图像特征信息融合，得到所述频闪条带的位置和/或强度信息，包括：
    在确定所述拍摄预览图像中存在频闪条带的情况下，通过第一模型提取所述拍摄预览图像和所述子图像的图像特征信息；
    将所述均值编码向量信息与所述图像特征信息融合，得到融合特征信息；
    根据所述融合特征信息回归预测，得到所述频闪条带在所述拍摄预览图像中的位置和/或强度信息。
  7. 根据权利要求6所述的方法，其中，所述第一模型为神经网络，所述神经网络包括第一网络层，第一网络层至少包括卷积层，所述卷积层包 括多个依次连接的带状卷积核，每个带状卷积核包括两个非对称卷积块，多个带状卷积核大小不同；
    所述通过第一模型提取所述拍摄预览图像和所述子图像的图像特征信息，包括：
    将所述拍摄预览图像输入所述第一网络层，通过所述多个带状卷积核提取图像特征，得到所述拍摄预览图像的图像特征；
    将所述子图像输入所述第一网络层，通过所述多个带状卷积核提取图像特征，得到所述子图像的图像特征；
    将所述拍摄预览图像的图像特征和子图像的图像特征连接，得到图像特征编码信息，
    将所述图像特征编码信息确定为所述图像特征信息。
  8. 根据权利要求1所述的方法，其中，在所述得到所述频闪条带在所述拍摄预览图像中的位置和/或强度信息之后，所述方法还包括：
    在得到所述频闪条带在所述拍摄预览图像中的位置信息的情况下，根据所述位置信息消除所述频闪条带；
    在得到所述频闪条带在所述拍摄预览图像中的强度信息的情况下，根据所述强度信息消除所述频闪条带；
    显示消除所述频闪条带的目标图像。
  9. 一种频闪条带信息识别装置，包括：
    获取模块，获取拍摄预览图像；
    第一确定模块，根据所述拍摄预览图像，确定其对应预设成像方向的像素均值向量；
    第一提取模块，用于根据所述像素均值向量提取向量特征信息，得到均值编码向量信息；
    融合模块，用于在根据所述均值编码向量信息确定所述拍摄预览图像中存在频闪条带的情况下，将所述均值编码向量信息与对所述拍摄预览图像提取的图像特征信息融合，得到所述频闪条带的位置和/或强度信息。
  10. 根据权利要求9所述的装置，其中，所述像素均值向量包括第一像素均值向量和第二像素均值向量，所述第一确定模块具体包括：
    第一获取子模块，用于获取所述拍摄预览图像对应预设成像方向的所述第一像素均值向量；以及，
    第二获取子模块，用于获取子图像对应预设成像方向的所述第二像素均值向量，所述子图像为多个，多个子图像为按照所述预设成像方向对所述拍摄预览图像分割得到。
  11. 一种电子设备，包括处理器，存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1至8任一项所述的频闪条带信息识别方法的步骤。
  12. 一种电子设备，包括所述电子设备被配置用于执行如权利要求1至8任一项所述的频闪条带信息识别方法的步骤。
  13. 一种可读存储介质，包括所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至8任一项所述的频闪条带信息识别方法的步骤。
  14. 一种芯片，包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1至8任一项所述的频闪条带信息识别方法的步骤。
  15. 一种计算机程序产品，包括所述计算机程序包含用于执行权利要求1-8任一项所述的频闪条带信息识别方法。
PCT/CN2022/113178 2021-08-25 2022-08-18 频闪条带信息识别方法、装置和电子设备 WO2023025010A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110982554.6 2021-08-25
CN202110982554.6A CN113688820A (zh) 2021-08-25 2021-08-25 频闪条带信息识别方法、装置和电子设备

Publications (1)

Publication Number Publication Date
WO2023025010A1 true WO2023025010A1 (zh) 2023-03-02

Family

ID=78582582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/113178 WO2023025010A1 (zh) 2021-08-25 2022-08-18 频闪条带信息识别方法、装置和电子设备

Country Status (2)

Country Link
CN (1) CN113688820A (zh)
WO (1) WO2023025010A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113688820A (zh) * 2021-08-25 2021-11-23 维沃移动通信有限公司 频闪条带信息识别方法、装置和电子设备
CN116055894B (zh) * 2023-01-28 2023-08-15 荣耀终端有限公司 基于神经网络的图像去频闪方法和装置
CN117692786A (zh) * 2024-02-01 2024-03-12 荣耀终端有限公司 一种拍摄方法、电子设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150022686A1 (en) * 2013-07-19 2015-01-22 Fujitsu Limited Flicker detection method and flicker detection apparatus
CN104378532A (zh) * 2014-11-28 2015-02-25 广东中星电子有限公司 一种抑制图像闪烁条纹的方法
CN105635594A (zh) * 2014-11-05 2016-06-01 富士通株式会社 闪烁校正方法和闪烁校正设备
JP2020113919A (ja) * 2019-01-15 2020-07-27 日本放送協会 映像処理装置及びプログラム
CN111741290A (zh) * 2020-04-24 2020-10-02 展讯通信（上海）有限公司 图像频闪检测方法及装置、存储介质、终端
CN113688820A (zh) * 2021-08-25 2021-11-23 维沃移动通信有限公司 频闪条带信息识别方法、装置和电子设备

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7715597B2 (en) * 2004-12-29 2010-05-11 Fotonation Ireland Limited Method and component for image recognition
CN104301624B (zh) * 2014-10-30 2018-04-03 青岛海信移动通信技术股份有限公司 一种图像拍摄亮度控制方法及装置
CN104378628B (zh) * 2014-11-28 2018-04-24 广东中星电子有限公司 一种检测图像闪烁条纹的方法
CN107171924A (zh) * 2016-03-07 2017-09-15 中国电信股份有限公司 基于频闪光实现的信息交互方法和系统以及相关设备
CN108416323B (zh) * 2018-03-27 2023-06-30 百度在线网络技术（北京）有限公司 用于识别人脸的方法和装置
CN109120863B (zh) * 2018-10-23 2021-01-01 Oppo广东移动通信有限公司 拍摄方法、装置、存储介质及移动终端
CN110855901B (zh) * 2019-11-28 2021-06-18 维沃移动通信有限公司 摄像头的曝光时间控制方法及电子设备
CN111510709B (zh) * 2020-04-24 2022-02-01 展讯通信（上海）有限公司 图像频闪检测方法及装置、存储介质、终端
CN112001248B (zh) * 2020-07-20 2024-03-01 北京百度网讯科技有限公司 主动交互的方法、装置、电子设备和可读存储介质
CN112788338B (zh) * 2020-12-31 2022-08-26 展讯通信(天津)有限公司 图像压缩及解压缩方法、设备、装置和存储介质
CN112887619A (zh) * 2021-01-28 2021-06-01 维沃移动通信有限公司 拍摄方法、装置及电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150022686A1 (en) * 2013-07-19 2015-01-22 Fujitsu Limited Flicker detection method and flicker detection apparatus
CN105635594A (zh) * 2014-11-05 2016-06-01 富士通株式会社 闪烁校正方法和闪烁校正设备
CN104378532A (zh) * 2014-11-28 2015-02-25 广东中星电子有限公司 一种抑制图像闪烁条纹的方法
JP2020113919A (ja) * 2019-01-15 2020-07-27 日本放送協会 映像処理装置及びプログラム
CN111741290A (zh) * 2020-04-24 2020-10-02 展讯通信（上海）有限公司 图像频闪检测方法及装置、存储介质、终端
CN113688820A (zh) * 2021-08-25 2021-11-23 维沃移动通信有限公司 频闪条带信息识别方法、装置和电子设备

Also Published As

Publication number Publication date
CN113688820A (zh) 2021-11-23

Similar Documents

Publication Publication Date Title
WO2023025010A1 (zh) 频闪条带信息识别方法、装置和电子设备
EP3611915B1 (en) Method and apparatus for image processing
AU2017261537B2 (en) Automated selection of keeper images from a burst photo captured set
TWI391876B (zh) 利用多重模組混合圖形切割之前景偵測方法、系統以及電腦程式產品
KR101303877B1 (ko) 얼굴 검출과 피부 영역 검출을 적용하여 피부의 선호색변환을 수행하는 방법 및 장치
Karaman et al. Comparison of static background segmentation methods
CN107635101A (zh) 拍摄方法、装置、存储介质和电子设备
JP2005316973A (ja) 赤目検出装置および方法並びにプログラム
CN104182721A (zh) 提升人脸识别率的图像处理系统及图像处理方法
CN101983507A (zh) 自动红眼检测
CN109948525A (zh) 拍照处理方法、装置、移动终端以及存储介质
CN110942456B (zh) 篡改图像检测方法、装置、设备及存储介质
CN112887610A (zh) 拍摄方法、装置、电子设备及存储介质
CN113947613B (zh) 目标区域检测方法、装置、设备及存储介质
US20130308829A1 (en) Still image extraction apparatus
JP2009123150A (ja) 対象物検出装置及び方法、対象物検出システム、プログラム
CN116503414B (zh) 屏幕缺陷检测方法、装置、计算机设备和存储介质
CN112887615A (zh) 拍摄的方法及装置
CN111428740A (zh) 网络翻拍照片的检测方法、装置、计算机设备及存储介质
CN111797694A (zh) 一种车牌检测方法及装置
CN112752110B (zh) 视频呈现方法及装置、计算设备、存储介质
CN114821441A (zh) 联合ads-b信息的基于深度学习的机场场面运动目标识别方法
CN113989387A (zh) 相机拍摄参数调整方法、装置及电子设备
CN111866383A (zh) 一种图像处理方法、终端及存储介质
Rahman et al. Real-time face-based auto-focus for digital still and cell-phone cameras

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE