CN113691724A - HDR scene detection method and device, terminal and readable storage medium - Google Patents

HDR scene detection method and device, terminal and readable storage medium Download PDF

Info

Publication number
CN113691724A
CN113691724A CN202110974142.8A CN202110974142A CN113691724A CN 113691724 A CN113691724 A CN 113691724A CN 202110974142 A CN202110974142 A CN 202110974142A CN 113691724 A CN113691724 A CN 113691724A
Authority
CN
China
Prior art keywords
pixels
preview image
feature vector
image
scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110974142.8A
Other languages
Chinese (zh)
Other versions
CN113691724B (en
Inventor
邹涵江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202110974142.8A priority Critical patent/CN113691724B/en
Publication of CN113691724A publication Critical patent/CN113691724A/en
Application granted granted Critical
Publication of CN113691724B publication Critical patent/CN113691724B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a High Dynamic Range (HDR) scene detection method and device, a terminal and a storage medium. The HDR scene detection method comprises the following steps: acquiring a preview image of a current scene; extracting multi-dimensional features in the preview image to construct an image feature vector; and determining the dynamic range of the current scene according to a preset Support Vector Machine (SVM) model and the image feature Vector. In the HDR scene detection method and device, the terminal and the storage medium, the preview image of the current scene is subjected to multi-dimensional feature extraction analysis to construct the image feature vector, and the dynamic range of the current scene is determined according to the SVM model and the image feature vector, so that the dynamic range of the current scene can be determined more reasonably and accurately, the HDR photographing mode can be triggered correctly during photographing, the photographing filming rate is improved, and the user experience is improved.

Description

HDR scene detection method and device, terminal and readable storage medium
Technical Field
The present application relates to the field of electronic technologies, and in particular, to a High Dynamic Range (HDR) scene detection method, an HDR scene detection apparatus, a terminal, and a non-volatile computer-readable storage medium.
Background
The "dynamic range" is used to describe the light amount intensity distribution range from the darkest shaded portion to the brightest highlight portion in the screen. In photographing/photography, there are two concepts of "dynamic range of scene" and "dynamic range of camera", where "dynamic range of scene" refers to the range or ratio of maximum brightness and minimum brightness in a photographed scene, that is, the difference between the brightest area and the darkest area in a picture; and "dynamic range of the camera" refers to the range of brightness variation that is acceptable for the light sensing element. A High Dynamic Range (HDR) scene, i.e., a scene in which the Dynamic Range of the scene is larger than the Dynamic Range of the camera, has too bright or too dark regions beyond the Range that can be recorded by the photosensitive elements, and shows that a completely white (highlight overflow becomes completely white) or completely black (shadow region becomes completely black) region appears in the shot picture, and the image quality is greatly reduced due to lack of details of bright or dark portions. Currently, for an HDR scene, a part of area details can be retained in a single image by multiple exposures of the same scene, and an HDR algorithm processes and synthesizes a high dynamic range image to improve an imaging effect. It is therefore first to be solved whether the shot scene is an HDR scene. If the scene is not an HDR scene, multiple exposures and HDR algorithm processing on the same scene may increase computational cost and time; if the scene is an HDR scene but not correctly discriminated, the shot imaging information is lost. It is important to detect HDR scenes correctly.
Disclosure of Invention
The embodiment of the application provides an HDR scene detection method, an HDR scene detection device, a terminal and a non-volatile computer readable storage medium.
The image processing method of the embodiment of the application comprises the following steps: acquiring a preview image of a current scene; extracting multi-dimensional features in the preview image to construct an image feature vector; and determining the dynamic range of the current scene according to a preset Support Vector Machine (SVM) model and the image feature Vector.
The HDR scene detection device comprises an acquisition module, a construction module and a determination module. The acquisition module is configured to: and acquiring a preview image of the current scene. The construction module is configured to: and extracting the multi-dimensional features in the preview image to construct an image feature vector. The determination module is to: and determining the dynamic range of the current scene according to a preset SVM model and the image feature vector.
The terminal of the embodiments of the present application includes one or more processors, memory, and one or more programs. Wherein one or more of the programs are stored in the memory and executed by one or more of the processors, the programs including instructions for performing the HDR scene detection method of embodiments of the present application. The HDR scene detection method comprises the following steps: acquiring a preview image of a current scene; extracting multi-dimensional features in the preview image to construct an image feature vector; and determining the dynamic range of the current scene according to a preset SVM model and the image feature vector.
A non-transitory computer-readable storage medium of an embodiment of the present application contains a computer program that, when executed by one or more processors, causes the processors to perform the following HDR scene detection method: acquiring a preview image of a current scene; extracting multi-dimensional features in the preview image to construct an image feature vector; and determining the dynamic range of the current scene according to a preset SVM model and the image feature vector.
In the HDR scene detection method, the HDR scene detection device, the terminal and the nonvolatile computer readable storage medium, the preview image of the current scene is subjected to multi-dimensional feature extraction analysis to construct the image feature vector, and then the dynamic range of the current scene is determined according to the SVM model and the image feature vector, so that the dynamic range of the current scene can be more reasonably and accurately determined, the HDR photographing mode can be correctly triggered during photographing, the photographing filming rate is improved, and the user experience is improved.
Additional aspects and advantages of embodiments of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow diagram of an HDR scene detection method according to some embodiments of the present application;
fig. 2 is a schematic structural diagram of an HDR scene detection apparatus according to some embodiments of the present application;
FIG. 3 is a schematic block diagram of a terminal according to some embodiments of the present application;
fig. 4 to 9 are schematic flow diagrams of HDR scene detection methods according to some embodiments of the present application;
FIG. 10 is a scene schematic diagram of an HDR scene detection method according to some embodiments of the present application;
fig. 11 to 12 are schematic flow diagrams of HDR scene detection methods according to some embodiments of the present application;
FIG. 13 is a scene schematic diagram of a preview image with a salient region in an HDR scene detection method according to some embodiments of the present application;
fig. 14-15 are schematic flow diagrams of HDR scene detection methods of some embodiments of the present application;
FIG. 16 is a schematic diagram of a connection between a non-volatile computer readable storage medium and a processor according to some embodiments of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of explaining the embodiments of the present application, and are not to be construed as limiting the embodiments of the present application.
Referring to fig. 1 to 4, an HDR scene detection method according to an embodiment of the present application includes:
01: acquiring a preview image of a current scene;
04: extracting multi-dimensional features in the preview image to construct an image feature vector; and;
05: and determining the dynamic range of the current scene according to a preset Support Vector Machine (SVM) model and an image feature Vector.
Referring to fig. 2, the present embodiment further provides an HDR scene detection apparatus 10, where the HDR scene detection apparatus 10 includes an obtaining module 11, a constructing module 13, and a determining module 15. The HDR scene detection method according to the embodiment of the present application is applicable to the HDR scene detection apparatus 10. Wherein the obtaining module 11 is configured to execute the method in 01. The construction module 13 is used to execute the method in 04. The determination module 15 is used to execute the method in 05. Namely, the obtaining module 11 is configured to: a preview image P0 of the current scene is acquired. The construction module 13 is used for: and extracting the multi-dimensional features in the preview image to construct an image feature vector. The determination module 15 is configured to: and determining the dynamic range of the current scene according to the preset SVM model and the image feature vector.
Referring to fig. 3, the present embodiment further provides a terminal 100, where the terminal 100 includes one or more processors 30, a memory 50, and one or more programs. Wherein one or more programs are stored in the memory 50 and executed by the one or more processors 30, the programs including instructions for performing the HDR scene detection method of embodiments of the present application. That is, when one or more processors 30 execute a program, the processors 30 may implement the methods in 01, 04, and 05. That is, the one or more processors 30 are operable to: acquiring a preview image of a current scene, extracting multi-dimensional features in the preview image, and constructing an image feature vector; and determining the dynamic range of the current scene according to the preset SVM model and the image feature vector.
Specifically, the terminal 100 may include, but is not limited to, a mobile phone, a notebook computer, a smart television, a tablet computer, a smart watch, a head display device, a drone, a digital camera, a digital camcorder, or a computer. The HDR scene detection apparatus 10 may be an integration of functional modules integrated in the terminal 100. The present application is described only by taking the terminal 100 as a mobile phone as an example, and the case where the terminal 100 is another type of device is similar to the mobile phone, and will not be described in detail.
The "dynamic range" is used to describe the light amount intensity distribution range from the darkest shaded portion to the brightest highlight portion in the screen. In photographing/photography, there are two concepts of "dynamic range of scene" and "dynamic range of camera", where "dynamic range of scene" refers to the range or ratio of maximum brightness and minimum brightness in a photographed scene, that is, the difference between the brightest area and the darkest area in a picture; and "dynamic range of the camera" refers to the range of brightness variation that is acceptable for the light sensing element. A High Dynamic Range (HDR) scene, i.e., a scene in which the Dynamic Range of the scene is larger than the Dynamic Range of the camera, has too bright or too dark regions beyond the Range that can be recorded by the photosensitive elements, and shows that a completely white (highlight overflow becomes completely white) or completely black (shadow region becomes completely black) region appears in the shot picture, and the image quality is greatly reduced due to lack of details of bright or dark portions. Currently, for an HDR scene, a part of area details can be retained in a single image by multiple exposures of the same scene, and an HDR algorithm processes and synthesizes a high dynamic range image to improve an imaging effect. It is therefore first to be solved whether the shot scene is an HDR scene. If the scene is not an HDR scene, multiple exposures and HDR algorithm processing on the same scene may increase computational cost and time; if the scene is an HDR scene but not correctly discriminated, the shot imaging information is lost. It is important to detect HDR scenes correctly.
In the HDR scene detection method, the HDR scene detection apparatus 10, the terminal 100, and the non-volatile computer-readable storage medium 200 of the present application, the dynamic range of the current scene can be determined more reasonably and accurately by performing multi-dimensional feature extraction analysis on the preview image of the current scene to construct an image feature vector, and then determining the dynamic range of the current scene according to the SVM model and the image feature vector, which is beneficial to correctly triggering the HDR photographing mode during photographing, improving the photographing filming rate, and improving user experience.
In method 01, the current scene is a scene in which the user takes a preview image, for example, various scenes (including buildings, people, scenery, etc.) from day to night. When the preview image of the current scene is acquired, the format of the preview image may be YUV format or RGB format, and the like, which is not limited.
In the method 04, since the HDR algorithm adopted in the HDR photographing mode can improve the imaging effect of the image to a certain extent from the aspects of brightness, detail, definition, color, transparency, and the like, when detecting the dynamic range of the current scene, the construction module 13 or the processor 30 may extract the multidimensional features in the preview image to construct the image feature vector, so as to analyze the dynamic range of the current scene more reasonably and accurately. The features may include the features of brightness, color naturalness, and salient regions of the preview image.
In the method 05, the determining module 13 or the processor 30 inputs the calculated image feature vector into a preset SVM model for discrimination, so that the dynamic range of the current scene can be more reasonably and accurately determined, and thus whether the current scene needs to be processed by an HDR algorithm is determined, the photo-taking filming rate is assisted to be improved, and the user experience is improved. Meanwhile, the image feature vector and the SVM model can reflect the high and low degree of the dynamic range of the current scene, and the reasonable algorithm and the exposure strategy are adapted favorably.
Referring to fig. 4, in some embodiments, the HDR scene detection method may further include:
02: acquiring shooting metadata parameters of a current scene;
03: and performing brightness correction on the preview image according to the shooting metadata parameters. Wherein, 04: extracting the multi-dimensional features in the preview image to construct an image feature vector comprises: 041: and extracting the multidimensional characteristics in the preview image after brightness correction to construct an image characteristic vector.
Referring to fig. 2, the obtaining module 11 is also used for executing the method in 02, and the constructing module 13 is also used for executing the methods in 03 and 041. That is, the obtaining module 11 is further configured to: and acquiring shooting metadata parameters of the current scene. The construction module 13 is also used for: and performing brightness correction on the preview image according to the shooting metadata parameters. The determining, by the construction module 13, the dynamic range of the current scene according to the preset SVM model and the image feature vector may include: and extracting the multidimensional characteristics in the preview image after brightness correction to construct an image characteristic vector.
Referring to FIG. 3, processor 30 is also used to execute the methods of 02, 03, and 041. That is, the processor 30 is further configured to: acquiring shooting metadata parameters of a current scene; the preview image P0 is subjected to brightness correction according to the shooting metadata parameters. Wherein the extracting, by the processor 30, the multi-dimensional features in the preview image to construct the image feature vector includes: 041: and extracting the multidimensional characteristics in the preview image after brightness correction to construct an image characteristic vector.
When the obtaining module 11 or the Processor 30 obtains a preview Image of a current scene, the preview Image is processed and influenced by each brightness module in an Image Signal Processor (ISP) of a high-pass platform, for example, when the obtaining module 11 or the Processor 30 obtains the preview Image, the camera effect debugging therein may adjust each module in the camera, for example, automatic exposure, automatic focusing, white balance, brightness and color adjustment, etc. are performed on the preview Image, and brightness information of the current scene cannot be truly reflected. It is therefore desirable to luminance correct the preview image to ensure that the determination module 15 or processor 30 can accurately determine whether the current scene is an HDR scene.
Referring to fig. 4 again, in the method 02, specifically, while the obtaining module 11 or the processor 30 obtains the preview image, the obtaining module obtains the shooting metadata parameters under the shooting condition of the current scene, where the shooting metadata parameters may include one or more of ISO (sensitivity), bright area brightness Gain (drcGain) in an Auto Exposure Control (AEC) module, Dark area brightness Gain (Dark Boost Gain, DBGain) in an AEC module with high pass, or face frame information. The drcGain parameter is a ratio calculated by the bright area information in the current scene in the high-pass AEC module, and the drcGain parameter is used in the ISP to enable the bright area of the preview image not to be excessively exposed and to be darkened to some extent. Similarly, the DBGain parameter is a ratio calculated by the dark area information in the current scene in the high-pass AEC module, and the ISP uses the DBGain parameter to brighten the dark area of the preview image.
In the method 03, the configuration module 13 or the processor 30 performs brightness correction on the preview image according to the shooting metadata parameters, specifically: the shooting metadata parameters are transmitted to the construction module 13 or the processor 30, and are reflected to the preview image, the real brightness information of the current scene is obtained through reverse calculation, and when the multi-dimensional features (brightness) in the preview image of the current scene are obtained, the accuracy of the image feature vectors obtained according to the multi-dimensional features of the preview image is ensured, so that the accuracy of scene detection is improved.
In the embodiment of the application, the obtaining module 11 or the processor 30 executes the method 01 and the method 02 first, the constructing module 13 or the processor 30 performs brightness correction on the preview image according to the shooting metadata parameters, and then executes the method 04 and the method 05 according to the corrected preview image, so that the real image feature information in the current scene is obtained, and the scene detection accuracy is improved.
Referring to fig. 5, in some embodiments, the shooting metadata parameters include a first brightness gain applied to a bright area of the preview image and a second brightness gain applied to a dark area of the preview image, 03: and performing brightness correction on the preview image according to the shooting metadata parameters, wherein the following method is executed by traversing all pixels of the preview image: can include the following steps:
031: when the current brightness value of the pixel is larger than a preset first brightness threshold value, taking the product of the current brightness value of the pixel and the first brightness gain as a correction brightness value of the pixel;
033: and when the current brightness value of the pixel is smaller than a preset second brightness threshold value, taking the ratio of the current brightness value of the pixel to the second brightness gain as the correction brightness value of the pixel.
Referring to fig. 2, the construction module 13 is also used for executing the methods 031 and 033. That is, the construction module 13 is also used for: and when the current brightness value of the pixel is larger than a preset first brightness threshold value, taking the product of the current brightness value of the pixel and the first brightness gain as the correction brightness value of the pixel. And when the current brightness value of the pixel is smaller than a preset second brightness threshold value, taking the ratio of the current brightness value of the pixel to the second brightness gain as the correction brightness value of the pixel.
Referring to fig. 3, the processor 30 is also configured to execute the methods 031 and 033. That is, the processor 30 is further configured to: and when the current brightness value of the pixel is larger than a preset first brightness threshold value, taking the product of the current brightness value of the pixel and the first brightness gain as the correction brightness value of the pixel. And when the current brightness value of the pixel is smaller than a preset second brightness threshold value, taking the ratio of the current brightness value of the pixel to the second brightness gain as the correction brightness value of the pixel.
In the embodiment of the present application, since the image feature vector includes luminance feature information, and the obtaining module 11 or the processor 30 uses drcGain and DBGain to process the preview image when obtaining the preview image P0, the first luminance gain may be a drcGain parameter obtained by calculating bright area information of the preview image in the high-pass AEC, and the second luminance gain may be a DBGain parameter obtained by calculating dark area information of the preview image in the high-pass AEC. When the construction module 13 or the processor 30 performs the brightness correction on the preview image, the brightness values of all the pixels in the preview image are subjected to the brightness correction according to the first brightness gain and the second brightness gain (the brightness effect of the inactive drcGain and DBGain is restored). The specific process is that the construction module 13 or the processor 30 executes the methods in 031 and 033 in a traversal manner on all pixels in the preview image, and obtains the corrected preview image.
It is assumed that a pixel having a luminance value greater than a first luminance threshold value is divided into bright-area pixels, a pixel having a luminance value less than a second luminance threshold value is divided into dark-area pixels, and a pixel having a luminance value greater than the second luminance threshold value and less than the first luminance threshold value does not need luminance correction. The setting of the first brightness threshold and the second brightness threshold may be set according to an actual scene, which is not limited in this respect. If the first luminance threshold value is Bright _ th, the second luminance threshold value is Dark _ th, the current luminance value of the pixel in the preview image is Y (x, Y), and Y (x, Y) represents the current luminance value of the pixel at (x, Y) in the preview image, the corrected luminance value of the pixel after correction is Y' (x, Y), and the corrected luminance value can be calculated by equation (1).
Figure BDA0003227025410000071
In one embodiment, the first brightness threshold value Bright _ th is 230, the second brightness threshold value Dark _ th is 50, and if the current brightness value 240 of the currently traversed pixel is greater than the first brightness threshold value Bright _ th (230), the corrected brightness value 240 drcGain of the pixel is obtained by the formula (1) through calculation. If the current brightness value of the currently traversed pixel is 10, the corrected brightness value of the pixel is 10/DBGain calculated by the formula (1) because the current brightness value (10) is smaller than the second brightness threshold value Dark _ th (50). If the current luminance value of the currently traversed pixel is 80, since the current luminance value (80) is greater than the second luminance threshold value Dark _ th (50) and less than the first luminance threshold value Bright _ th (250), no correction is required to be performed on the current luminance value of the pixel, and the luminance value of the pixel is still 80.
The construction module 13 or the processor 30 recovers the luminance effects of the inactive drcGain and DBGain through the formula (1) by reverse calculation, so that the luminance information of the preview image is more consistent with the luminance information in the current scene, the accuracy of the image feature vector constructed according to the multidimensional features is ensured, and the accuracy of the detection of the current scene is ensured.
Referring to fig. 6, in some embodiments, 04: extracting multi-dimensional features in the preview image to construct an image feature vector may include:
043: constructing a first sub-feature vector according to the brightness value of each pixel in the preview image;
045: constructing a second sub-feature vector according to the color naturalness of each pixel in the preview image;
047: constructing a third sub-feature vector according to the features of the salient region in the preview image; and
049: and constructing a feature vector according to the first sub-feature vector, the second sub-feature vector and the third sub-feature vector.
Referring to fig. 2, the configuration module 13 is also used to execute the methods in 043, 045, 047 and 049. That is, the construction module 13 is also used for: constructing a first sub-feature vector according to the brightness value of each pixel in the preview image; constructing a second sub-feature vector according to the color naturalness of each pixel in the preview image; constructing a third sub-feature vector according to the features of the salient region in the preview image; and constructing a feature vector according to the first sub-feature vector, the second sub-feature vector and the third sub-feature vector.
Referring to fig. 3, the processor 30 is further configured to perform the methods of 043, 045, 047 and 049. That is, the processor 30 is further configured to: constructing a first sub-feature vector according to the brightness value of each pixel in the preview image; constructing a second sub-feature vector according to the color naturalness of each pixel in the preview image; constructing a third sub-feature vector according to the features of the salient region in the preview image; and constructing a feature vector according to the first sub-feature vector, the second sub-feature vector and the third sub-feature vector.
Since the HDR algorithm process can improve the imaging effect to some extent in terms of brightness, detail, sharpness, color, transparency, etc., it is also necessary to analyze and consider from multi-dimensional information when determining whether to perform the HDR process (i.e., HDR detection). The method and the device calculate the characteristics of the preview image in the brightness, color naturalness and saliency areas, and jointly construct the image characteristic vector, so that the multi-dimensional combination analysis is carried out on the current scene according to the image characteristic vector.
Referring to fig. 7, in some embodiments, 043: constructing the first sub-feature vector according to the brightness value of each pixel in the preview image may include:
0431: based on a k-means clustering algorithm, dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels SlSecond set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlLuminance values of all pixels in (1);
0432: respectively calculating a first set of pixels ShIs compared with the area of all pixels N (I) in the preview imagehA second set of pixels SmIs compared with the area of all pixels n (i) in the preview imagemAnd a third set of pixels SlArea of and preview ofThird ratio R between areas of all pixels N (I) in the imagel；
0433: respectively calculating a first set of pixels ShOf all pixels in (1) the first information entropy EhA second set of pixels SmOf all pixels in (2)mA third set of pixels SlOf all pixels in (1)lAnd the entropy E of the full-image information of all pixels in the preview imageg；
0434: according to a first ratio RhA second ratio RmA third ratio RlFirst information entropy EhSecond information entropy EmThird information entropy ElAnd full-picture information entropy EgA first sub-feature vector is constructed.
Referring to fig. 2, the module 13 is also used to execute the methods in 0431, 0432, 0433 and 0434. That is, the construction module 13 is also used for: based on a k-means clustering algorithm, dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels SlSecond set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlThe luminance values of all pixels in (1). Respectively calculating a first set of pixels ShIs compared with the area of all pixels N (I) in the preview imagehA second set of pixels SmIs compared with the area of all pixels n (i) in the preview imagemAnd a third set of pixels SlIs compared with the area of all pixels n (i) in the preview imagel. Respectively calculating a first set of pixels ShOf all pixels in (1) the first information entropy EhA second set of pixels SmOf all pixels in (2)mA third set of pixels SlOf all pixels in (1)lAnd the entropy E of the full-image information of all pixels in the preview imageg. According to a first ratio RhA second ratio RmA third ratio RlFirst information entropy EhSecond information entropy EmThird information entropy ElAnd full-picture information entropy EgA first sub-feature vector is constructed.
Referring to fig. 3, the processor 30 is further configured to execute the methods in 0431, 0432, 0433 and 0434. That is, the processor 30 is further configured to: based on a k-means clustering algorithm, dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels SlSecond set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlThe luminance values of all pixels in (1). Respectively calculating a first set of pixels ShIs compared with the area of all pixels N (I) in the preview imagehA second set of pixels SmIs compared with the area of all pixels n (i) in the preview imagemAnd a third set of pixels SlIs compared with the area of all pixels n (i) in the preview imagel. Respectively calculating a first set of pixels ShOf all pixels in (1) the first information entropy EhA second set of pixels SmOf all pixels in (2)mA third set of pixels SlOf all pixels in (1)lAnd the entropy E of the full-image information of all pixels in the preview imageg. According to a first ratio RhA second ratio RmA third ratio RlFirst information entropy EhSecond information entropy EmThird information entropy ElAnd full-picture information entropy EgA first sub-feature vector is constructed.
In one embodiment, the construction module 13 or the processor 30 obtains the first sub-feature vector according to methods 0431, 0432, 0433 and 0434.
In the method 0431, the preview images are clustered according to brightness by using a k-means clustering algorithm on a brightness/gray level channel of the preview images. In the embodiments of the present application, the modules 13 orThe processor 30 is divided into three pixel sets according to a k-means clustering algorithm. For example, all pixels in the preview image are divided into a first set of pixels S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels Sl. For example, the first set of pixels ShA second set of pixels S for the bright areamIn the middle bright area, the third pixel set SlIs a dark area. Specifically, the configuration module 13 or the processor 30 may preset a bright-area clustering center C according to the brightness of the actual scenehAnd dark region clustering center Cl，ChA pixel brightness critical value for dividing a bright area and a middle bright area, ClIs a pixel brightness critical value for dividing a middle bright area and a dark area. All pixels in the preview image can be divided into three sets of pixels according to a k-means clustering algorithm. As shown in the following equation (2):
Figure BDA0003227025410000091
i (x, y) in formula (2) represents the luminance value of the preview image I at the (x, y) pixel point. Thus, after partitioning according to the k-means clustering algorithm, the second set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlThe luminance values of all pixels in (1).
In the method 0432, the area ratio of all pixels in the three pixel sets to all pixels n (i) of the entire preview image is calculated, respectively. For a first set of pixels ShIf the preview image includes a total of 8 × 8 (i.e., n (i) ═ 64) pixels, each pixel has an area size of 1 × 1, the first set S of pixels is assumed to have a smaller area size than the first set S of pixelshComprising 35 pixels, a second set S of pixelsmIncluding 18 pixels and a third set S of pixelslIncluding 11 pixels. Calculating the area ratio R of the three pixel setsh、RmAnd RlAs shown in equation (3):
Figure BDA0003227025410000092
where N () represents the sum of pixels, I represents the preview image, h represents the first set of pixels ShM denotes a second set of pixels SmL represents the third set of pixels SlThe total number of pixels in (a). E.g. N (S)h) Representing a first set of pixels ShAll pixels in (1), N (S)m) Representing a second set of pixels SmAll pixels in (1), N (S)l) Representing a third set of pixels SlN (i) represents all pixels in the preview image (i.e., the sum of the pixels in the three sets). The construction module 13 or the processor 30 may determine the dynamic range of the current scene according to the sizes of the different luminance regions by calculating the area ratio of the different luminance regions.
In the method 0433, the information entropy of the preview image can reflect the information content of the image, and therefore, the information entropy of the preview image in the three pixel sets and the information entropy of the full image are calculated, which is specifically shown in formula (4);
Ei═ p (x) log p (x) dx, i ∈ { h, m, l, g } formula (4)
Where E in formula (4) is the information entropy of the image, P is the probability density function, and P (x) represents the probability that the pixel brightness x appears in the preview image I. h. The specific meanings of m and l are the same as those of h, m and l in the formula (3), and g represents the sum of pixels in the preview image I. The first information entropy E obtained by calculationhSecond information entropy EmThird information entropy ElAnd full-picture information entropy EgIt can be used as a case of judging the dynamic range of the current scene. E.g. when the entropy takes the maximum value EmaxLog (256) is 8, indicating that the luminance of the region is completely uniformly distributed and that the region has no dynamics. After the four information entropies are obtained through calculation, the difference between the four information entropies can be compared to judge the dynamic range of the current scene.
In summary, the construction module 13 or the processor 30 is configured according to the first ratio RhA second ratio RmA third ratio RlFirst letterEntropy EhSecond information entropy EmThird information entropy ElAnd full-picture information entropy EgA first sub-feature vector is constructed. If the first sub-feature vector is denoted as B ═ Rh,Rm,Rl,Eh,Em,El,EgIs a 1 x 7 dimensional vector.
Referring to fig. 8, in some embodiments, 043: constructing the first sub-feature vector according to the brightness value of each pixel in the preview image, and may further include:
0431: based on a k-means clustering algorithm, dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels SlSecond set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlLuminance values of all pixels in (1);
0435: respectively calculating a first set of pixels ShOf all pixels in the imageh 2A second set of pixels SmSecond variance σ of luminance values of all pixels inm 2And a third set of pixels SlThird variance σ of luminance values of all pixels inl 2；
0436: respectively calculating a first set of pixels ShFirst mean value X of all pixels in (1)hA second set of pixels SmSecond mean value X of all pixels in (1)mA third set of pixels SlThird mean value X of all pixels in (1)lAnd the fourth mean value X of all pixels in the preview imageg；
0437: according to the first square difference sigmah 2The second variance σm 2Third difference sigmal 2First mean value XhSecond mean value XmThe third mean value XlAnd the fourth mean value XgA first sub-feature vector is constructed.
Please refer to fig. 2, the configuration module 13 is further used for executing0431. 0435, 0436 and 0437. That is, the construction module 13 is also used for: based on a k-means clustering algorithm, dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels SlSecond set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlThe luminance values of all pixels in (1). Respectively calculating a first set of pixels ShOf all pixels in the imageh 2A second set of pixels SmSecond variance σ of luminance values of all pixels inm 2And a third set of pixels SlThird variance σ of luminance values of all pixels inl 2. Respectively calculating a first set of pixels ShFirst mean value X of all pixels in (1)hA second set of pixels SmSecond mean value X of all pixels in (1)mA third set of pixels SlThird mean value X of all pixels in (1)lAnd the fourth mean value X of all pixels in the preview imageg. According to the first square difference sigmah 2The second variance σm 2Third difference sigmal 2First mean value XhSecond mean value XmThe third mean value XlAnd the fourth mean value XgA first sub-feature vector is constructed.
Referring to fig. 3, the processor 30 is further configured to execute the methods in 0431, 0435, 0436 and 0437. That is, the processor 30 is further configured to: based on a k-means clustering algorithm, dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixelhA second set of pixels SmAnd a third set of pixels SlSecond set of pixels SmIs smaller than the first set of pixels SmAnd is greater than the third set of pixels SlThe luminance values of all pixels in (1). Respectively calculating a first set of pixels ShOf all pixels in the imageh 2A second set of pixels SmSecond variance σ of luminance values of all pixels inm 2And a third set of pixels SlThird variance σ of luminance values of all pixels inl 2. Respectively calculating a first set of pixels ShFirst mean value X of all pixels in (1)hA second set of pixels SmSecond mean value X of all pixels in (1)mA third set of pixels SlThird mean value X of all pixels in (1)lAnd the fourth mean value X of all pixels in the preview imageg. According to the first square difference sigmah 2The second variance σm 2Third difference sigmal 2First mean value XhSecond mean value XmThe third mean value XlAnd the fourth mean value XgA first sub-feature vector is constructed.
In another embodiment, the construction module 13 or the processor 30 may obtain the first sub-feature vector according to methods 0431, 0435, 0436 and 0437.
Specifically, the dividing method in the method 0431 is the same as the implementation process of dividing the preview image based on the k-means clustering algorithm, and is not described herein again.
In method 0435, the construction module 13 or the processor 30 respectively counts the first brightness set ShFirst mean value X of all pixels based on brightnesshA second set of pixels SmBased on the second mean value X of the luminance of all the pixels in (2)mA third set of pixels SlAll pixels in (2) are based on the third mean value X of the luminancelAnd a fourth mean value X of all pixels in the preview image based on brightnessg. Specifically, the corresponding luminance mean value (first mean value X) can be obtained by calculating the gray level histogram in each pixel sethSecond mean value XmThe third mean value XlAnd the fourth mean value Xg) And then based on the mean value of the luminance of the three pixel sets (first mean value X)hSecond mean value XmAnd a third mean value Xl) The brightness variance (first variance σ) of each pixel set is calculatedh 2The second variance σm 2Third partyDifference sigmal 2) Finally, the construction block 13 or the processor 30 is adapted to determine the first variance σh 2The second variance σm 2Third difference sigmal 2First mean value XhSecond mean value XmThe third mean value XlAnd the fourth mean value XgConstructing a first sub-feature vector
Figure BDA0003227025410000111
In yet another embodiment, the first sub-feature vector may be based on a first ratio RhA second ratio RmA third ratio RlFirst information entropy EhSecond information entropy EmThird information entropy ElFull picture information entropy EgFirst square difference sigmah 2The second variance σm 2Third difference sigmal 2First mean value XhSecond mean value XmThe third mean value XlAnd the fourth mean value XgConstruction of, i.e.
Figure BDA0003227025410000121
Or according to the first ratio RhA second ratio RmA third ratio RlFirst information entropy EhSecond information entropy EmThird information entropy ElFull picture information entropy EgFirst square difference sigmah 2The second variance σm 2Third difference sigmal 2First mean value XhSecond mean value XmThe third mean value XlAnd the fourth mean value XgThe first sub-feature vector can be actually constructed according to the parameters capable of representing the distinguishing features of the current scene.
Referring to fig. 9, in some embodiments, the format of the preview image is YUV format, 045: constructing a second sub-feature vector according to the color naturalness of each pixel in the preview image, wherein the second sub-feature vector comprises the following steps:
0451: for all pixels in a U color channel of the preview image;
0452: acquiring first color coefficients of all pixels in a U color channel;
0453: obtaining a first shape parameter alpha of a first color coefficient based on generalized Gaussian function distributionuAnd a first standard deviation σu；
0454: all pixels in the V color channel for the preview image;
0455: acquiring second color coefficients of all pixels in the V color channel;
0456: obtaining a second shape parameter alpha of a second color coefficient based on generalized Gaussian function distributionvAnd a second standard deviation σv；
0457: according to a first shape parameter alphauFirst standard deviation σuA second shape parameter alphavAnd a second standard deviation σvA second sub-feature vector is constructed.
Referring to fig. 2, the configuration module 13 is further used to perform the methods of 0451, 0452, 0453, 0454, 0455, 0456 and 0457. That is, the construction module 13 is also used for: for all pixels in a U color channel of the preview image; acquiring first color coefficients of all pixels in a U color channel; obtaining a first shape parameter alpha of a first color coefficient based on generalized Gaussian function distributionuAnd a first standard deviation σu(ii) a All pixels in the V color channel for the preview image; acquiring second color coefficients of all pixels in the V color channel; obtaining a second shape parameter alpha of a second color coefficient based on generalized Gaussian function distributionvAnd a second standard deviation σv(ii) a According to a first shape parameter alphauFirst standard deviation σuA second shape parameter alphavAnd a second standard deviation σvA second sub-feature vector is constructed.
Referring to fig. 3, the processor 30 is further configured to perform the methods of 0451, 0452, 0453, 0454, 0455, 0456, and 0457. That is, the processor 30 is further configured to: for all pixels in a U color channel of the preview image; acquiring first color coefficients of all pixels in a U color channel; obtaining a first shape parameter alpha of a first color coefficient based on generalized Gaussian function distributionuAnd a first standard deviation σu(ii) a For preview imagesAll pixels in the V color channel of the image; acquiring second color coefficients of all pixels in the V color channel; obtaining a second shape parameter alpha of a second color coefficient based on generalized Gaussian function distributionvAnd a second standard deviation σv(ii) a According to a first shape parameter alphauFirst standard deviation σuA second shape parameter alphavAnd a second standard deviation σvA second sub-feature vector is constructed.
And when the format of the preview image is YUV format, constructing a second sub-feature vector (color naturalness feature vector) according to the color coefficient of each color channel of the preview image. Specifically, the Mean subtracted Contrast Normalized (SMCN) coefficients of the color components of the preview image are Gaussian-Distribution-compliant, and the shape parameters of a Generalized Gaussian Distribution (GGD) fitting model of the color coefficients can be used as the color statistical features. Y in the YUV color model is a black-and-white luminance value, and U, V is a chrominance value, so that when constructing the second sub-feature vector of the preview image, only the color coefficient of the U color channel and the color coefficient of the V color channel need to be calculated.
Among them, methods 0451, 0452, and 0453 are used to obtain a first shape parameter and a first standard deviation for a first color coefficient of the U color channel, and methods 0454, 0455, and 0456 are used to obtain a second shape parameter and a second standard deviation for a second color coefficient of the V color channel. Finally according to the first shape parameter alphauFirst standard deviation σuA second shape parameter alphavAnd a second standard deviation σvConstructing a second sub-feature vector
Figure BDA0003227025410000131
A vector of dimensions 1 x 4 is obtained.
Specifically, the color coefficients (the first color coefficient and the second color coefficient) of the preview image I at U, V for two color channels can be represented by equation (5):
Figure BDA0003227025410000132
Figure BDA0003227025410000133
Figure BDA0003227025410000134
Figure BDA0003227025410000135
where CC (i, j) represents the Color Coefficients (Color Coefficients, CC) of the preview image at U, V pixels of the two Color channels, respectively, and the denominator is set to be a constant 1 in order to avoid instability of the denominator toward 0 when the preview image is in a flat area. I isc(i, j) is the value of the C (C ∈ { U, V }) color channel of the preview image at the (i, j) pixel. Equation (6) calculates a local mean, i.e., over the Gaussian blur window ([ -7, 7)]) Average of all pixels in (a). To Ic(i, j) is obtained by performing Gaussian blur calculation. The range of the parameters k, l is the size of the local block (corresponding to the radius of the local block), and can be set by user, such as [ -3,3 [ ]]、[-5,5]、[-7,7]、[-9,9]Etc., generally, the radius is odd. As shown in FIG. 10, the parameters k, l preferably range from [ -7,7 [ ]]This corresponds to a gaussian blur window of radius 7 centered at the (i, j) pixel. Equation (7) is to determine the weight of all pixels in the gaussian blur window. Equation (8) is a two-dimensional circularly symmetric gaussian weighting function used to obtain the weights of all pixels in the gaussian blur window. Thus, a first color coefficient of the U color channel and a second color coefficient of the V color channel are obtained by equation (5).
Since the GGD distribution can effectively capture the distribution characteristics of the color coefficients, after the construction module 13 or the processor 30 acquires the first color coefficients and the second color coefficients, the GGD can be used to fit the first color coefficients and the second color coefficients. The probability density function of GGD is defined as formula (9):
Figure BDA0003227025410000136
Figure BDA0003227025410000141
Figure BDA0003227025410000142
where τ (x) is the gamma function. The parameter alpha is a shape parameter and controls the shape of GGD distribution; σ is the standard deviation. For the U color channel and the V color channel, the parameters (alpha) of Gaussian distribution are obtained by using GGD (Gaussian distribution method) to carry out fittingu，σu 2) And (alpha)v，σv 2) Finally, the construction module 13 or the processor 30 calculates a second sub-feature vector (color naturalness feature) according to the above formula
Figure BDA0003227025410000143
Is a 1 x 4 dimensional vector.
Referring to fig. 11, in some embodiments, the preview image is in RGB format, 045: constructing a second sub-feature vector according to the color naturalness of each pixel in the preview image, wherein the second sub-feature vector comprises the following steps:
0458: all pixels in the R color channel for the preview image;
0459: acquiring first color coefficients of all pixels in an R color channel;
0460: obtaining a first shape parameter alpha of a first color coefficient based on generalized Gaussian function distributionrAnd a first standard deviation σr；
0461: all pixels in the G color channel for the preview image;
0462: acquiring second color coefficients of all pixels in the G color channel;
0463: obtaining a second shape parameter alpha of a second color coefficient based on generalized Gaussian function distributiongAnd a second standard deviation σg；
0464: all pixels in the B color channel for the preview image;
0465: acquiring third color coefficients of all pixels in the B color channel;
0466: obtaining a third shape parameter alpha of a third color coefficient based on generalized Gaussian function distributionbAnd third standard deviation σb；
0467: according to a first shape parameter alpharFirst standard deviation σrA second shape parameter alphagSecond standard deviation σgA third shape parameter alphabAnd third standard deviation σbA second sub-feature vector is constructed.
Referring to fig. 2, the building block 13 is also used to execute the methods of 0458, 0459, 0460, 0461, 0462, 0463, 0464, 0465, 0466 and 0467. That is, the construction module 13 is also used for: all pixels in the R color channel for the preview image; acquiring first color coefficients of all pixels in an R color channel; obtaining a first shape parameter alpha of a first color coefficient based on generalized Gaussian function distributionrAnd a first standard deviation σr(ii) a All pixels in the G color channel for the preview image; acquiring second color coefficients of all pixels in the G color channel; obtaining a second shape parameter alpha of a second color coefficient based on generalized Gaussian function distributiongAnd a second standard deviation σg(ii) a All pixels in the B color channel for the preview image; acquiring third color coefficients of all pixels in the B color channel; obtaining a third shape parameter alpha of a third color coefficient based on generalized Gaussian function distributionbAnd third standard deviation σb(ii) a According to a first shape parameter alpharFirst standard deviation σrA second shape parameter alphagSecond standard deviation σgA third shape parameter alphabAnd third standard deviation σbA second sub-feature vector is constructed.
Referring to fig. 3, the processor 30 is further configured to perform the methods of 0458, 0459, 0460, 0461, 0462, 0463, 0464, 0465, 0466 and 0467. That is, the processor 30 is further configured to: all pixels in the R color channel for the preview image; obtaining R color channel mesogensA first color coefficient of a pixel; obtaining a first shape parameter alpha of a first color coefficient based on generalized Gaussian function distributionrAnd a first standard deviation σr(ii) a All pixels in the G color channel for the preview image; acquiring second color coefficients of all pixels in the G color channel; obtaining a second shape parameter alpha of a second color coefficient based on generalized Gaussian function distributiongAnd a second standard deviation σg(ii) a All pixels in the B color channel for the preview image; acquiring third color coefficients of all pixels in the B color channel; obtaining a third shape parameter alpha of a third color coefficient based on generalized Gaussian function distributionbAnd third standard deviation σb(ii) a According to a first shape parameter alpharFirst standard deviation σrA second shape parameter alphagSecond standard deviation σgA third shape parameter alphabAnd third standard deviation σbA second sub-feature vector is constructed.
In another embodiment, when the preview image is in RGB format, the color coefficients of three color channels may also be calculated R, G, B, and the color coefficient corresponding to each color channel is determined based on the shape parameter and variance of the GGD distribution. The specific implementation process is the same as the implementation process of obtaining the U color channel and the V color channel, and is not described herein again. Finally, the construction module 13 or the processor 30 obtains a second sub-feature vector
Figure BDA0003227025410000151
Referring to fig. 12, in some embodiments, 047: according to the characteristics of the salient region in the preview image, the method comprises the following steps:
0471: calculating the significance values of all pixels in the preview image, wherein the significance values are defined according to the contrast of the current pixel and other color pixels;
0473: dividing a pixel area with a significance value larger than a preset significance threshold value into a significance area X;
0475: calculating the average luminance B of all pixels in the saliency region XxAnd significance information entropy E of significance region Xx(ii) a And
0477: according to the average brightness BxAnd significant information entropy ExA third sub-feature vector is constructed.
Referring to fig. 2, the construction module 13 is also used for executing the methods in 0471, 0473, 0475 and 0477. That is, the construction module 13 is also used for: calculating the significance values of all pixels in the preview image, wherein the significance values are defined according to the contrast of the current pixel and other color pixels; dividing a pixel area with a significance value larger than a preset significance threshold value into a significance area X; calculating the average luminance B of all pixels in the saliency region XxAnd significance information entropy E of significance region Xx(ii) a And according to the average brightness BxAnd significant information entropy ExA third sub-feature vector is constructed.
Referring to fig. 3, the processor 30 is further configured to execute the methods in 0471, 0473, 0475 and 0477. That is, the processor 30 is further configured to: calculating the significance values of all pixels in the preview image, wherein the significance values are defined according to the contrast of the current pixel and other color pixels; dividing a pixel area with a significance value larger than a preset significance threshold value into a significance area X; calculating the average luminance B of all pixels in the saliency region XxAnd significance information entropy E of significance region Xx(ii) a And according to the average brightness BxAnd significant information entropy ExA third sub-feature vector is constructed.
When a person looks at an image, attention is often attracted to a part of an area in the image, the part of the area is called a saliency area and is considered as the most important area in the image by most people, and the image quality of the saliency area influences the evaluation of the image effect by the person. Therefore, the construction module 13 or the processor 30 calculates the average brightness and the information entropy of the salient region in the preview image as the parameters of the third sub-feature vector.
In one embodiment, as shown in fig. 13, it is assumed that the salient region in the preview image is a shaded portion in the figure. When the construction module 13 or the processor 30 calculates the saliency area of the preview image, the saliency area of the preview image may be calculated using a Histogram contrast based image pixel saliency value detection method (HC). Specifically, when calculating the saliency value of a pixel in the preview image, the saliency value is defined by the contrast between the pixel and the colors of other pixels in the preview image, as shown in equation (12):
Figure BDA0003227025410000161
d (I) in formula (12)k,Ii) Is a pixel IkAnd a pixel IiAfter the color distance measurement of the Lab color space is optimized and calculated by formula (12) and the HC method, the saliency value of the preview image is obtained, and then the saliency region (as the shaded portion in fig. 13) of the preview image is screened out by threshold segmentation. As formula (13), the region formed by the pixels whose saliency value is greater than the division threshold is regarded as the saliency region:
Figure BDA0003227025410000162
wherein, S (I)(i,j)) The parameter threshold is the segmentation threshold, which is the significance value of pixel (i, j). X(i,j)The pixel region of 1 is a saliency region, X(i,j)The pixel region of 0 is a non-salient region. Construction module 13 or processor 30 calculates the average luminance B of the saliency region (e.g. the box 1 in fig. 13)xAnd entropy of saliency information ExThus, the third sub-feature vector X ═ B is obtainedx,ExIs a 1 x 2 dimensional vector.
Assuming that the format of the preview image is YUV format, the construction module 13 or the processor 30 obtains the image feature vector according to the first sub-feature vector, the second sub-feature vector and the third sub-feature vector
Figure BDA0003227025410000163
Figure BDA0003227025410000164
Referring to fig. 14, in some embodiments, the SVM model has been trained to obtain a plurality of sample images under different scenes, each sample image has a corresponding image feature vector and a scene tag, the scene tag corresponds to a dynamic range of the scene, 05: determining the dynamic range of the current scene according to the preset SVM model and the image feature vector, which may include:
051: screening out a sample image matched with the image feature vector of the current scene from the SVM model, and taking a scene label of the matched sample image as a scene label of the current scene; and
053: and determining the dynamic range of the current scene according to the scene label.
Please refer to fig. 2, the determining module 15 is also used for executing the methods 051 and 053. That is, the determining module 15 is further configured to: screening out a sample image matched with the image feature vector of the current scene from the SVM model, and taking a scene label of the matched sample image as a scene label of the current scene; and determining the dynamic range of the current scene according to the scene label.
Referring to fig. 3, the processor 30 is also used for executing the methods 051 and 053. That is, the processor 30 is further configured to: screening out a sample image matched with the image feature vector of the current scene from the SVM model, and taking a scene label of the matched sample image as a scene label of the current scene; and determining the dynamic range of the current scene according to the scene label.
Specifically, after the construction module 13 or the processor 30 constructs and obtains the image feature vector F, the image feature vector F is input into a preset SVM model, wherein the SVM model is trained to obtain sample images in different scenes, each sample image has an image feature vector and a scene tag corresponding to a scene, and the scene tag corresponds to a dynamic range of the scene. For example, the dynamic ranges may include no dynamic, low dynamic, medium dynamic, and high dynamic, each dynamic range corresponding to a different scene, and respectively adapted to different photographing algorithms. Therefore, after the determination module 15 or the processor 30 obtains the image feature vector F of the current scene, the SVM model finds a closest image feature vector F1, and outputs the scene label of the image feature vector F1 as the scene label of the current scene, thereby obtaining the dynamic range of the current scene.
Wherein the construction module 13 or the processor 30 can be used for training to obtain the SVM model. Specifically, sample image acquisition mainly acquires enough preview images through a camera of the terminal 100 under different scenes, so as to obtain an observation set of the sample.
The construction module 13 or processor 30 then performs label labeling on the sample image. For example, by viewing and analyzing the acquired preview image, the subjective classification identifies scenes that are obviously HDR (high dynamic, label 3) and scenes that are obviously not HDR, and further calculates the brightness difference of the preview image for the scenes that are obviously not HDR, so as to distinguish whether the dynamic range of the scenes that are not obviously HDR belongs to no dynamic (label 0), low dynamic (label 1) or medium dynamic (label 2). Aiming at boundary scenes which are not easy to judge, a user investigation mode is adopted for calibration, and specifically: forcibly opening the HDR function to shoot and closing the HDR function to shoot at the boundary during image acquisition to obtain two images (namely one image is an effect after HDR processing, and the other image is an effect after the HDR processing is not carried out), taking the two images as a group of comparison images to be blindly selected by a plurality of users, and then taking the comparison images as a basis for marking labels according to the user selection rate condition, wherein if the user selection rate of the images subjected to HDR shooting is high, the effect of the scene subjected to HDR shooting is higher in recognition rate, and the scene labels are high in dynamic state (namely the labels are 3); on the contrary, if the user selection rate of the image without HDR photographing is high, the scene is further calculated whether the scene belongs to no dynamic state, low dynamic state or medium dynamic state.
After each image sample is calibrated, the image feature vector F1 of the image sample is extracted by extracting the image feature vector of the current scene. And finally, constructing a sample image set according to the calibrated label and the image feature vector, wherein each sample image is composed of the image feature vector of the sample image and a corresponding label (F1, Y), wherein Y belongs to {0,1,2,3}, and the total image sample set is { (F) ini,YiAnd i is 1,2,3, … …, n, and n is the number of the whole image sample set, so that the SVM model is obtained through training.
Referring to fig. 15, in some embodiments, the HDR scene detection method may further include:
06: inputting the image feature vector into an SVM model to obtain a prediction probability;
07: when the prediction probability is larger than a preset probability threshold value, a high dynamic HDR algorithm is adapted to take a picture;
08: and when the prediction probability is smaller than the probability threshold, the low dynamic HDR algorithm is adapted to take a picture.
Please refer to fig. 2, the determining module 15 is also used for executing the methods in 06, 07 and 08. That is, the determining module 15 is further configured to: inputting the image feature vector into an SVM model to obtain a prediction probability; when the prediction probability is larger than a preset probability threshold value, a high dynamic HDR algorithm is adapted to take a picture; and when the prediction probability is smaller than the probability threshold, the low dynamic HDR algorithm is adapted to take a picture.
Referring to fig. 3, the processor 30 is also used for executing the methods in 06, 07 and 08. That is, the processor 30 is further configured to: inputting the image feature vector into an SVM model to obtain a prediction probability; when the prediction probability is larger than a preset probability threshold value, a high dynamic HDR algorithm is adapted to take a picture; and when the prediction probability is smaller than the probability threshold, the low dynamic HDR algorithm is adapted to take a picture.
Specifically, when the image feature vector F is input into the SVM model, the output result of the discriminator in the SVM model is combined with the sigmoid function, and can be converted into a classified probability to be output, so as to obtain the prediction probability of the current scene. If the image feature vector of the current scene is F, F (F) is a non-threshold output result of the SVM model; combining f (f) with sigmoid function, converting to probability:
Figure BDA0003227025410000181
a, B is a parameter to be fitted, and maximum likelihood solving is carried out to obtain the prediction probability of the current scene. When the prediction probability is smaller than a preset probability threshold value, a high dynamic HDR algorithm is adapted to take a picture; when the prediction probability is smaller than the probability threshold, the low dynamic HDR algorithm is adapted to take a picture; when the prediction probability is equal to the probability threshold, the adaptive dynamic HDR algorithm takes a picture. From this, supplementary promotion is shot and is filmed the rate, improves user experience.
Referring to fig. 16, the present embodiment further provides a non-volatile computer-readable storage medium 200 containing a computer program 201. The computer program 301, when executed by one or more processors 30, causes the processors 30 to perform scene detection methods in 01, 02, 03, 04, 05, 06, 07, 08, 031, 033, 041, 043, 045, 047, 049, 0431, 0432, 0433, 0434, 0435, 0436, 0437, 0451, 0452, 0453, 0454, 0455, 0456, 0457, 0458, 0459, 0460, 0461, 0462, 0463, 0464, 0465, 0466, 0467, 0471, 0473, 0475, 0477, HDR 051, and 053.
In the description herein, references to the description of the terms "certain embodiments," "one example," "exemplary," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
Although embodiments of the present application have been shown and described above, it is to be understood that the above embodiments are exemplary and not to be construed as limiting the present application, and that changes, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims (12)

1. A High Dynamic Range (HDR) scene detection method, comprising:
acquiring a preview image of a current scene;
extracting multi-dimensional features in the preview image to construct an image feature vector; and
and determining the dynamic range of the current scene according to a preset Support Vector Machine (SVM) model and the image feature Vector.
2. The HDR scene detection method of claim 1, further comprising:
acquiring shooting metadata parameters of the current scene;
performing brightness correction on the preview image according to the shooting metadata parameters; wherein the extracting the multi-dimensional features in the preview image to construct an image feature vector comprises: extracting the multi-dimensional features in the preview image after brightness correction to construct an image feature vector.
3. The HDR scene detection method of claim 2, wherein the capture metadata parameters comprise a first luminance gain and a second luminance gain, the first luminance gain is applied to bright areas of the preview image, the second luminance gain is applied to dark areas of the preview image, and the luminance correction of the preview image according to the capture metadata parameters comprises
Traversing all pixels of the preview image performing the following method:
when the current brightness value of the pixel is larger than a preset first threshold value, taking the product of the current brightness value of the pixel and the first brightness gain as a correction brightness value of the pixel;
and when the current brightness value of the pixel is smaller than a preset second threshold value, taking the ratio of the current brightness value of the pixel to the second brightness gain as a correction brightness value of the pixel.
4. The HDR scene detection method of claim 1, wherein said extracting multi-dimensional features in the preview image to construct an image feature vector comprises:
constructing a first sub-feature vector according to the brightness value of each pixel in the preview image;
constructing a second sub-feature vector according to the color naturalness of each pixel in the preview image;
constructing a third sub-feature vector according to the features of the salient region in the preview image; and
constructing the feature vector according to the first sub-feature vector, the second sub-feature vector and the third sub-feature vector.
5. The HDR scene detection method of claim 4, wherein said constructing a first sub-feature vector according to the luminance values of each pixel in the preview image comprises:
dividing all pixels in the preview image into a first pixel set S according to the brightness value of each pixel based on a k-means clustering algorithmhA second set of pixels SmAnd a third set of pixels SlThe second set of pixels SmIs smaller than the first set S of pixelsmAnd is greater than the third set of pixels SlLuminance values of all pixels in (1);
respectively calculating the first pixel set ShIs compared with the area of all pixels n (i) in the preview imagehThe second set of pixels SmIs compared with the area of all pixels n (i) in the preview imagemAnd the third set of pixels SlIs compared with the area of all pixels n (i) in the preview imagel；
Respectively calculating the first pixel set ShOf all pixels in (1) the first information entropy EhThe second set of pixels SmOf all pixels in (2)mThe third set of pixels SlOf all pixels in (1)lAnd the full image information entropy E of all pixels in the preview imageg；
According to the first ratio RhThe second ratio RmThe third ratio RlThe first information entropy EhThe second information entropy EmThe third information entropy ElAnd the full-image information entropy EgConstructing the first sub-feature vector.
6. The HDR scene detection method of claim 4, wherein the preview image is in YUV format, and the constructing the second sub-feature vector according to the color naturalness of each pixel in the preview image comprises:
for all pixels in a U color channel of the preview image;
acquiring first color coefficients of all pixels in the U color channel;
obtaining a first shape parameter alpha of the first color coefficient based on generalized Gaussian function distributionuAnd a first standard deviation σu；
For all pixels in a V color channel of the preview image;
acquiring second color coefficients of all pixels in the V color channel;
obtaining a second shape parameter alpha of the second color coefficient based on the generalized Gaussian function distributionvAnd a second standard deviation σv；
According to the first shape parameter alphauThe first standard deviation sigmauThe second shape parameter αvAnd the second standard deviation σvConstructing the second sub-feature vector.
7. The HDR scene detection method of claim 4, wherein said detecting, according to the feature of the salient region in the preview image, comprises:
calculating saliency values of all pixels in the preview image, the saliency values being defined according to the contrast of the current pixel with other color pixels;
dividing the pixel area with the significance value larger than a preset significance threshold value into a significance area X;
calculating the average brightness B of all pixels in the saliency area XxAnd significance information entropy E of the significance region Xx(ii) a And
according to the average brightness BxAnd the significant information entropy ExConstructing the third sub-feature vector.
8. The HDR scene detection method of claim 1, wherein the SVM model is trained to obtain sample images under a plurality of different scenes, each sample image has a corresponding image feature vector and a scene tag, the scene tag corresponds to a dynamic range of a scene, and the determining the dynamic range of the current scene according to a preset SVM model and the image feature vector comprises:
screening the sample image matched with the image feature vector of the current scene from the SVM model, and taking a scene label of the matched sample image as a scene label of the current scene; and
and determining the dynamic range of the current scene according to the scene label.
9. The HDR scene detection method of claim 1, comprising:
inputting the image feature vector into the SVM model to obtain a prediction probability;
when the prediction probability is larger than a preset probability threshold value, a high dynamic HDR algorithm is adapted to take a picture;
and when the prediction probability is smaller than the probability threshold, adapting a low dynamic HDR algorithm to take a picture.
10. An HDR scene detection apparatus, comprising:
the acquisition module is used for acquiring a preview image of a current scene;
the construction module is used for extracting multi-dimensional features in the preview image so as to construct an image feature vector; and
and the determining module is used for determining the dynamic range of the current scene according to a preset SVM model and the image feature vector.
11. A terminal, characterized in that the terminal comprises:
one or more processors, memory; and
one or more programs, wherein one or more of the programs are stored in the memory and executed by one or more of the processors, the programs comprising instructions for performing the HDR scene detection method of any of claims 1 to 9.
12. A non-transitory computer readable storage medium storing a computer program which, when executed by one or more processors, implements the HDR scene detection method of any of claims 1 to 9.
CN202110974142.8A 2021-08-24 2021-08-24 HDR scene detection method and device, terminal and readable storage medium Active CN113691724B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110974142.8A CN113691724B (en) 2021-08-24 2021-08-24 HDR scene detection method and device, terminal and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110974142.8A CN113691724B (en) 2021-08-24 2021-08-24 HDR scene detection method and device, terminal and readable storage medium

Publications (2)

Publication Number Publication Date
CN113691724A true CN113691724A (en) 2021-11-23
CN113691724B CN113691724B (en) 2023-04-28

Family

ID=78581837

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110974142.8A Active CN113691724B (en) 2021-08-24 2021-08-24 HDR scene detection method and device, terminal and readable storage medium

Country Status (1)

Country Link
CN (1) CN113691724B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114449176A (en) * 2022-01-10 2022-05-06 瑞芯微电子股份有限公司 Automatic exposure method, dynamic range identification method, device, medium, and apparatus
CN114897745A (en) * 2022-07-14 2022-08-12 荣耀终端有限公司 Method for expanding dynamic range of image and electronic equipment
CN116363017A (en) * 2023-05-26 2023-06-30 荣耀终端有限公司 Image processing method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104038704A (en) * 2014-06-12 2014-09-10 小米科技有限责任公司 Backlight portrait scene photographing processing method and device
CN107451969A (en) * 2017-07-27 2017-12-08 广东欧珀移动通信有限公司 Image processing method, device, mobile terminal and computer-readable recording medium
CN111724447A (en) * 2019-03-18 2020-09-29 浙江宇视科技有限公司 Image processing method, system, electronic equipment and storage medium
CN112969032A (en) * 2021-04-13 2021-06-15 深圳市海雀科技有限公司 Illumination pattern recognition method and device, computer equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104038704A (en) * 2014-06-12 2014-09-10 小米科技有限责任公司 Backlight portrait scene photographing processing method and device
CN107451969A (en) * 2017-07-27 2017-12-08 广东欧珀移动通信有限公司 Image processing method, device, mobile terminal and computer-readable recording medium
CN111724447A (en) * 2019-03-18 2020-09-29 浙江宇视科技有限公司 Image processing method, system, electronic equipment and storage medium
CN112969032A (en) * 2021-04-13 2021-06-15 深圳市海雀科技有限公司 Illumination pattern recognition method and device, computer equipment and storage medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114449176A (en) * 2022-01-10 2022-05-06 瑞芯微电子股份有限公司 Automatic exposure method, dynamic range identification method, device, medium, and apparatus
CN114897745A (en) * 2022-07-14 2022-08-12 荣耀终端有限公司 Method for expanding dynamic range of image and electronic equipment
CN114897745B (en) * 2022-07-14 2022-12-20 荣耀终端有限公司 Method for expanding dynamic range of image and electronic equipment
CN116363017A (en) * 2023-05-26 2023-06-30 荣耀终端有限公司 Image processing method and device
CN116363017B (en) * 2023-05-26 2023-10-24 荣耀终端有限公司 Image processing method and device

Also Published As

Publication number Publication date
CN113691724B (en) 2023-04-28

Similar Documents

Publication Publication Date Title
US11882357B2 (en) Image display method and device
CN111402135B (en) Image processing method, device, electronic equipment and computer readable storage medium
CN110062160B (en) Image processing method and device
CN108764208B (en) Image processing method and device, storage medium and electronic equipment
US10929718B2 (en) Image recognition apparatus, learning apparatus, image recognition method, learning method, and storage medium
US8977056B2 (en) Face detection using division-generated Haar-like features for illumination invariance
US8605955B2 (en) Methods and apparatuses for half-face detection
CN113691724B (en) HDR scene detection method and device, terminal and readable storage medium
US7680342B2 (en) Indoor/outdoor classification in digital images
EP1800259B1 (en) Image segmentation method and system
EP3598736B1 (en) Method and apparatus for processing image
US8295606B2 (en) Device and method for detecting shadow in image
US7595823B2 (en) Providing optimized digital images
US8811733B2 (en) Method of chromatic classification of pixels and method of adaptive enhancement of a color image
CN108616689B (en) Portrait-based high dynamic range image acquisition method, device and equipment
CN110349163B (en) Image processing method and device, electronic equipment and computer readable storage medium
CN111739110B (en) Method and device for detecting image over-darkness or over-exposure
CN112258380A (en) Image processing method, device, equipment and storage medium
CN110365897B (en) Image correction method and device, electronic equipment and computer readable storage medium
CN108769543B (en) Method and device for determining exposure time
Guthier et al. Algorithms for a real-time HDR video system
CN113747062B (en) HDR scene detection method and device, terminal and readable storage medium
CN115988310A (en) Face brightness calculation method, device, medium and equipment
CN114697483A (en) Device and method for shooting under screen based on compressed sensing white balance algorithm
TW201435756A (en) Method for determining dynamic range mode and image capturing device thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant