WO2023000878A1 - Photographing method and apparatus, and controller, device and computer-readable storage medium - Google Patents

Abstract

A photographing method and apparatus, and a controller, a device and a computer-readable storage medium. The photographing method comprises: acquiring a preview image of a current scenario (S100); performing feature extraction on the preview image, so as to obtain brightness feature information of the preview image (S200); inputting the brightness feature information into a strategy generation model, so as to obtain an exposure strategy, wherein the strategy generation model is obtained by training a neural network according to brightness feature information, which corresponds to a sample image (S300); and photographing the current scenario on the basis of the exposure strategy (S400).

Description

Shooting method, device, controller, device, and computer-readable storage medium

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202110828028.4 and a filing date of July 22, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference into this application.

technical field

The embodiments of the present application relate to but are not limited to the technical field of photographing, and in particular, relate to a photographing method, a photographing device, a controller, a photographing device, and a computer-readable storage medium.

Background technique

At present, due to the physical characteristics of the sensor, such as dynamic range, noise, etc., when shooting outdoor backlit scenes, some areas need to be enhanced, and some areas need to be weakened, so it can be processed by HDR (High-Dynamic Range, High Dynamic Light Rendering) Get images with an ideal wide dynamic range. Existing processing technology is to use multi-group exposure synthesis to suppress the brightness of high-brightness scenes and increase the brightness of low-brightness scenes. Therefore, the existing processing technology needs accurate exposure parameters. However, most of the existing HDR trigger mechanisms use the statistical features of the overall brightness, the number of decisions, or the preset threshold judgment, all of which have limited ability to recognize the scene, and cannot accurately guide the exposure parameters after the scene is recognized, thus affecting the shooting quality.

Contents of the invention

The following is an overview of the topics described in detail in this article. This summary is not intended to limit the scope of the claims.

Embodiments of the present application provide a photographing method, a photographing device, a controller, a photographing device, and a computer-readable storage medium.

In the first aspect, an embodiment of the present application provides a shooting method, including: acquiring a preview image of the current scene; performing feature extraction on the preview image to obtain brightness characteristic information of the preview image; inputting the brightness characteristic information Go to the strategy generation model to obtain an exposure strategy, wherein the strategy generation model is trained by a neural network according to the brightness feature information corresponding to the sample image; and the current scene is photographed based on the exposure strategy.

In the second aspect, the embodiment of the present application also provides a photographing device, including a processor, a memory, a photographing component, and a display screen, and the processor is respectively connected to the memory, the photographing component, and the display screen, so The processor includes a feature extraction unit and a policy generation unit, the photographing component includes an optical camera; the photographing component is configured to obtain a preview image corresponding to the current scene through the optical camera; the display screen is configured to display the The preview image corresponding to the current scene; the feature extraction unit is configured to acquire the preview image displayed on the display screen, and perform feature extraction on the preview image to obtain brightness feature information of the preview image; The strategy generation unit is configured to input the brightness feature information into a pre-trained strategy generation model in the processor to obtain an exposure strategy, wherein the strategy generation model is configured by the neural network according to the brightness corresponding to the sample image The feature information is obtained through training; the photographing component is further configured to acquire the exposure strategy, and photograph the current scene through the optical camera based on the exposure strategy.

In the third aspect, the embodiment of the present application also provides a controller, including: a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor executes the computer program. The program implements the photographing method described in the first aspect above.

In a fourth aspect, an embodiment of the present application further provides a photographing device, including the photographing device described in the second aspect above or the controller described in the third aspect above.

In a fifth aspect, the embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the photographing method as described in the above-mentioned first aspect.

Additional features and advantages of the application will be set forth in the description which follows, and, in part, will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Description of drawings

The accompanying drawings are used to provide a further understanding of the technical solution of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical solution of the present application, and do not constitute a limitation to the technical solution of the present application.

FIG. 1 is a schematic diagram of a system architecture platform for executing a shooting method provided by an embodiment of the present application;

FIG. 2 is a flowchart of a shooting method provided by an embodiment of the present application;

Fig. 3 is a flow chart of extracting brightness feature information in a shooting method provided by an embodiment of the present application;

FIG. 4 is a flow chart of obtaining brightness position weights in a shooting method provided by an embodiment of the present application;

FIG. 5 is a flow chart of obtaining brightness proportion weights in a shooting method provided by an embodiment of the present application;

Fig. 6 is a flow chart of obtaining brightness characteristic information according to brightness position weight, brightness proportion weight and brightness interval ratio in the shooting method provided by an embodiment of the present application;

Fig. 7 is a schematic diagram of exposure strategy combinations provided by an embodiment of the present application;

Fig. 8 is a schematic structural diagram of a neural network model provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a photographing device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

It should be noted that although the functional modules are divided in the schematic diagram of the device, and the logical sequence is shown in the flowchart, in some cases, it can be executed in a different order than the module division in the device or the flowchart in the flowchart. steps shown or described. The terms "first", "second" and the like in the specification, claims or the above drawings are used to distinguish similar objects, and not necessarily used to describe a specific order or sequence.

In related technologies, due to the physical characteristics of the sensor, such as dynamic range and noise, when shooting outdoor backlit scenes, some areas need to be enhanced, and some areas need to be weakened, so an ideal wide dynamic range image can be obtained through HDR processing . Existing processing technology is to use multi-group exposure synthesis to suppress the brightness of high-brightness scenes and increase the brightness of low-brightness scenes. Therefore, the existing processing technology needs accurate exposure parameters. However, most of the existing HDR trigger mechanisms use the statistical features of the overall brightness, the number of decisions, or the preset threshold judgment, all of which have limited ability to recognize the scene, and cannot accurately guide the exposure parameters after the scene is recognized, thus affecting the shooting quality.

It should be noted that the existing multi-frame dynamic calculation, single-frame scene decision tree recognition, and the entire feature brightness recognition of HDR scenes are not accurate enough to guide the selection of multiple exposure parameters in HDR.

Based on the above situation, an embodiment of the present application provides a shooting method, a shooting device, a controller, a shooting device, and a computer-readable storage medium. The shooting method includes but is not limited to the following steps: acquiring a preview image of the current scene; Perform feature extraction to obtain the brightness feature information of the preview image; input the brightness feature information into the policy generation model to obtain the exposure strategy, wherein the strategy generation model is trained by the neural network according to the brightness feature information corresponding to the sample image; based on the exposure strategy to shoot the current scene. According to the technical solution of the embodiment of the present application, during the preview period of shooting, the embodiment of the present application will extract the brightness feature information of the preview image of the current scene and input it into the trained policy generation model, because the policy generation model of the embodiment of the present application is composed of The neural network is trained based on the brightness feature information corresponding to the sample image. Therefore, the strategy generation model will respond to output the exposure strategy corresponding to the preview image of the current scene, and then the exposure strategy output by the strategy generation model can be adopted when the shooting button is pressed. During the shooting process, because the accuracy of the exposure strategy output by the policy generation model is higher, the embodiment of the present application can improve the shooting quality.

The embodiments of the present application will be further described below in conjunction with the accompanying drawings.

As shown in FIG. 1 , FIG. 1 is a schematic diagram of a system architecture platform for executing a shooting method provided by an embodiment of the present application.

In the example shown in FIG. 1 , the system architecture platform 100 is provided with a processor 110 and a memory 120 , wherein the processor 110 and the memory 120 may be connected via a bus or in other ways. In FIG. 1 , connection via a bus is taken as an example.

The memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some implementations, the memory 120 may optionally include memory 120 located remotely relative to the processor 110, and these remote memories may be connected to the system architecture platform through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Those skilled in the art can understand that the system architecture platform can be applied to 3G communication network systems, LTE communication network systems, 5G communication network systems and subsequent evolved mobile communication network systems, etc., which is not specifically limited in this embodiment.

Those skilled in the art can understand that the system architecture platform shown in FIG. 1 does not constitute a limitation to the embodiment of the present application, and may include more or less components than those shown in the illustration, or combine some components, or have different Part placement.

In the system architecture platform shown in FIG. 1 , the processor 110 can call the shooting program stored in the memory 120 to execute the shooting method.

Based on the above-mentioned system architecture platform, various embodiments of the shooting method of the present application are proposed below.

As shown in FIG. 2 , FIG. 2 is a flowchart of a shooting method provided by an embodiment of the present application, and the method includes but is not limited to step S100 , step S200 , step S300 and step S400 .

Step S100, acquiring a preview image of the current scene.

Step S200, performing feature extraction on the preview image to obtain brightness feature information of the preview image.

Step S300 , input the brightness feature information into the strategy generation model to obtain the exposure strategy, wherein the strategy generation model is trained by the neural network according to the brightness feature information corresponding to the sample image.

Step S400, shoot the current scene based on the exposure strategy.

Specifically, in the embodiment of the present application, during the shooting preview period, the brightness feature information of the preview image of the current scene will be extracted and input to the trained policy generation model. Since the policy generation model in the embodiment of the present application is based on the neural network The corresponding brightness feature information is trained. Therefore, the policy generation model will respond to output the exposure strategy corresponding to the preview image of the current scene, and then use the exposure strategy output by the policy generation model to perform shooting processing when the shooting button is pressed. The accuracy of the exposure strategy output by the strategy generation model is higher, so the embodiment of the present application can improve the shooting quality.

It should be noted that, regarding the acquisition of the preview image of the current scene in the above step S100, the preview image refers to the image displayed on the screen of the shooting device before the shutter is pressed. Exemplarily, the preview image refers to the image displayed on the screen of the shooting device before the user holds the shooting device against the scenery and does not press the shooting button.

In addition, it is worth noting that the above-mentioned policy generation model is a pre-trained model. During the training period, the embodiment of the present application will extract the brightness characteristic information of the sample image, and use it as the input of the neural network, so that the neural network can output the corresponding exposure strategy. After the strategy generation model is trained, if the brightness feature information of the preview image of the current scene is input to the strategy generation model, the strategy generation model will respond to output a more accurate exposure strategy.

It can be understood that the number of the above sample images may be multiple.

In addition, it can be understood that, regarding the above-mentioned exposure strategy, there may be multiple exposure parameters.

In addition, it can be understood that the above-mentioned neural network is an algorithmic mathematical model that imitates the behavior characteristics of animal neural networks and performs distributed parallel information processing. This kind of network depends on the complexity of the system, and achieves the purpose of processing information by adjusting the interconnection relationship between a large number of internal nodes.

In addition, as shown in Figure 3, Figure 3 is a flow chart of extracting brightness feature information in the shooting method provided by an embodiment of the present application. Regarding the above step S200, it includes but is not limited to step S410, step S420, step S430 and step S440 .

Step S410, divide the preview image to obtain multiple sub-regions, and determine multiple target sub-regions from the multiple sub-regions.

Step S420, for each target sub-region, calculate the cumulative brightness distribution value according to the brightness value of the target sub-region, and obtain the brightness interval ratio of the target sub-region according to the cumulative brightness distribution value and the preset brightness threshold.

Step S430, obtaining the brightness position weight of each target sub-region.

Step S440, according to the luminance position weights and luminance interval ratios of all target sub-regions, calculate the luminance characteristic information of the preview image.

Specifically, in the embodiment of the present application, during the extraction process, the preview image is divided to obtain multiple sub-regions, and a certain number of target sub-regions are selected from the multiple sub-regions; then the cumulative brightness distribution of each target sub-region is calculated Value, that is, the CDF (cumulative distribution function, cumulative distribution function) value of the high and low brightness; since the embodiment of the present application will set a preset brightness threshold, which represents a certain brightness range, the embodiment of the present application will be based on the cumulative distribution function. The brightness distribution value and the preset brightness threshold obtain the brightness interval ratio of the target sub-area, that is, the proportion in each brightness interval; then, because the positions of different target sub-areas are different, their distances from the exposure center are also different. Therefore, The embodiment of the present application will also obtain the brightness position weight of the target sub-region; finally, the embodiment of the present application will calculate the brightness feature information of the preview image according to the brightness position weight and brightness interval ratio of all target sub-regions, so as to realize the brightness of the preview image Extraction of feature information.

It should be noted that the embodiment of the present application calculates the brightness distribution of different areas of the preview image corresponding to the current scene, extracts the brightness feature value as the input of the neural network, and uses the calibrated scene classification as the output of the neural network. After the pre-training is completed, The network structure can effectively solve the selection of exposure parameters in scenes that require exposure fusion.

It is worth noting that the preset brightness threshold mentioned above may include but not limited to a preset high brightness threshold and a preset low brightness threshold. Correspondingly, since there are different ranges of preset brightness thresholds, for each target sub region, the brightness interval ratio obtained in the embodiment of the present application also includes, but is not limited to, the ratio of high brightness interval and the ratio of low brightness interval; The information also correspondingly includes feature information of high-brightness intervals and feature information of low-brightness intervals.

Specifically, when the preset brightness threshold includes a preset high-brightness threshold and a preset low-brightness threshold, the step S420 to obtain the brightness interval ratio of the target sub-region according to the cumulative brightness distribution value and the preset brightness threshold includes: according to the accumulated The brightness distribution value and the preset high-brightness threshold obtain the ratio of the high-brightness interval of the target sub-region, and the ratio of the low-brightness interval of the target sub-region is obtained according to the accumulated brightness distribution value and the preset low-brightness threshold.

In addition, correspondingly, with regard to the brightness feature information of the preview image calculated according to the brightness position weights and brightness interval ratios of all target sub-regions in the above step S440, including: according to the brightness position weights and highlight interval ratios of all target sub-regions The feature information of the high-brightness interval of the preview image is calculated, and the feature information of the low-brightness interval of the preview image is calculated according to the brightness position weights of all target sub-regions and the ratio of the low-brightness interval.

In addition, it is worth noting that the number of target sub-regions is less than or equal to the number of sub-regions, specifically, the number of sub-regions and the number of target sub-regions can be consistent, or the number of sub-regions and the number of target sub-regions can be inconsistent .

Specifically, when the number of sub-regions is consistent with the number of target sub-regions, all divided sub-regions are target sub-regions. Therefore, the above-mentioned brightness position weight may refer to the global brightness position weight of the preview image.

Or, when the number of sub-regions is inconsistent with the number of target sub-regions, that is, when the number of target sub-regions is less than the number of sub-regions, that is, only part of the divided sub-regions are target sub-regions, therefore, The aforementioned brightness position weights may refer to local brightness position weights of the preview image.

In addition, when the number of target sub-regions is smaller than the number of sub-regions, the target sub-region is located at the center of the preview image or at the periphery of the preview image, or, in this embodiment of the present application, an interested position can be selected according to user requirements.

In addition, as shown in FIG. 4, FIG. 4 is a flow chart of acquiring brightness position weights in the shooting method provided by an embodiment of the present application. The above step S430 includes but not limited to step S510 and step S520.

Step S510, acquiring distance information between the target sub-region and a preset position in the preview image.

Step S520, calculating and obtaining the brightness position weight of each target sub-region according to the distance information.

Specifically, since the positions of different target sub-regions are different, their distances from the exposure center are also different. Therefore, the embodiment of the present application will also obtain the distance information between the target sub-region and the preset position in the preview image, and according to the distance The information is calculated to obtain the brightness position weight of each target sub-region.

It can be understood that, with regard to the aforementioned preset position, it may, but is not limited to, refer to the position of the exposure center or the center position of the preview image.

In addition, as shown in FIG. 5 , FIG. 5 is a flowchart of obtaining brightness proportion weights in a shooting method provided by an embodiment of the present application. The shooting method in the embodiment of the present application also includes but is not limited to step S600 .

Step S600 , according to the brightness interval ratio of the target sub-region, calculate the brightness proportion weight.

Specifically, the embodiment of the present application also adjusts the weight value according to the brightness proportion value, that is, the brightness interval ratio.

It should be noted that, compared with the above-mentioned brightness position weights, the brightness proportion weights mentioned here are used for fine-tuning the exposure strategy, while the above-mentioned brightness position weights are used for rough adjustment of the exposure strategy.

Based on Fig. 5, as shown in Fig. 6, Fig. 6 is a flow chart of obtaining luminance feature information according to luminance position weight, luminance proportion weight and luminance interval ratio in the shooting method provided by an embodiment of the present application. Regarding the above step S440, it includes But not limited to step S700.

Step S700 , according to the luminance position weights, luminance proportion weights and luminance interval ratios of all target sub-regions, calculate the luminance characteristic information of the preview image.

Specifically, in the embodiment of the present application, when inputting into the policy generation model, the brightness proportion weight can also be input into the policy generation model. Since the brightness proportion weight can play a role in fine-tuning the exposure strategy, therefore, in step S700 The brightness feature information output by the policy generation model will be more accurate.

Based on the method steps in Figure 2 to Figure 6, when the number of sub-regions is M*N, the number of target sub-regions is i*j, where i≤M, j≤N; the cumulative brightness distribution value is cdf _i,j ,i∈[1,M],j∈[1,N]; when the preset highlight threshold is [lum3,lum4], the preset low brightness threshold is [lum1,lum2], where, lum4＞lum3 ≥ lum2 > lum1.

Specifically, the highlight interval ratio is obtained by the following formula: L _i,j =cdf _i,j (lum4)-cdf _i,j (lum3), wherein, L _i,j is the highlight interval ratio.

The ratio of the low-brightness interval is obtained by the following formula: D _i,j =cdf _i,j (lum2)-cdf _i,j (lum1), where D _i,j is the ratio of the low-brightness interval.

In addition, the brightness position weight can be obtained by the following formula:

Among them, W _d (i, j) is the brightness position weight, and δ ₁ is an adjustable parameter.

In addition, the brightness proportion weight can be obtained by the following formula:

Among them, W _a (lum) is the brightness position weight, δ ₂ and u are adjustable parameters.

In addition, the feature information of the highlighted interval can be obtained by the following formula:

Among them, vl _k is the feature information of the highlighted interval.

In addition _{, the} feature information of the low-brightness interval can be obtained by the following formula:

Among them, vd _k is the feature information of the low-brightness interval.

Based on the method steps in Fig. 2 to Fig. 6 and the above-mentioned derivation formula, the embodiment of the present application provides an overall implementation scheme, specifically as follows:

Step 1: Obtain the brightness value of the image with size (w, h), and Lum _{i, j} represents the brightness at position (i, j):

Lum _i,j ＝Max(R _i,j ,G _i,j ,B _i,j ),i≤h,j≤w

Step 2: Divide the obtained luminance image into M*N sub-regions, and calculate the cumulative luminance distribution value of each sub-region respectively: cdf _m,n ,m∈[1,M],n∈[1,N]. Set the preset low brightness threshold: [lum1, lum2], set the preset high brightness threshold: [lum3, lum4], where, lum4＞lum3≥lum2＞lum1.

Obtain the highlight ratio of each sub-region, that is, the ratio of the highlight interval: L _m,n = cdf _m,n (lum4)-cdf _m,n (lum3)

Obtain the low-brightness ratio of each sub-region, that is, the low-brightness interval ratio: D _m,n = cdf _m,n (lum2)-cdf _m,n (lum1)

Step 3, obtain the weight of the area to be counted:

The global brightness weight W _d , that is, the global brightness position weight W _d , adjusts the weight according to the distance from the center. Examples of weights are as follows but not limited to:

Among them, δ ₁ is an adjustable parameter, m≤M, n≤N.

Proportion weight W _a , that is, brightness proportion weight W _a , adjust the weight value according to the brightness proportion value. Examples of weights are as follows but not limited to the following forms:

Wherein, W _a (lum) is the brightness position weight, δ ₂ and u are adjustable parameters, m≤M, n≤N.

It should be noted that the weight can also be stored in a table, and the calculation is realized by using a LUT (Look-Up-Table, display look-up table) to simplify calculation and increase efficiency.

Local brightness weight W _d : Assign weight values in sub-regions of different positions to obtain ROI (region of interest, region of interest) area: W _d (i, j).

Exemplarily, for example, the central area W _d (i, j), is distributed in the form of the following table 1:

0	0	0	0	0	0	0
0	0	0	0	0	0	0
0	0	1	1	1	0	0
0	0	1	1	1	0	0
0	0	0	0	0	0	0
0	0	0	0	0	0	0

Table 1

Or, exemplarily, for example, the surrounding area W _d (i, j), in a similar form as in Table 2 below:

1	1	1	1	1	1	1
1	1	1	1	1	1	1
1	1	0	0	0	1	1
1	1	0	0	0	1	1
1	1	1	1	1	1	1
1	1	1	1	1	1	1

Table 2

Alternatively, a similar method may be used for local areas such as other corners and diagonals, so as to obtain different local feature weight tables.

Step 4: Obtain the brightness characteristic distribution value of the image through weighted average, that is, the brightness characteristic information.

The 2k eigenvalues of the entire image (k>1) are obtained through local or global weights, where the two weights W _d and W _a of the bright area and the dark area can be different, and the weights of the bright area and the dark area can be calculated as follows :

The feature value of the bright area, that is, the feature information of the highlighted interval:

The feature value of the dark area, that is, the feature information of the low-brightness interval:

It should be noted that the weight can also be realized by LUT during calculation, so as to obtain a 2k-dimensional vector composed of a set of brightness feature information:

vec _in ＝vector(vl ₁ ,vd ₁ ,...,vl _k ,vd _k )

Step 5: Input the trained neural network with the calculated feature vector to obtain an accurate exposure strategy to start the HDR algorithm and synthesize multiple images.

Classify the exposure strategy into the corresponding output value, and calibrate the exposure strategy value required in the HDR algorithm by actually taking photos. For example, the output of the exposure strategy is an L-dimensional vector: vec _out = vector(exp ₁ ,...,exp _L )

Regarding the selection method of exposure parameters, for example, HDR synthesis requires three photos under different exposure parameters. In addition to the normal exposure photos, other high and low exposure parameters are combined as the output of the neural network, for example, the low exposure EV ^-Divided into three levels [ev ^- 1, ev ^- 2, ev ^- 3], high exposure EV ⁺ is divided into three levels: [ev ⁺ 1, ev ⁺ 2, ev ⁺ 3], where the combination of exposure strategies can be as follows As shown in FIG. 7 , FIG. 7 is a schematic diagram of an exposure strategy combination provided by an embodiment of the present application.

Specifically, in the embodiment of the present application, the exposure strategy required for the corresponding sample photo can be selected as the output of the network, and then the network is trained to obtain the optimal structure.

When HDR fusion is not limited to three exposure levels (normal, low exposure, high exposure), each time a photo to be fused is added, an exposure setting is added. For example, when five photos are fused, the Low-exposure refinement: On the basis of the original three-fusion, two more exposure settings are added: EV ^-- ＝[ev ^-- 1,ev ^-- 2,ev ^-- 3,...], EV ⁺⁺ ＝[ ev ⁺⁺ 1,ev ⁺⁺ 2,ev ⁺⁺ 3,...].

It can be understood that the structure of the neural network model used above can refer to the form shown in FIG. 8 . That is, one or more hidden layers are included between the input layer and the output layer to better divide or improve the classification effect, and add a softmax layer after the output layer to optimize the network output effect. In some other implementation manners, any other suitable neural network model can also be used.

When there are many input feature vectors and output values, adjust the network results as needed, such as increasing or decreasing network nodes, increasing network levels, etc. The output can also increase the softmax layer to meet the computing needs. When shooting, the trained network runs in the terminal through hardware support or software solidification. When previewing, calculate the brightness characteristic value of the current scene, and input it into the network to calculate the exposure parameters required to start HDR. After it takes effect, the photo is taken. HDR photo fusion.

Regarding the shooting method of the embodiment of the present application, the luminance distribution of the whole picture is counted by the sub-region, the CDF value of the sub-region is calculated to obtain the high-brightness and low-brightness ratio values, and several weight coefficients are designed to obtain the luminance distribution characteristics of the current scene: through The distance weight is obtained according to the distance from the center, and the global bright and low brightness distribution characteristics are obtained; the weight of different brightness is adjusted through the difference in brightness ratio, and the bright and low brightness distribution characteristics are obtained; the middle of the weight distribution is adjusted to 1, and the surrounding is 0, to obtain the local The characteristic value of high and low brightness distribution; adjust the weight distribution to 0 in the middle and 1 around to obtain the local characteristic value of high and low brightness distribution.

According to the calibrated picture, the backlighting degree of the scene or the exposure combination of HDR is used as the output value, and the network is trained to obtain the network structure. When shooting, the value of the global or local brightness feature is obtained through several weights and input into the trained network, so as to obtain the degree of backlight or the required exposure parameters triggered by HDR.

In addition, it should be noted that the method steps of the training process of the policy generation model and the shooting process of the current scene are similar.

Specifically, the training process of the policy generation model is as follows: obtain multiple sample images, and divide the sample images to obtain multiple sub-regions, and determine multiple target sub-regions from the multiple sub-regions; then, for each of the sample images Target sub-area, calculate the cumulative brightness distribution value according to the brightness value of the target sub-region in the sample image, and obtain the highlight interval ratio of the target sub-region according to the cumulative brightness distribution value and the preset highlight threshold value, and according to the cumulative brightness distribution value and the preset low-brightness threshold to obtain the low-brightness interval ratio of the target sub-region; then, obtain the distance information between the target sub-region and the preset position in the sample image, and calculate the brightness position weight of each target sub-region according to the distance information ; Then, according to the brightness interval ratio of the target sub-region, calculate the brightness proportion weight; then, calculate the brightness feature information of the sample image according to the brightness position weight, brightness proportion weight and brightness interval ratio of all target sub-regions; finally , use the calculated brightness feature information to train the neural network, and use the exposure parameters to combine the corresponding category vectors as the output of the network, so as to obtain the degree of backlight or the exposure parameters required for HDR triggering.

Based on the photographing methods of the above embodiments, various embodiments of the photographing device of the present application are proposed below.

As shown in FIG. 9, FIG. 9 is a schematic structural diagram of a photographing device provided by an embodiment of the present application. The photographing device 200 includes but is not limited to a processor 110, a memory 120, a photographing component 130, and a display screen 140, wherein the processor 110 Respectively connected to the memory 120, the photographing component 130 and the display screen 140, the processor 110 includes but not limited to a feature extraction unit 111 and a strategy generation unit 112, the processor 110 can call the photographing program stored in the memory 120, thereby executing the photographing method , the photographing component 130 includes but is not limited to an optical camera 131 .

Specifically, the photographing part 130 aims at the current scene to be photographed through the optical camera 131, and displays a preview image corresponding to the current scene on the display screen 140; The displayed preview image, and feature extraction is performed on the preview image to obtain the brightness feature information of the preview image; then, the policy generation unit 112 in the processor 110 will input the brightness feature information into the pre-trained policy generation model in the processor 110 , to obtain the exposure strategy, wherein, the strategy generation model is trained by the neural network according to the brightness feature information corresponding to the sample image; finally, the photographing component 130 will obtain the obtained exposure strategy, and based on the exposure strategy, the optical camera 131 will shoot the current scene .

It should be noted that for the specific implementation manners and technical effects of the photographing device in the embodiment of the present application, reference may be made to the specific implementation manners and technical effects of the photographing method in the foregoing embodiments.

Based on the photographing methods of the above embodiments, various embodiments of the controller, the photographing device, and the computer-readable storage medium of the present application are proposed below.

In addition, an embodiment of the present application provides a controller, which includes: a memory, a processor, and a computer program stored in the memory and operable on the processor.

The processor and memory can be connected by a bus or other means.

It should be noted that the controller in this embodiment may correspond to include the memory and the processor in the embodiment shown in Figure 1, which can constitute a part of the system architecture platform in the embodiment shown in Figure 1, and both belong to The same idea, so both have the same implementation principle and beneficial effect, which will not be described in detail here.

The non-transitory software programs and instructions required to realize the shooting method of the above-mentioned embodiment are stored in the memory, and when executed by the processor, the shooting method of the above-mentioned embodiment is executed, for example, the method step S100 in FIG. 2 described above is executed to S400 , method steps S410 to S440 in FIG. 3 , method steps S510 to S520 in FIG. 4 , method step S600 in FIG. 5 , method step S700 in FIG. 6 .

The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, an embodiment of the present application provides a photographing and photographing device, including the photographing device of the foregoing embodiment or the controller of the foregoing embodiment.

Specifically, the shooting and shooting device in this embodiment of the present application may be a terminal device, such as a mobile phone, a tablet computer, a wearable shooting device, and the like. Wherein, the photographing device itself may carry a camera for capturing images.

It should also be understood that, in some other embodiments of the present application, the shooting device in the above embodiments may also not be a terminal shooting device, but a desktop computer or a server capable of realizing the shooting function.

In addition, an embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are used to execute the above-mentioned photographing method, for example, to execute the above-described Method steps S100 to S400 in FIG. 2 , method steps S410 to S440 in FIG. 3 , method steps S510 to S520 in FIG. 4 , method steps S600 in FIG. 5 , and method steps S700 in FIG. 6 .

The embodiment of the present application includes: first acquiring the preview image of the current scene, and then performing feature extraction on the preview image to obtain the brightness characteristic information of the preview image, and then inputting the brightness characteristic information into a strategy generation model to obtain an exposure strategy , wherein the strategy generation model is trained by a neural network according to the brightness feature information corresponding to the sample image, and finally the current scene is photographed based on the above-mentioned exposure strategy. According to the technical solution of the embodiment of the present application, during the preview period of shooting, the embodiment of the present application will extract the brightness feature information of the preview image of the current scene and input it into the trained policy generation model, because the policy generation model of the embodiment of the present application is composed of The neural network is trained based on the brightness feature information corresponding to the sample image. Therefore, the strategy generation model will respond to output the exposure strategy corresponding to the preview image of the current scene, and then the exposure strategy output by the strategy generation model can be adopted when the shooting button is pressed. During the shooting process, because the accuracy of the exposure strategy output by the policy generation model is higher, the embodiment of the present application can improve the shooting quality.

Those skilled in the art can understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware and an appropriate combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit . Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

The above is a specific description of some implementations of the present application, but the present application is not limited to the above-mentioned embodiments. Those skilled in the art can also make various equivalent deformations or replacements without violating the spirit of the present application. Equivalent modifications or replacements are all within the scope defined by the claims of the present application.

Claims (14)

1. A shooting method, comprising:

   Get the preview image of the current scene;

   performing feature extraction on the preview image to obtain brightness feature information of the preview image;

   Inputting the brightness feature information into a strategy generation model to obtain an exposure strategy, wherein the strategy generation model is trained by a neural network according to the brightness feature information corresponding to the sample image;

   Shooting the current scene based on the exposure strategy.
2. The shooting method according to claim 1, wherein said performing feature extraction on said preview image to obtain brightness feature information of said preview image comprises:

   dividing the preview image to obtain a plurality of sub-regions, and determining a plurality of target sub-regions from the plurality of sub-regions;

   For each of the target sub-regions, calculate the cumulative brightness distribution value according to the brightness value of the target sub-region, and obtain the brightness interval ratio of the target sub-region according to the cumulative brightness distribution value and a preset brightness threshold;

   Acquiring the brightness position weight of each target sub-region;

   The brightness characteristic information of the preview image is calculated according to the brightness position weights and the brightness interval ratios of all the target sub-regions.
3. The shooting method according to claim 2, wherein the preset brightness threshold comprises a preset high brightness threshold and a preset low brightness threshold;

   Correspondingly, the obtaining the brightness interval ratio of the target sub-region according to the cumulative brightness distribution value and the preset brightness threshold includes: obtaining the target sub-region according to the cumulative brightness distribution value and the preset highlight threshold the ratio of the high-brightness interval of the region, and the ratio of the low-brightness interval of the target sub-region obtained according to the cumulative brightness distribution value and the preset low-brightness threshold;

   Correspondingly, the calculating according to the brightness position weights and the brightness interval ratios of all the target sub-regions to obtain the brightness feature information of the preview image includes: according to the brightness positions of all the target sub-regions calculating the weight and the ratio of the highlighted interval to obtain the characteristic information of the highlighted interval of the preview image; Bright area feature information.
4. The shooting method according to claim 2, wherein the brightness position weight of each of the target sub-regions is obtained as follows:

   Obtaining distance information between the target sub-region and a preset position in the preview image;

   The brightness position weight of each of the target sub-regions is calculated according to the distance information.
5. The photographing method according to claim 3, wherein the photographing method further comprises: calculating and obtaining a brightness proportion weight according to the brightness interval ratio of the target sub-area;

   Correspondingly, the calculation according to the brightness position weights and the brightness interval ratios of all the target sub-regions to obtain the brightness characteristic information of the preview image includes:

   The brightness feature information of the preview image is calculated according to the brightness position weights, the brightness proportion weights and the brightness interval ratios of all the target sub-regions.
6. The shooting method according to claim 5, wherein, when the number of the sub-areas is M*N, the number of the target sub-areas is i*j, where i≤M, j≤N;

   The value of the cumulative brightness distribution is cdf _i,j , i∈[1,M],j∈[1,N]; when the preset high brightness threshold is [lum3,lum4], the preset low brightness threshold is [ lum1, lum2], wherein, lum4>lum3≥lum2>lum1;

   The highlight interval ratio is obtained by the following formula: L _i,j =cdf _i,j (lum4)-cdf _i,j (lum3), wherein, the L _i,j is the highlight interval ratio;

   The ratio of the low-brightness interval is obtained by the following formula: D _i,j =cdf _i,j (lum2)-cdf _i,j (lum1), wherein, the D _i,j is the ratio of the low-brightness interval.
7. The shooting method according to claim 6, wherein the brightness position weight is obtained by the following formula:

   

   Wherein, the W _d (i, j) is the brightness position weight, and the δ ₁ is an adjustable parameter.
8. The shooting method according to claim 7, wherein the brightness proportion weight is obtained by the following formula:

   

   Wherein, the W _a (lum) is the brightness position weight, and the δ ₂ and the u are adjustable parameters.
9. The shooting method according to claim 8, wherein the feature information of the highlighted interval is obtained by the following formula:

   

   Wherein, the vl _k is the feature information of the highlighted interval;

   The feature information of the low brightness interval is obtained by the following formula:

   

   Wherein, the vd _k is characteristic information of the low brightness interval.
10. The shooting method according to any one of claims 2 to 9, wherein, when the number of the target sub-areas is less than the number of the sub-areas, the target sub-area is located at the center of the preview image or at the The surrounding position of the preview image described above.
11. A photographing device, including a processor, a memory, a photographing component, and a display screen, the processor is respectively connected to the memory, the photographing component, and the display screen, and the processor includes a feature extraction unit and a strategy A generating unit, the photographing component includes an optical camera;

    The photographing component is configured to obtain a preview image corresponding to the current scene through the optical camera;

    The display screen is set to display the preview image corresponding to the current scene;

    The feature extraction unit is configured to acquire the preview image displayed on the display screen, and perform feature extraction on the preview image to obtain brightness feature information of the preview image;

    The strategy generation unit is configured to input the brightness feature information into a pre-trained strategy generation model in the processor to obtain an exposure strategy, wherein the strategy generation model is configured by the neural network according to the brightness corresponding to the sample image The feature information is trained;

    The photographing component is further configured to acquire the exposure strategy, and photograph the current scene through the optical camera based on the exposure strategy.
12. A controller, comprising: a memory, a processor, and a computer program stored on the memory and operable on the processor, when the processor executes the computer program, the claims 1 to 10 are implemented. Any one of the shooting methods described in.
13. A photographing device, comprising the photographing device according to claim 11 or the controller according to claim 12.
14. A computer-readable storage medium, wherein computer-executable instructions are stored, and the computer-executable instructions are used to execute the photographing method according to any one of claims 1-10.

Images