CN111915529A

CN111915529A - Video dim light enhancement method and device, mobile terminal and storage medium

Info

Publication number: CN111915529A
Application number: CN202010780015.XA
Authority: CN
Inventors: 杨敏
Original assignee: Guangzhou Baiguoyuan Information Technology Co Ltd
Current assignee: Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date: 2020-08-05
Filing date: 2020-08-05
Publication date: 2020-11-10

Abstract

The embodiment of the invention provides a method, a device, a mobile terminal and a storage medium for enhancing the dim light of a video, wherein the method comprises the following steps: collecting video data; collecting parameters related to dim light enhancement processing in the environment where the mobile terminal is located, and taking the parameters as environment parameters; collecting parameters matched with environmental parameters and related to dim light enhancement processing in the mobile terminal as auxiliary parameters; and setting the state of executing the dim light enhancement processing on the video data in the environment by combining the environment parameter and the auxiliary parameter. On one hand, under the condition that the environment allows, the normal execution of the dim light enhancement processing is ensured, so that the quality of video data is ensured, on the other hand, the execution of the dim light enhancement processing is forbidden in an improper environment, the frequency of the dim light enhancement processing is reduced, the computing power of the dim light enhancement processing is saved, the occupation of resources is reduced, more computing power is reserved, the normal execution of business operation is ensured, the flexibility degree of the dim light enhancement processing is finally improved, and the robustness of the dim light enhancement processing is improved.

Description

Video dim light enhancement method and device, mobile terminal and storage medium

Technical Field

The embodiment of the invention relates to the technical field of computer vision, in particular to a method and a device for enhancing the dim light of a video, a mobile terminal and a storage medium.

Background

With the rapid development of the mobile internet and the mobile terminal, the video data in the mobile terminal has become a common information carrier in human activities, such as live broadcast, video call, etc., and they contain a lot of information of objects, which becomes one of the ways for people to obtain the external original information.

When a user shoots video data in a dark light environment, the image data in the video data can present the problems of poor contrast, low visibility and the like, the appearance of the user is influenced, and due to poor quality of the image data, the performance of subsequent image processing, such as beauty and face detection, can be reduced, and finally the quality of the video data is seriously reduced.

At present, for video data shot in a dim light environment, the brightness of the outside is generally estimated, so that a proper parameter is determined for dim light enhancement, the dim light enhancement process is continuously performed, the consumed calculation power is high, and the robustness of dim light enhancement is low under the condition that the performance of a mobile terminal is limited.

Disclosure of Invention

The embodiment of the invention provides a method and a device for enhancing the dim light of a video, a mobile terminal and a storage medium, aiming at solving the problem of how to improve the robustness of the dim light enhancement processing when the performance of the video data is limited to be enhanced.

In a first aspect, an embodiment of the present invention provides a method for enhancing dim light of a video, which is applied in a mobile terminal, and the method includes:

collecting video data;

collecting parameters related to dim light enhancement processing in the environment where the mobile terminal is located, and taking the parameters as environment parameters;

collecting parameters matched with the environmental parameters and related to the dim light enhancement processing in the mobile terminal as auxiliary parameters;

setting a state of performing the dim-light enhancement processing on the video data in the environment in combination with the environment parameter and the auxiliary parameter.

In a second aspect, an embodiment of the present invention further provides a device for enhancing dim light of a video, which is applied in a mobile terminal, and the device includes:

the video data acquisition module is used for acquiring video data;

the environment parameter acquisition module is used for acquiring parameters related to the dim light enhancement processing in the environment where the mobile terminal is located, and the parameters are used as environment parameters;

the auxiliary parameter acquisition module is used for acquiring parameters which are matched with the environmental parameters and are related to the dim light enhancement processing in the mobile terminal as auxiliary parameters;

and the dim light enhancement decision module is used for combining the environment parameter and the auxiliary parameter and setting the state of executing the dim light enhancement processing on the video data in the environment.

In a third aspect, an embodiment of the present invention further provides a mobile terminal, where the mobile terminal includes:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method for dim-light enhancement of video according to the first aspect.

In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for enhancing the dim light of a video according to the first aspect.

In this embodiment, video data is collected, parameters related to dim light enhancement processing are collected in the environment where the mobile terminal is located, the parameters are used as environment parameters, parameters matched with the environment parameters and related to dim light enhancement processing are collected in the mobile terminal, the parameters are used as auxiliary parameters, the state of performing dim light enhancement processing on the video data in the environment is set in combination with the environment parameters and the auxiliary parameters, two or more than two items of environment parameters and auxiliary parameters are combined to perform decision on the dim light enhancement processing, the accuracy of the decision is improved, the phenomena of too bright and the like caused by performing the dim light enhancement processing under unfavorable conditions are prevented, on one hand, the normal execution of the dim light enhancement processing is ensured under the condition that the environment allows, so that the quality of the video data is ensured, on the other hand, the execution of the dim light enhancement processing is prohibited in the unsuitable environment, the frequency of the dim light enhancement processing is reduced, the computing power of the dim light enhancement processing is saved, the occupation of resources such as a CPU (central processing unit), a memory and the like is reduced, more computing power is reserved to ensure the normal execution of business operation, the flexibility degree of the dim light enhancement processing is finally improved, and the robustness of the dim light enhancement processing is improved.

Drawings

Fig. 1 is a flowchart of a method for enhancing dim light of a video according to an embodiment of the present invention;

FIG. 2 is a block diagram of a dim light enhancement decision making framework according to an embodiment of the present invention;

FIG. 3 is a comparison between video data and environmental parameters according to an embodiment of the present invention;

fig. 4 is a flowchart of a method for enhancing dim light of a video according to a second embodiment of the present invention;

fig. 5 is a schematic diagram of a Retinex according to a second embodiment of the present invention;

fig. 6 is a schematic structural diagram of a video dim light enhancement device according to a third embodiment of the present invention;

fig. 7 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a flowchart of a video dim light enhancement method according to an embodiment of the present invention, where the embodiment is applicable to a situation that whether dim light enhancement processing is performed or not according to an environment of a mobile terminal and a decision of matching, and the method can be executed by a video dim light enhancement device, where the video dim light enhancement device can be implemented by software and/or hardware, and can be configured in a mobile terminal, for example, a mobile phone, a tablet computer, and an intelligent wearable device (e.g., a smart watch, smart glasses, etc.), and the method specifically includes the following steps:

step 101, collecting video data.

In this embodiment, the mobile terminal is configured with one or more cameras (cameras) for taking and recording pictures, where the Camera may be disposed on the back of the mobile terminal (also referred to as a rear Camera) or disposed on the front of the mobile terminal (also referred to as a front Camera), which is also not limited in this embodiment of the present invention.

In addition, the operating system of the mobile terminal includes Android (Android), iOS, Windows, and the like, and may support the running of various applications, such as a shopping application, an instant messaging application, a live application, a video conference application, a short video application, and the like.

As shown in fig. 2, in S201, the applications perform service operations, in S202, in response to the service operations, video data is acquired by calling a camera, for example, in an Android system, MediaRecorder may be called to acquire video data, in an iOS system, avcapturefileoutputrecording delete under AVFoundation may be called to acquire video data, and so on.

It should be noted that, the service operations performed by the video data collected by these applications are different due to different service scenes.

Generally, the video data collected by these applications can be used to undertake business operations with real-time performance.

For example, in a business scenario of shopping, video data collected by a shopping application can be used for carrying business operations of selling commodities, i.e. an anchor user explains, demonstrates and the like the commodities, and provides links of shopping in the video data.

For another example, in an instant messaging service scenario, video data collected by an instant messaging tool may be used to carry service operations of a video call, that is, a multiparty user of a communication performs a conversation.

For another example, in a live service scenario, video data acquired by a live application may be used to carry live service operations, that is, a main user performs talent performance, game commentary, and the like.

For another example, in a service scene of a conference, video data collected by a video conference application may be used to carry service operations of the conference, that is, a plurality of users participating in the conference speak in turn as hosts.

Of course, the video data collected by these applications may also bear business operations with lower real-time requirements, for example, the short video application collects the video data as a short video, and the like, which is not limited in this embodiment.

And 102, collecting parameters related to dim light enhancement processing in the environment where the mobile terminal is located, wherein the parameters are used as environment parameters.

In the mobile terminal, one or more sensors such as an acceleration sensor, a magnetic force sensor, a gravity sensor, a temperature sensor, a light sensor, a color temperature sensor, a gyroscope, a pressure sensor, and the like are provided

In this embodiment, as shown in fig. 2, while the mobile terminal collects the video data, in S203, for an environment where the mobile terminal is located, a corresponding sensor may be called to collect a parameter related to the dim light enhancement processing, as an environmental parameter, that is, the environmental parameter may cause a positive and/or negative influence on the dim light enhancement processing.

Further, the environment of the mobile terminal includes an external environment and an internal environment, where the external environment may refer to an environment outside the mobile terminal, in the external environment, the light sensor may be used to detect the brightness value, the color temperature sensor may be used to detect the color temperature value, the pressure sensor may be used to detect the atmospheric pressure value, and the like, and the internal environment may refer to an environment inside the mobile terminal, in the internal environment, the temperature sensor may be used to detect the temperature value, and the like.

In one example, as shown in FIG. 2, the environmental parameters associated with the dim light enhancement process include at least one of:

1. first brightness value in external environment of mobile terminal

In an external environment with strong light, the brightness value of the acquired video data is high, the noise level is low, and if the dim light enhancement Processing is performed at this time, the occupation of resources such as a Central Processing Unit (CPU) and a memory in the mobile terminal is increased, but there is no actual benefit.

In this example, if a sensor for detecting light, such as a light sensor, is disposed in the mobile terminal and an Application Programming Interface (API) is opened, the API may be directly called and the sensor may be read to detect the brightness value of light in the external environment as the first brightness value.

For example, in the Android system, a first brightness value of LIGHT may be detected by a SensorManager register listening LIGHT intensity sensor (LIGHT).

2. A temperature value in an internal environment in which the mobile terminal is located.

The mobile terminal collects video data for a long time to carry out operations such as video call, video conference and the like, and the temperature is increased due to the overweight load, so that measures of the mobile terminal for reducing the frequency of the CPU are triggered, the real-time coding speed of the video data is influenced by the frequency reduction of the CPU, and the business operation is further seriously influenced.

Therefore, in the process of increasing the temperature of the mobile terminal, if the dim light enhancement processing is performed at this time, the occupation of resources such as a CPU and a memory in the mobile terminal is increased by related work, and negative influence is generated on the service operation.

In a specific implementation, a command of the operating system, such as "cat/sys/class/thermal/thermal _ zonex/temp" in the Android system, may be directly called to read a temperature value of an internal environment where the mobile terminal is located.

Furthermore, there are multiple devices inside the mobile terminal, which can detect temperature, such as the temperature of the CPU, the temperature value of the battery, and the like, at this time, the temperature of one device (such as the CPU) may be designated as the temperature value of the internal environment of the mobile terminal, the temperature values of all the devices may also be calculated in a linear fusion manner to obtain the temperature value of the internal environment of the mobile terminal, the temperature value with the maximum value may also be used as the temperature of the internal environment of the mobile terminal, and the like, which is not limited in this embodiment.

Of course, the above-mentioned environmental parameters and the collection manner thereof are only examples, and when the embodiment is implemented, other environmental parameters and the collection manner thereof may be set according to actual situations, for example, a color temperature of an external environment where the mobile terminal is located, and the like, which is not limited in this embodiment. In addition, besides the above environmental parameters and the collection method thereof, those skilled in the art may also adopt other environmental parameters and collection methods thereof according to actual needs, which is not limited in this embodiment.

In practical application, the frame rate of the mobile terminal for acquiring the video data is different from the frame rate of the sensor for acquiring the environmental parameters, and the frame rate for acquiring the environmental parameters is generally greater than the frame rate for acquiring the video data, that is, a plurality of environmental parameters generally exist between every two frames of image data in the video data.

For example, as shown in fig. 3, the frame rate of the mobile terminal acquiring the video data is lower than the frame rate of the sensor acquiring the environment parameter, and there are 7 frames of environment parameters between the 1 st frame of video data and the 2 nd frame of video data.

Therefore, in order to ensure the stability of the denoising process and the environmental parameters, in the present embodiment, a segmentation method may be adopted to align the image data and the environmental parameters.

In a specific implementation, an environmental parameter acquired between current frame image data and previous frame image data is acquired as a reference parameter, so that the environmental parameter is segmented into a plurality of segments by taking video data as a reference.

For each section of environment parameter, the reference parameter can be smoothed to serve as the environment parameter corresponding to the current frame image data, so that the difference of correlation between the image data at different time points on the time sequence and the environment parameter is highlighted, the environment parameter is subsequently used for performing dim light enhancement processing on the frame image data, and the quality of the dim light enhancement processing is improved.

In one way of smoothing, the reference parameter may be configured with an environmental weight, which may reflect the importance of the reference parameter, i.e., the more important the reference parameter, the greater the environmental weight.

In one example, the time for acquiring the reference parameter may be determined, and an environment weight is set for the reference parameter based on the time, where the environment weight is positively correlated with the time, that is, the closer the time for acquiring the reference parameter is to the current time, the stronger the correlation between the reference parameter and the current frame image data is, the larger the configured environment weight is, and conversely, the worse the correlation between the reference parameter and the current frame image data is, the smaller the configured environment weight is, the more the time for acquiring the reference parameter is from the current frame image data to the previous frame image data is, so that the environment weight is monotonically decreased from the current frame image data to the previous frame image data.

Of course, besides the acquisition time, the environmental weight may be configured in other manners, for example, the reciprocal of the number of the reference parameters is used as the environmental weight, and the like, which is not limited in this embodiment.

After each reference parameter is configured with a corresponding environment weight, a first product between the reference parameter and the environment weight may be calculated, and a ratio between a first target value and a second target value may be calculated as the environment parameter corresponding to the current frame image data, where the first target value is a sum of all the first products, and the second target value is a sum of all the environment weights, then the environment parameter is expressed as follows:

wherein the content of the first and second substances,

for the environmental parameter corresponding to the t-th frame image data, n environmental parameters (reference parameters) exist between the t-th frame image data and the t-1 th frame image data, y_t-iFor the i-th environmental parameter, w, preceding the t-th frame of image data_iIs the ith environmental weight.

And 103, collecting parameters which are matched with the environmental parameters and are related to the dim light enhancement processing in the mobile terminal as auxiliary parameters.

In a specific implementation, the dim light enhancement processing at least includes brightening processing, that is, increasing the brightness of the video data, where the brightening processing may cause great damage to the picture in the video data and great influence on the image quality in the video data, for example, in the case of high brightness in the video data, the dim light enhancement processing is performed on the video data, which may result in the picture being too bright.

Therefore, in this embodiment, as shown in fig. 2, in S204, in addition to invoking the sensor to collect the environment parameter related to the dim light enhancement processing, in the process of performing the service operation by the mobile terminal, a parameter matching the environment parameter and related to the dim light enhancement processing may be collected as an auxiliary parameter, that is, the auxiliary parameter may cause a positive and/or negative impact on the dim light enhancement processing, and may act on the decision of the dim light enhancement processing together with the environment parameter.

In one example, as shown in fig. 2, the auxiliary parameters matched with the environmental parameters include at least one of:

1. target brightness value and brightness change value of video data

For a first brightness value in an external environment where the mobile terminal is located, a target brightness value and a brightness change value of the video data can be matched as auxiliary parameters.

In a specific implementation, in consideration of the credibility of the video data collected by turning on the camera, N frames of continuous image data may be selected from the video data from the K-th frame, and the statistics of the luminance information (i.e. the target luminance value, the luminance change value) is performed in the N frames of image data, i.e. the statistics of the luminance information is increased, thereby improving the accuracy of the luminance information.

For a single frame of image data, the luminance values of N frames of image data may be counted respectively as a second luminance value.

Further, to improve the accuracy of the second luminance, a plurality of (two or more) anchor points may be set at positions specified in the image data for each frame of the image data.

For example, if the resolution of the video data is H × W, anchor points may be set at the center, such as (W × 0.5, H × 0.5), or at the edges, such as (W × 0.25, H × 0.25), (W × 0.25, H × 0.75), (W × 0.75, H × 0.25), (W × 0.75, H × 0.75), and so on.

For each anchor point, a range containing the anchor point is determined in the image data as an image block.

For example, a circular image block is generated with an anchor point as the center and a designated value R (e.g., 32) as the radius, and with respect to an anchor point of an edge, if the distance between the anchor point and the edge (e.g., min (W, H) × 0.25) is smaller than R, a value smaller than the distance between the anchor point and the edge (e.g., min (W, H) × 0.25-1) is used as the radius.

For another example, a rectangle of a specified side length is determined as the image block with the anchor point as a center point.

For another example, an ellipse with a long side and a short side specified is determined as an image block with the anchor point as a center point.

And counting the brightness value of each image block to serve as a block brightness value, and configuring a position weight for the block brightness value according to the position, wherein the position weight is decreased from the position at the center to the position at the edge, namely the position weight of the center is greater than that of the edge.

Second products between the block luminance values and the position weights are calculated, thereby calculating a sum value between all the second products as second luminance values of the image data.

And calculating values such as an average value, a median and the like based on the second brightness value, and setting the values as target brightness values to serve as auxiliary parameters matched with the first brightness value.

Further, the variance, standard deviation, or the like of the second luminance value is calculated and set as a luminance variation value indicating the degree of variation of the second luminance value in the N-frame image data as an auxiliary parameter matching the first luminance value.

For the variance, a square of a luminance difference, which is a difference between the second luminance value and the target luminance value, may be calculated as a single frame change value.

The average value between the single-frame change values is calculated as the luminance change value.

Taking the average value of the second brightness value as the target brightness value and the variance as the brightness change value as an example, for the image data of N frames starting from the K-th frame, N images are set in one frame of image dataBlock, target luminance value mu_IBrightness variation value_IThe expression is as follows:

wherein, mu_tiIndicating the luminance value, alpha, of the i-th image block in the t-th frame image data_iRepresenting the weight of the ith image block.

2. Time occupied by dim light enhancement treatment

For the temperature value in the internal environment where the mobile terminal is located, the time occupied by the dim light enhancement processing can be used as an auxiliary parameter.

In a specific implementation, the time occupied by performing the dim light enhancement processing on the video data in advance may be counted for the current service operation, and used as an auxiliary parameter matched with the temperature value.

Further, at the initial stage of the current service operation, if the dim light enhancement processing is not executed, the time occupied by the dim light enhancement processing may be set in an estimated manner, for example, the time occupied by the dim light enhancement processing in the mobile terminal of the same model is counted through experiments, the time occupied by the dim light enhancement processing executed on the video data in advance is counted, the time is set as the default time occupied by the mobile terminal of the model to execute the dim light enhancement processing, and for example, when the mobile terminal executes other service operations, the time occupied by the dim light enhancement processing executed on the video data in advance is counted, the time is set as the time occupied by the dim light enhancement processing executed in the current service operation, and the like.

When the current business operation has executed the dim light enhancement processing, the ratio between the number and the time period can be set as the time occupied by executing the dim light enhancement processing on one frame of image data by counting the number of image data of the dim light enhancement processing in the time period, that is, the time occupied by executing the dim light enhancement processing on one frame of image data is the average time of executing the dim light enhancement processing.

Of course, the auxiliary parameters and the calculation method thereof are only examples, and when the embodiment is implemented, other auxiliary parameters and calculation methods thereof, for example, CPU occupancy, memory occupancy, etc., may be set according to the situation of the actual environment parameters, which is not limited in this embodiment. In addition, besides the above auxiliary parameters and the calculation methods thereof, those skilled in the art may also adopt other auxiliary parameters and calculation methods thereof according to actual needs, which is not limited in this embodiment.

And 104, combining the environmental parameters and the auxiliary parameters, and setting the state of performing the dim light enhancement processing on the video data in the environment.

In this embodiment, as shown in fig. 2, in S205, the environmental parameters related to the dim light enhancement processing and matched with each other are combined with the auxiliary parameters, and the degree of adaptation between the current environment of the mobile terminal and the dim light enhancement processing is estimated, so as to make a decision based on the degree of adaptation, determine the state of performing the dim light enhancement processing on the video data, that is, determine whether to perform the dim light enhancement processing in S206, when a certain condition is satisfied, the execution of the dim light enhancement processing on the video data is allowed, and when the condition is not satisfied, the execution of the dim light enhancement processing on the video data is prohibited.

In an embodiment of the present invention, the mutually matched environment parameter and auxiliary parameter are a first luminance value in an external environment of the mobile terminal, a target luminance value of the video data, and a luminance variation value, in this embodiment, the first luminance value and the target luminance value may be fused to obtain an overall luminance value, where the overall luminance value is positively correlated with the first luminance value and the target luminance value, that is, the higher the first luminance value is, the higher the overall luminance value is, and conversely, the lower the first luminance value is, the lower the overall luminance value is, and similarly, the higher the target luminance value is, the higher the overall luminance value is, and conversely, the lower the target luminance value is, the lower the overall luminance value is.

In one example of fusion, the fusion is a weighted average, that is, a first luminance weight is assigned to the first luminance value, a second luminance weight is assigned to the target luminance value, and a sum value between a second product, which is a product between the first luminance value and the first luminance weight, and a third product, which is a product between the target luminance value and the second luminance weight, is calculated.

And substituting the sum value into a preset linear function to calculate the overall brightness value, so that the overall brightness value is positively correlated with the first brightness value and the second brightness weight.

In this example, the relationship between the overall luminance value and the first and second luminance values and the second luminance weight is expressed as follows:

S_t＝f(L′_t,μ_I；α,β,k,b)＝k*(α*μ_I+β*L′_t)+b

wherein S is_tIs the overall brightness value, mu_IIs a target luminance value, L'_tIs the first luminance value, α is the second luminance weight, β is the first luminance weight, and k () + b is a linear function.

Respectively comparing the brightness variation value with a first threshold value tau₁Comparing the overall brightness value with a second threshold value tau₂A comparison is made.

If the brightness variation value is less than or equal to the first threshold value tau₁And the overall brightness value is less than or equal to the second threshold value tau₂The mobile terminal is in a state of dark external environment and stable light, and the dim light enhancement processing is allowed to be executed on the video data according to the external environment of the mobile terminal.

If the target brightness value is larger than the first threshold value tau₁And/or the overall brightness value is greater than a second threshold value tau₂If the external environment light intensity is higher and/or the light intensity is more fluctuant, the mobile terminal is in the state, and the dark light enhancement processing is forbidden to be executed on the video data aiming at the external environment where the mobile terminal is located.

In another embodiment of the present invention, the mutually matched environment parameter and auxiliary parameter are a temperature value in an internal environment where the mobile terminal is located and time occupied by the dim light enhancement processing, and in this embodiment, the temperature value and the time can be linearly fused to obtain a performance consumption value, where the performance consumption value is positively correlated with the temperature value and the time, that is, the higher the temperature value is, the larger the performance consumption value is, and conversely, the lower the temperature value is, the smaller the performance consumption value is, and in the same way, the longer the time is, the larger the performance consumption value is, and conversely, the shorter the time is, the smaller the performance consumption value is.

In a specific implementation, a first coefficient of performance is configured for time and a second coefficient of performance is configured for temperature values.

And calculating the sum value of a fourth product, a fifth product and a preset third performance coefficient as the performance consumption value, wherein the fourth product is the product between time and the first performance coefficient, and the fifth product is the product between the temperature value and the second performance coefficient.

That is, the relationship between the performance consumption value and the temperature value and time is expressed as follows:

f(μ_T,T′_t)＝a·μ_T+b·T′_t+c

wherein f () is a linear function, a is a first coefficient of performance, b is a second coefficient of performance, c is a third coefficient of performance, μ_TIs time, T'_tIs a temperature value.

Comparing the performance consumption value with a preset third threshold value tau₃A comparison is made.

If the performance consumption value is greater than or equal to the third threshold value tau₃If so, it means that the performance consumption of the mobile terminal is high and the influence on the current service operation is large when the dim light enhancement processing is performed, and the dim light enhancement processing is prohibited from being performed on the video data according to the internal environment of the mobile terminal.

If the performance consumption value is greater than or equal to the third threshold value tau₃If the mobile terminal is in the dark light enhancement mode, the mobile terminal is subjected to dark light enhancement processing, the performance consumption of the mobile terminal is low, the influence on the current business operation is small, the dark light enhancement processing is supported in the performance aspect, and the temperature value in the internal environment where the mobile terminal is located and the time occupied by the dark light enhancement processing can be continuously used for detection and decision making in other modes in an auxiliary mode.

Of course, the above-mentioned manner of deciding the state of the dim light enhancement processing is only an example, and when implementing the embodiment, other manners of deciding the state of the dim light enhancement processing may be set according to the situations of the actual environment parameter and the auxiliary parameter, which is not limited in this embodiment. In addition, besides the above-mentioned manner of deciding the state of the dim light enhancement processing, a person skilled in the art may also adopt other manners of deciding the state of the dim light enhancement processing according to actual needs, and this embodiment is not limited to this.

Example two

Fig. 4 is a flowchart of a video dim light enhancement method according to a second embodiment of the present invention, where the present embodiment further adds operations of dim light enhancement processing based on the foregoing embodiments, and the method specifically includes the following steps:

step 401, collecting video data.

Step 402, collecting parameters related to the dim light enhancement processing in the environment where the mobile terminal is located, and using the parameters as environment parameters.

And 403, collecting parameters which are matched with the environmental parameters and are related to the dim light enhancement processing in the mobile terminal as auxiliary parameters.

And step 404, setting a state of performing dim light enhancement processing on the video data in the environment by combining the environment parameter and the auxiliary parameter.

Step 405, if the dim light enhancement processing is allowed to be performed on the video data, selecting the environmental parameter and the auxiliary parameter participating in the dim light enhancement processing as the target parameters.

In this embodiment, when it is determined that the video data is currently allowed to be subjected to the dim light enhancement processing, some of the previously collected environment parameters and auxiliary parameters may be selected as target parameters, and the target parameters may be transmitted to a code corresponding to the dim light enhancement processing in a form of JNI (Java Native Interface, Java local Interface) or the like.

In an example, if the previously collected environment parameters and auxiliary parameters include a first brightness value in an external environment where the mobile terminal is located, a target brightness value and a brightness variation value of the video data, a temperature value in an internal environment where the mobile terminal is located, and a time occupied by the dim light enhancement processing, at this time, the first brightness value in the external environment where the mobile terminal is located and the target brightness value of the video data may be selected to participate in the dim light enhancement processing, that is, the target parameters include a first brightness value in the external environment where the mobile terminal is located and a target brightness value of the video data, where the first brightness value in the external environment where the mobile terminal is located, the target brightness value and the brightness variation value of the video data may serve as a priori functions, the dim light denoising processing is adjusted so that the brightness of the video data after the dim light denoising processing is adapted to the current environment brightness, and the first brightness value in the external environment of the mobile terminal is smaller, the global noise of the collected video data is increased along with the increase of the global noise, and the method can adapt to adjustment of dim light enhancement processing.

Of course, besides the first luminance value in the external environment where the mobile terminal is located, the target luminance value and the luminance variation value of the video data, other environment parameters and auxiliary parameters may be selected as the target parameters, for example, the motion intensity of the mobile terminal in the external environment may be selected to participate in the denoising process, where the motion intensity of the mobile terminal in the external environment may be used as a priori, if the current motion of the mobile terminal is too strong, the global motion vector of the acquired video data is too large, and when the global motion vector of human eyes is too large, the human eyes are not sensitive to noise, and are adaptable to adjust the dim light enhancement process, and the like.

Step 406, calculating an adjustment parameter based on the target parameter.

In this embodiment, the parameters of the dim light enhancement process may be adjusted according to the target parameters, so that the dim light enhancement process is adapted to the current environment.

In a particular implementation, an adjustment parameter may be calculated based on the target parameter, the adjustment parameter being used to adjust the intensity of the dim light enhancement process.

In general, for a highlight, the intensity of the dim light enhancement treatment is inversely related to the conditioning parameter, i.e. the smaller the conditioning parameter, the greater the intensity of the dim light enhancement treatment, whereas the larger the conditioning parameter, the less the intensity of the dim light enhancement treatment.

For the denoising part, the intensity of the dim light enhancement treatment is positively correlated with the adjusting parameter, namely the larger the adjusting parameter is, the larger the intensity of the dim light enhancement treatment is, and conversely, the smaller the adjusting parameter is, the smaller the intensity of the dim light enhancement treatment is.

For example, for a first luminance value in an external environment where the mobile terminal is located and a target luminance value of video data, if the first luminance value and the target luminance value are larger and the luminance of light in the external environment of the mobile terminal is lower, a dark light enhancement process with higher intensity can be performed on the video data, so that not only can the luminance of a picture be recovered, but also a code rate of current video data can be compressed, and a pressure of network transmission is relieved, so that a picture blocking phenomenon possibly caused in a network transmission process is reduced.

In one example, the overall brightness value used in the previous decision may be reused as the adjustment parameter, that is, the adjustment parameter is the overall brightness value after the first brightness value is fused with the target brightness value, and the calculation amount is reduced under the condition that the effect of the dim light enhancement processing is ensured.

Further, a first luminance weight is assigned to the first luminance value, a second luminance weight is assigned to the target luminance value, and a sum value between a second product, which is a product between the first luminance value and the first luminance weight, and a third product, which is a product between the target luminance value and the second luminance weight, is calculated.

The sum is substituted into a preset linear function to calculate the overall brightness value as the intensity of the dim light enhancement process.

And 407, performing dim light enhancement processing on the video data according to the adjusting parameters.

After the adjustment parameter of the current dim light enhancement processing is determined, the relevant parameter characterizing the intensity in the dim light enhancement processing may be adjusted according to the adjustment parameter, so as to perform the dim light enhancement processing on the video data.

In this embodiment, if it is allowed to perform the dim light enhancement processing on the video data, the environmental parameter and the auxiliary parameter participating in the dim light enhancement processing are selected as target parameters, an adjustment parameter is calculated based on the target parameters, the adjustment parameter is used to adjust the intensity of the dim light enhancement processing, and the dim light enhancement processing is performed on the video data according to the adjustment parameter, so that the dim light enhancement processing is adapted to the environment, and the utilization efficiency of computational power is improved.

In the environment of ultra-dim light (0-50lux), the dim light enhancement processing generally includes brightening and denoising the pictures of the video data, and the intensity can be used for adjusting the brightening and denoising parameters.

In this case, step 407 may include the steps of:

step 4071 generates raw illumination image data based on image data in the video data.

In a specific implementation, LOG conversion may be performed on luminance image data from luminance image data in a luminance channel (e.g., a Y channel in a YUV format, an L channel in an HSL format) in the image data, and Filter processing (e.g., Fast Guide Filter (FGF), Bilateral Filter (BF), Edge Preserving Filter (EPF), etc.) may be performed on the luminance image data, and inverse LOG conversion may be performed to obtain original illumination image data.

Step 4072, increasing the brightness of the original illumination image data according to the adjustment parameter to obtain the target illumination image data.

In this embodiment, the original illumination image data is subjected to brightness enhancement processing according to the intensity, so as to increase the brightness, and obtain the target illumination image data, wherein the adjustment parameter is inversely related to the increase of the brightness, that is, the larger the adjustment parameter is, the smaller the increase of the brightness is, and conversely, the smaller the adjustment parameter is, the larger the increase of the brightness is.

In one brightening mode, the adjustment parameter is substituted into a preset mapping function, so that the gamma coefficient is calculated based on the adjustment parameter, wherein the gamma coefficient is inversely related to the adjustment parameter, namely the gamma coefficient is smaller when the adjustment parameter is larger, and vice versa.

And carrying out gamma correction on the original illumination image data according to the gamma coefficient so as to improve the brightness value of each pixel point in the original illumination image data as target illumination image data.

The gamma correction is to edit a gamma curve of the original illumination image data to perform nonlinear tone editing on the original illumination image data, detect a part with high gray scale and a part with low gray scale in the original illumination image data, and increase the proportion of the two parts, thereby improving the contrast of the original illumination image data^γI denotes original illumination image data, γ denotes a gamma coefficient of gamma correction, and I' denotes target illumination image data after gamma correction.

The gamma value is divided by 1, the smaller the value is, the stronger the expansion effect on the low gray part of the original illumination image data is, the larger the value is, the stronger the expansion effect on the high gray part of the original illumination image data is, and the effect of enhancing the details of the low gray part or the high gray part can be achieved through different gamma values.

Step 4073, if the image data is the superposition between the original illumination image data and the reflection image data, synthesizing the target illumination image data and the reflection image data into the characteristic image data.

Retinex belongs to the group of synthetic words consisting of Retina (Retina) and Cortex, which are based on:

first, the real world is colorless, and the perceived color is the result of the interaction of light with matter.

And each color zone is composed of three primary colors of red, green and blue with given wavelengths.

And thirdly, the three primary colors determine the color of each unit area.

In Retinex theory, the color of an object is determined by the reflection ability of the object for long-wave (red), medium-wave (green), and short-wave (blue) light, rather than the absolute value of the intensity of the reflected light, and the color of the object is not affected by illumination non-uniformity and has uniformity.

That is, Retinex theory is based on color sense uniformity (color constancy), which is the ability of human eyes to recognize the original color of an object even at different brightness levels.

As shown in fig. 5, in Retinex theory, image data obtained by the human eye depends on incident light and the reflection of the incident light by the surface of an object. The image data is first illuminated by the incident light and reflected by the object into the imaging system to form what is seen. In this process, the reflectivity is determined by the object itself, is not affected by the incident light, and can be expressed by the following formula:

L＝I·T

where L represents image data received by an observed or camera, I represents an illumination component of ambient light, i.e., illumination image data, and T represents a reflection component of a target object carrying image detail information, i.e., reflection image data.

In this embodiment, the original illumination image data I determines a dynamic range that each pixel in a frame of image data can reach, the reflection image data T determines an intrinsic property of the image data, and after the original illumination image data I in the image data L is determined, the Retinex model is run, the reflection image data is resolved from the image data with reference to the original illumination image data, that is, the property of the original illumination image data I is removed from the image data L, so as to separate an original appearance of an object, obtain the reflection image data T, and eliminate an influence of illumination unevenness.

The image data is in a first color space, and the Retinex model decomposed reflected image data is usually in a second color space, so the original image data can be converted from the first color space to the second color space by a specified conversion relation.

In order to make those skilled in the art better understand the present embodiment, in the present embodiment, a YUV color space is taken as an example of the first color space, and an RGB color space is taken as an example of the second color space.

For example, image data may be converted from YUV color space to RGB color space by converting the relationship:

R＝Y+1.4075*(V-128)

G＝Y-0.3455*(U-128)-0.7169*(V-128)

B＝Y+1.779*(U-128)

and traversing each pixel point in the image data and the original illumination image data, and respectively determining the original color component of the pixel point expressed in the second color space and the brightness value of the pixel point expressed in the first color space aiming at the pixel point at the same position.

And calculating the ratio of the original color component to the brightness value as the reflection color component represented by the pixel points in the reflection image data.

For example, the reflected color components R ' G ' B ' represented by pixel points in the reflected image data are represented as follows:

R＇＝R/Y

G＇＝G/Y

B＇＝B/Y

where RGB represents the original color components in the original image data and Y represents the luminance values.

In the Retinex model, target illumination image data after brightness is increased and reflection image data are synthesized to obtain characteristic image data, and the process is represented as follows:

I＇＝I^γ

L＇＝I＇·T

wherein I denotes original illumination image data, I^γRepresenting the original illumination image data I highlighted,i 'denotes target illumination image data after brightness enhancement, T denotes reflection image data, and L' denotes feature image data.

Under the framework of the Retinex model, the original illumination image data is brightened, and the low-brightness image data can be brightened under the condition of ensuring the real color.

Thereafter, the feature image data may be converted from the second color space to the first color space by the specified conversion relation for subsequent processing.

For example, the feature image data may be converted from an RGB color space to a YUV color space by the following conversion relation:

Y＝0.299*R+0.587*G+0.114*B

the feature image data is converted into a second color space such as RGB, and the like, so that the feature image data after being brightened can be maximally ensured not to generate color difference.

Step 4074, filtering the feature image data with the target illumination image data as the guide image data.

In this embodiment, a non-deep learning method may be applied, and the target illumination image data is used as a reference for denoising processing, and the characteristic image data is denoised by filtering or the like.

Further, the protection of the feature image data by guiding the edge protection operation of the image data during the filtering may specifically include the following manners:

1. guided filtering

The guided filtering filters the input feature image data P by using the guide image data R to obtain the denoised image data Q, and the mathematical formula of the guided filtering is expressed as:

wherein i represents a pixel point, j represents a filtering window, W_ij(R) denotes a weight value employed in the weighted average operation determined by the guide image data R, which is illumination image data separated from the feature image data P and highlighted (i.e., target illumination image data).

2. Joint bilateral filtering

The joint bilateral filtering utilizes the guide image data R to filter the input feature image data, so as to obtain the denoised image data J, and if the guide image data R is illumination image data (i.e. target illumination image data) separated and brightened by the input feature image data, the expression is as follows:

wherein, p represents a pixel point, q represents a filtering window, f (-) is a spatial filter, the weight is calculated by using the distance between the current pixel point and the surrounding pixel points, and g (-) is a range filter, which represents the difference between the pixel values of the current pixel point and the surrounding pixel points of the guide image data to calculate the weight. k is a radical of_pThe edge preserving method is a normalization parameter, and when the difference between the distance and the pixel value is large, the product of the distance and the pixel value is small, so that the edge preserving operation is realized.

In the dark light (50lux-300lux) environment, the dark light enhancement process generally includes brightening the picture of the video data, which intensity can be used to adjust the brightness enhancement parameters.

Specifically, original illumination image data is generated based on image data in video data; improving the brightness of the original illumination image data according to the adjusting parameters to obtain target illumination image data; and if the image data is the superposition between the original illumination image data and the reflection image data, synthesizing the target illumination image data and the reflection image data into characteristic image data.

Of course, the above-mentioned dim light enhancement processing by applying the Retinex model is only an example, and when the embodiment is implemented, other dim light enhancement processing may be set according to actual situations, for example, based on histogram equalization, and the like, which is not limited in this embodiment. In addition to the above-mentioned dark light enhancement treatment, a person skilled in the art may also use other dark light enhancement treatments according to actual needs, and this embodiment is not limited to this.

In this embodiment, the illumination image data is generated based on the image data, the calculation amount is small, the quality is good, the illumination image data can be used for brightening the Retinex model, so as to ensure the brightening effect of the Retinex model on the image data, i.e. improve the brightness and contrast of the image data, and can also be used for denoising, improve the negative influence of noise on brightening, and combine brightening and denoising in the same frame, so as to optimize the whole processing flow After the video data is highlighted, a higher input is provided for subsequent image processing, so that the quality of the subsequent image processing is ensured.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

EXAMPLE III

Fig. 6 is a block diagram of a structure of a video dim light enhancement device according to a third embodiment of the present invention, which is applied to a mobile terminal, and the device specifically includes the following modules:

a video data acquisition module 601, configured to acquire video data;

an environment parameter collecting module 602, configured to collect, in an environment where the mobile terminal is located, a parameter related to dim light enhancement processing as an environment parameter;

an auxiliary parameter collecting module 603, configured to collect, in the mobile terminal, a parameter that is matched with the environment parameter and is related to the dim light enhancement processing, as an auxiliary parameter;

a dim-light enhancement decision module 604, configured to combine the environmental parameter and the auxiliary parameter to set a state of performing the dim-light enhancement processing on the video data in the environment.

In one embodiment of the present invention, the video data has a plurality of frames of image data;

the video data acquisition module 601 includes:

a reference parameter obtaining sub-module, configured to obtain an environmental parameter acquired between the image data of the current frame and the image data of the previous frame, as a reference parameter;

and the smoothing sub-module is used for smoothing the reference parameter to serve as an environment parameter corresponding to the image data of the current frame.

In one embodiment of the invention, the smoothing sub-module comprises:

the environment weight configuration unit is used for configuring environment weights for the reference parameters;

a first product calculation unit for calculating a first product between the reference parameter and the environment weight;

and a ratio calculation unit, configured to calculate a ratio between a first target value and a second target value as an environmental parameter corresponding to the image data of the current frame, where the first target value is a sum of all the first products, and the second target value is a sum of all the environmental weights.

In an embodiment of the present invention, the configuring the environmental weight for the reference parameter includes:

the time determining subunit is used for determining the time for acquiring the reference parameters;

an environment weight setting subunit, configured to set an environment weight for the reference parameter based on the time, the environment weight being positively correlated with the time.

In one embodiment of the present invention, the environment parameter includes a first luminance value in an external environment where the mobile terminal is located;

the auxiliary parameter acquisition module comprises:

an image data selection sub-module for selecting N frames of continuous image data from the video data;

the second brightness value calculation submodule is used for respectively counting the second brightness values of the N frames of image data;

a target brightness value calculation sub-module for calculating a target brightness value based on the second brightness value as an auxiliary parameter matched with the first brightness value;

and the brightness change value operator module is used for calculating a brightness change value as an auxiliary parameter matched with the first brightness value, wherein the brightness change value is used for expressing the change degree of the second brightness value in the image data of N frames.

In one embodiment of the present invention, the second luminance value calculation sub-module includes:

an anchor setting unit configured to set a plurality of anchors at positions specified in the image data for each frame of the image data;

an image block determination unit, configured to determine, for each anchor point, an image block including the anchor point;

a block luminance value counting unit for counting luminance values of the image blocks as block luminance values;

a position weight configuration unit configured to configure a position weight for the block luminance value according to the position, wherein the position weight decreases from a position at the center to a position at the edge;

a second product calculation unit for calculating a second product between the block luminance value and the position weight;

a product-sum-value calculating unit for calculating a sum value between the second products as a second luminance value of the image data.

In one embodiment of the invention, the luminance change value operator module comprises:

a single-frame change value calculation unit configured to calculate a square of a luminance difference value as a single-frame change value, the luminance difference value being a difference between the second luminance value and the target luminance value;

and the average value calculating unit is used for calculating the average value between the single-frame change values as the brightness change value.

In one embodiment of the present invention, the dim light enhancement decision module 604 comprises:

the overall brightness value calculation submodule is used for fusing the first brightness value and the target brightness value to obtain an overall brightness value, and the overall brightness value is positively correlated with the first brightness value and the target brightness value;

the execution permission submodule is used for allowing the video data to be subjected to dim light enhancement processing aiming at the external environment where the mobile terminal is located if the brightness change value is smaller than or equal to a preset first threshold value and the overall brightness value is smaller than or equal to a preset second threshold value;

and the first execution prohibition sub-module is used for prohibiting dark light enhancement processing from being executed on the video data aiming at the external environment where the mobile terminal is located if the target brightness value is greater than a preset first threshold value and/or the overall brightness value is greater than a preset second threshold value.

In one embodiment of the present invention, the overall luminance value calculation sub-module includes:

a first luminance weight configuration unit configured to configure a first luminance weight for the first luminance value;

a second luminance weight configuration unit configured to configure a second luminance weight for the target luminance value;

a weight sum value calculation unit for calculating a sum value between a second product that is a product between the first luminance value and the first luminance weight and a third product that is a product between the target luminance value and the second luminance weight,

and the brightness linear calculation unit is used for substituting the sum value into a preset linear function so as to calculate the overall brightness value.

In one embodiment of the present invention, the environment parameter includes a temperature value in an internal environment where the mobile terminal is located;

the auxiliary parameter collecting module 603 includes:

and the time counting submodule is used for counting the time occupied by executing the dim light enhancement processing on the video data and taking the counted time as an auxiliary parameter matched with the temperature value.

the performance consumption value operator module is used for carrying out linear fusion on the temperature value and the time to obtain a performance consumption value, and the performance consumption value is positively correlated with the temperature value and the time;

and the second prohibition submodule is used for prohibiting dark light enhancement processing from being executed on the video data according to the internal environment where the mobile terminal is located if the performance consumption value is greater than or equal to a preset third threshold value.

In one embodiment of the invention, the performance consumption value operator module comprises:

a first performance coefficient configuration unit, configured to configure a first performance coefficient for the time;

a second performance coefficient configuration unit, configured to configure a second performance coefficient for the temperature value;

and the performance linearity calculation unit is used for calculating a sum value between a fourth product, a fifth product and a preset third performance coefficient as a performance consumption value, wherein the fourth product is a product between the time and the first performance coefficient, and the fifth product is a product between the temperature value and the second performance coefficient.

In one embodiment of the present invention, further comprising:

a target parameter selection module, configured to select, as a target parameter, an environmental parameter and an auxiliary parameter that participate in dim light enhancement processing if the dim light enhancement processing is allowed to be performed on the video data;

an adjustment parameter calculation module for calculating an adjustment parameter based on the target parameter, the adjustment parameter being used for adjusting the intensity of the dim light enhancement processing;

and the dim light enhancement processing execution module is used for executing the dim light enhancement processing on the video data according to the adjusting parameters.

In an embodiment of the present invention, the target parameter includes a first luminance value in an external environment where the mobile terminal is located, and a target luminance value of the video data, and the adjustment parameter is an overall luminance value after the first luminance value and the target luminance value are fused;

the dim light enhancement processing execution module includes:

the original illumination image data generation submodule is used for generating original illumination image data based on image data in the video data;

the target illumination image data generation submodule is used for improving the brightness of the original illumination image data according to the adjusting parameters to obtain target illumination image data;

the characteristic image data generation submodule is used for synthesizing the target illumination image data and the reflection image data into characteristic image data if the image data is superposition between the original illumination image data and the reflection image data;

and the filtering processing submodule is used for performing filtering processing on the characteristic image data by taking the target illumination image data as guide image data.

In one embodiment of the present invention, the target illumination image data generation sub-module includes:

a gamma coefficient unit for calculating a gamma coefficient based on the adjustment parameter, the gamma coefficient being inversely related to the adjustment parameter;

and the gamma correction unit is used for carrying out gamma correction on the original illumination image data according to the gamma coefficient so as to improve the brightness value of each pixel point in the original illumination image data and take the brightness value as target illumination image data.

The video dim light enhancement device provided by the embodiment of the invention can execute the video dim light enhancement method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

Example four

Fig. 7 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present invention. Fig. 7 illustrates a block diagram of an exemplary mobile terminal 12 suitable for use in implementing embodiments of the present invention. The mobile terminal 12 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 7, the mobile terminal 12 is embodied in the form of a general purpose computing device. The components of the mobile terminal 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

The mobile terminal 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by mobile terminal 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The mobile terminal 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, and commonly referred to as a "hard drive"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.

The mobile terminal 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the mobile terminal 12, and/or any device (e.g., network card, modem, etc.) that enables the mobile terminal 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the mobile terminal 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network such as the Internet) via the network adapter 20. As shown, the network adapter 20 communicates with the other modules of the mobile terminal 12 via the bus 18. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the mobile terminal 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes various functional applications and data processing, such as implementing a dim light enhancement method for video provided by an embodiment of the present invention, by running a program stored in the system memory 28.

EXAMPLE five

An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the dim light enhancement method for a video, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

A computer readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A dim light enhancement method for video, which is applied to a mobile terminal, is characterized in that the method comprises the following steps:

collecting video data;

2. The method according to claim 1, wherein the video data has a plurality of frames of image data;

the collecting parameters related to the dim light enhancement processing in the environment where the mobile terminal is located as environment parameters comprises:

acquiring environmental parameters acquired between the image data of the current frame and the image data of the previous frame as reference parameters;

and smoothing the reference parameter to serve as an environment parameter corresponding to the image data of the current frame.

3. The method according to claim 2, wherein the smoothing the reference parameter as the environment parameter corresponding to the image data of the current frame comprises:

configuring an environment weight for the reference parameter;

calculating a first product between the reference parameter and the environmental weight;

and calculating a ratio between a first target value and a second target value as the environmental parameter corresponding to the image data of the current frame, wherein the first target value is a sum of all the first products, and the second target value is a sum of all the environmental weights.

4. The method of claim 3, wherein configuring the environmental weight for the reference parameter comprises:

determining the time for acquiring the reference parameter;

setting an environmental weight for the reference parameter based on the time, the environmental weight being positively correlated with the time.

5. The method according to claim 1, wherein the environment parameter comprises a first brightness value in an environment external to the mobile terminal;

the collecting of the parameters matched with the environmental parameters and related to the dim light enhancement processing in the mobile terminal as auxiliary parameters comprises:

selecting N frames of continuous image data from the video data;

respectively counting second brightness values of N frames of the image data;

calculating a target luminance value based on the second luminance value as an auxiliary parameter matched with the first luminance value;

and calculating a brightness change value as an auxiliary parameter matched with the first brightness value, wherein the brightness change value is used for indicating the change degree of the second brightness value in the image data of N frames.

6. The method according to claim 5, wherein said separately counting the second luminance values of the N frames of image data comprises:

setting a plurality of anchor points at positions specified in the image data for each frame of the image data;

for each anchor point, determining an image block containing the anchor point;

counting the brightness value of the image block as a block brightness value;

configuring a position weight for the block brightness value according to the position, wherein the position weight decreases from a position at the center to a position at the edge;

calculating a second product between the block luminance value and the position weight;

calculating a sum value between the second products as a second luminance value of the image data.

7. The method of claim 5, wherein the calculating the brightness change value comprises:

calculating the square of a brightness difference value as a single-frame change value, wherein the brightness difference value is the difference value between the second brightness value and the target brightness value;

and calculating the average value between the single-frame change values as the brightness change value.

8. The method of claim 5, wherein said setting a state of performing the dim-light enhancement processing on the video data in the environment in combination with the environment parameter and the auxiliary parameter comprises:

fusing the first brightness value and the target brightness value to obtain an overall brightness value, wherein the overall brightness value is positively correlated with the first brightness value and the target brightness value;

if the brightness change value is smaller than or equal to a preset first threshold value and the overall brightness value is smaller than or equal to a preset second threshold value, dark light enhancement processing is allowed to be executed on the video data aiming at the external environment where the mobile terminal is located;

and if the target brightness value is greater than a preset first threshold value and/or the overall brightness value is greater than a preset second threshold value, forbidding to execute dim light enhancement processing on the video data according to the external environment where the mobile terminal is located.

9. The method of claim 8, wherein said fusing the first luminance value with the target luminance value to obtain an overall luminance value comprises:

configuring a first brightness weight for the first brightness value;

configuring a second brightness weight for the target brightness value;

calculating a sum value between a second product and a third product, the second product being a product between the first luminance value and the first luminance weight, the third product being a product between the target luminance value and the second luminance weight,

and substituting the sum value into a preset linear function to calculate the overall brightness value.

10. The method according to claim 1, wherein the environment parameter comprises a temperature value in an internal environment of the mobile terminal;

and counting the time occupied by executing the dim light enhancement processing on the video data as an auxiliary parameter matched with the temperature value.

11. The method of claim 10, wherein said setting a state of performing the dim-light enhancement processing on the video data in the environment in combination with the environment parameter and the auxiliary parameter comprises:

linearly fusing the temperature value and the time to obtain performance consumption values, wherein the performance consumption values are positively correlated with the temperature value and the time;

and if the performance consumption value is greater than or equal to a preset third threshold value, forbidding to execute dim light enhancement processing on the video data according to the internal environment where the mobile terminal is located.

12. The method of claim 11, wherein said linearly fusing said temperature value with said time to obtain a performance consumption value comprises:

configuring a first performance coefficient for the time;

configuring a second performance coefficient for the temperature value;

and calculating a sum value of a fourth product, a fifth product and a preset third performance coefficient as a performance consumption value, wherein the fourth product is a product between the time and the first performance coefficient, and the fifth product is a product between the temperature value and the second performance coefficient.

13. The method of any one of claims 1-12, further comprising:

if the video data is allowed to be subjected to dim light enhancement processing, selecting environmental parameters and auxiliary parameters participating in the dim light enhancement processing as target parameters;

calculating an adjustment parameter based on the target parameter, the adjustment parameter being used to adjust the intensity of the dim light enhancement process;

and executing the dim light enhancement processing on the video data according to the adjusting parameters.

14. The method according to claim 13, wherein the target parameter includes a first luminance value in an external environment where the mobile terminal is located, a target luminance value of the video data, and the adjustment parameter is an overall luminance value after the first luminance value is fused with the target luminance value;

the performing the dim light enhancement processing on the video data according to the adjustment parameter includes:

generating original illumination image data based on image data in the video data;

improving the brightness of the original illumination image data according to the adjusting parameters to obtain target illumination image data;

if the image data is the superposition between the original illumination image data and the reflection image data, synthesizing the target illumination image data and the reflection image data into characteristic image data;

and taking the target illumination image data as guide image data, and carrying out filtering processing on the characteristic image data.

15. The method of claim 14, wherein said increasing the brightness of said original illumination image data according to said adjustment parameter to obtain target illumination image data comprises:

calculating a gamma coefficient based on the adjustment parameter, the gamma coefficient being inversely related to the adjustment parameter;

and performing gamma correction on the original illumination image data according to the gamma coefficient so as to improve the brightness value of each pixel point in the original illumination image data to be used as target illumination image data.

16. A dim light enhancement device of video, characterized in that, applied in a mobile terminal, the device comprises:

the video data acquisition module is used for acquiring video data;

17. A mobile terminal, characterized in that the mobile terminal comprises:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method for dim-light enhancement of a video according to any one of claims 1-15.

18. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method of dim-light enhancement of a video according to any one of claims 1 to 15.