WO2022111269A1 - Method and device for enhancing video details, mobile terminal, and storage medium - Google Patents

Abstract

A method and device for enhancing video details, a mobile terminal, and a storage medium. The method comprises: collecting video data (101); during the process of collecting the video data, collecting a parameter related to detail enhancement processing from the environment to serve as an environment parameter (102); collecting a parameter related to detail enhancement processing from the video data to serve as a video parameter (103), and determining, on the basis of the environment parameter and of the video parameter, a state of detail enhancement processing executed with respect to the video data (104).

Description

Video detail enhancement method, device, mobile terminal and storage medium

This application claims the priority of the Chinese Patent Application No. 202011348089.2 filed with the China Patent Office on November 26, 2020, the entire contents of which are incorporated herein by reference.

technical field

The embodiments of the present application relate to the technical field of computer vision, for example, to a video detail enhancement method, apparatus, mobile terminal, and storage medium.

Background technique

With the rapid development of mobile Internet and mobile terminals, video data in mobile terminals has become a common information carrier for many business operations, such as live broadcasts, video calls, etc. They contain a large amount of information about objects and become a way for people to obtain original information from the outside world. one.

The sharpness of video data is an evaluation index of video quality. Therefore, the sharpness of video data will be improved in business operations, and most of the methods to improve sharpness are detail enhancement. Afterwards, detail enhancement processing is typically performed on the video data.

However, in some cases, the detail enhancement processing will bring about negative effects such as distortion, and the detail enhancement processing is continuously performed, which consumes a lot of computing power. Enhancement processing is less robust.

SUMMARY OF THE INVENTION

The embodiments of the present application propose a video detail enhancement method, device, mobile terminal, and storage medium, so as to perform detail enhancement processing on video data under the condition of limited performance, reduce negative effects and improve robustness.

In a first aspect, an embodiment of the present application provides a video detail enhancement method, which is applied to a mobile terminal, and the method includes:

collect video data;

In the process of collecting the video data, parameters related to detail enhancement processing are collected from the environment as environment parameters;

Collect parameters related to detail enhancement processing from the video data as video parameters;

The state in which the detail enhancement processing is performed on the video data is determined according to the environment parameter and the video parameter.

In a second aspect, an embodiment of the present application further provides a video detail enhancement device, which is applied to a mobile terminal, and the device includes:

The video data acquisition module is set to collect video data;

an environment parameter collection module, configured to collect parameters related to detail enhancement processing from the environment as environment parameters during the process of collecting the video data;

A video parameter collection module, configured to collect parameters related to detail enhancement processing from the video data as video parameters;

An enhancement state determination module, configured to determine the state of performing the detail enhancement processing on the video data according to the environment parameter and the video parameter.

In a third aspect, an embodiment of the present application further provides a mobile terminal, where the mobile terminal includes:

one or more processors;

memory, arranged to store one or more programs,

When executed by the one or more processors, the one or more programs cause the one or more processors to implement the video detail enhancement method as described in the first aspect.

In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, realizes the video according to the first party. Detail enhancement method.

Description of drawings

FIG. 1 is a flowchart of a method for enhancing details of a video provided by an embodiment of the present application;

FIG. 2 is a flowchart of a service operation provided by an embodiment of the present application;

3 is a frequency comparison diagram of video data and environmental parameters provided by an embodiment of the present application;

FIG. 4 is a flowchart of a method for enhancing details of a video provided by another embodiment of the present application;

5 is a comparative example diagram of a detail enhancement process provided by an embodiment of the present application;

6 is a schematic structural diagram of a video detail enhancement apparatus provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application.

Detailed ways

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all the structures related to the present application.

Detail enhancement processing for image data has always attracted attention as a process of computer vision from development to prosperity. From the beginning of the detail enhancement algorithm based on single frame image data, it has developed to the present due to the rapid increase of computing power of various platforms. And the emergence of detail enhancement algorithms based on video data.

These detail enhancement processing can be sorted into template-based detail enhancement processing, filtering algorithm-based detail enhancement processing, and deep learning-based detail enhancement processing according to the timeline.

With the rapid development of live broadcast, short video and other business operations, the quality of video data has become a problem that cannot be ignored. Improving the definition of video data collected by mobile terminals has become an important part of improving the quality of video data.

Compared with traditional single-frame image data or dark light enhancement of video data, the detail enhancement processing for mobile terminals has more stringent computational complexity requirements, so how to balance the effect and performance has become a new problem. Most of the detail enhancement algorithms in related technologies are still based on PC (personal computer, personal computer) or server with GPU (Graphics Processing Unit, graphics processor), which requires a large amount of calculation and generally cannot be migrated to very limited computing resources. mobile terminal to execute.

This embodiment proposes an algorithm that takes both performance and effect into consideration. The detail enhancement processing is applied to the video capture of the mobile terminal. While retaining the effect of the detail enhancement processing, it adapts to complex and diverse environments and filters out artifacts that are prone to produce artifacts. In the process of image processing, there are unnatural and abnormal traces, areas, flaws, etc. that can be seen as artificially processed, thus improving the quality of video data.

1 is a flowchart of a video detail enhancement method provided by an embodiment of the present application. This embodiment is applicable to the case where a decision on whether to perform detail enhancement processing is performed according to the environment of the mobile terminal and the characteristics of the video data itself. Executed by a video detail enhancement device, which can be implemented by software and/or hardware, and can be configured in a mobile terminal, such as a mobile phone, tablet computer, smart wearable devices (such as smart watches, smart glasses, etc.) , etc., including the following steps:

Step 101: Collect video data.

In this embodiment, the mobile terminal is configured with one or more cameras (Camera), which can be used for taking pictures and recording videos. In general, the camera can be set on the back of the mobile terminal (also known as a rear camera), or It may be set on the front of the mobile terminal (also known as a front-facing camera), etc., which is not limited in this embodiment of the present application.

In addition, the operating system of the mobile terminal includes Android (Android), iOS, Windows, etc., and can support the running of various applications, such as shopping applications, instant messaging applications, live broadcasting applications, video conference applications, short video applications, and so on.

As shown in Figure 2, in S201, these applications perform business operations, in S202, by calling the camera to collect video data, so as to carry out related business operations, for example, in the Android system, MediaRecorder can be called to collect video data, in iOS In the system, you can call AVCaptureFileOutputRecordingDelegate under AVFoundation to collect video data, and so on.

It should be noted that, due to different business scenarios, the business operations undertaken by the video data collected by these applications are also different.

Generally speaking, the video data collected by these applications can be used to undertake real-time business operations.

For example, in the shopping business scenario, the video data collected by the shopping application can be used to carry the business operations of selling goods, that is, the anchor user explains and demonstrates the goods, and provides shopping links in the video data.

For another example, in the instant messaging service scenario, the video data collected by the instant messaging tool can be used to carry the business operation of the video call, that is, the multi-party users of the communication conduct a conversation.

For another example, in a live broadcast business scenario, the video data collected by the live broadcast application can be used to carry the live broadcast business operations, that is, the host user performs talent performances, game commentary, and the like.

For another example, in a business scenario of a conference, the video data collected by the video conference application can be used to carry the business operations of the conference, that is, multiple users participating in the conference sequentially speak as the host.

Of course, the video data collected by these applications can also undertake business operations with lower real-time requirements. For example, a short video application collects video data as a short video, and so on, which is not limited in this embodiment.

Step 102: In the process of collecting video data, parameters related to detail enhancement processing are collected from the environment as environment parameters.

As shown in FIG. 2 , while the mobile terminal collects video data, in S203 , parameters related to detail enhancement processing are collected for the environment in which the mobile terminal is located, as environment parameters.

In an embodiment of the present application, the environment parameter may be a brightness value of light in an environment where the mobile terminal is located.

In the dark environment, the noise level of the collected video data is relatively high. If the detail enhancement processing is performed at this time, the artifact phenomenon is likely to appear in the area where the noise is concentrated. The detail enhancement processing increases the CPU (central processing unit, central processing unit) in the mobile terminal. ), memory and other resources, but there is no obvious actual benefit.

If the mobile terminal is equipped with a sensor that detects light, such as a light intensity sensor, and its API (Application Programming Interface) is opened, the sensor can be directly called to detect the brightness value of the light in the environment where the mobile terminal is located.

For example, in the Android system, the brightness value of the light detected by the Light Intensity Sensor (LIGHT) can be registered through the SensorManager.

If the mobile terminal is equipped with a sensor for detecting light, but its API has not been opened, the mobile terminal can automatically adjust the screen brightness according to the brightness value of the light, and automatically adjust the white balance according to the brightness value of the light when the camera is turned on. The brightness value is mapped to the brightness value of the light in the environment where the mobile terminal is located, or the brightness value of the light in the environment where the mobile terminal is located is detected from the camera, and so on.

Of course, the above-mentioned environmental parameters and their collection methods are only examples. When implementing this embodiment, other environmental parameters and their collection methods may be set according to actual conditions, such as CPU occupancy rate, memory occupancy rate, etc. This embodiment does not be restricted. In addition, in addition to the above-mentioned environmental parameters and their collection methods, those skilled in the art may also adopt other environmental parameters and their collection methods according to actual needs, which are not limited in this embodiment.

In practical applications, the frame rate at which the mobile terminal collects video data is generally different from the frame rate at which the environment parameters are collected, and the frame rate at which the environment parameters are collected is generally greater than the frame rate at which the video data is collected. There are often multiple environmental parameters in between.

As shown in FIG. 3 , the frame rate of video data collected by the mobile terminal is lower than the frame rate of environmental parameters collected by the sensor, and there are 7 frames of environmental parameters between the first frame of video data and the second frame of video data.

Therefore, in order to ensure the stability of the detail enhancement processing and the environmental parameters, in this embodiment, the image data and the environmental parameters may be aligned in a segmented manner.

In an embodiment, parameters related to detail enhancement processing collected between the image data of the current frame and the image data of the previous frame are acquired in the environment as environmental parameters, so that the video data is used as a reference to divide the environmental parameters into Multi-segment.

For each environmental parameter, the environmental parameter is smoothed, and the smoothed environmental parameter is used as the environmental parameter corresponding to the current frame image data, thereby highlighting the difference in the correlation between the image data and the environmental parameters at different time points in the time series , and subsequently use the environment parameter to perform detail enhancement processing on the frame of image data, so as to improve the quality of the detail enhancement processing.

In a smoothing method, a parameter weight can be configured for each environmental parameter, and the environmental weight can reflect the importance of the environmental parameter, that is, the more important environmental parameter, the greater the environmental weight.

In one example, the time stamp for collecting each piece of environmental parameters may be determined, and a parameter weight is set for each piece of environmental parameters based on the time stamp, wherein the parameter weight is positively correlated with the time stamp, that is, the closer the time of collecting environmental parameters to the current time, the more The stronger the correlation between the environmental parameters and the image data of the current frame, the greater the weight of the configured parameters. On the contrary, the farther the time of collecting the environmental parameters is from the current time, the worse the correlation between the environmental parameters and the image data of the current frame, the weight of the configured parameters. The smaller the value, the parameter weight decreases monotonically from the current frame of image data to the previous frame of image data.

Of course, in addition to the collection time, the parameter weight may also be configured in other manners, for example, the inverse of the number of environmental parameters is used as the parameter weight, etc., which is not limited in this embodiment.

If a corresponding parameter weight has been configured for each environmental parameter, the ratio between the first target value and the second target value can be calculated, and the ratio can be used as the environmental parameter corresponding to the current frame image data, wherein the first target value is a multi-segment environmental parameter The sum of the products of the corresponding parameter weights respectively, the second target value is the sum of the parameter weights corresponding to the multi-stage environmental parameters respectively, and the environmental parameters are expressed as follows:

in,

is the environmental parameter corresponding to the t-th frame of image data, there are n environmental parameters between the t-th frame of image data and the t-1-th frame of image data, y _ti is the i-th frame environmental parameter before the t-th frame of image data, w _i is the weight of the i-th parameter.

Step 103: Collect parameters related to detail enhancement processing from the video data as video parameters.

As shown in FIG. 2 , for the video data collected by the mobile terminal, in S204 , parameters related to detail enhancement processing can be collected from itself as video parameters, so as to characterize the characteristics of the shooting object.

In an embodiment of the present application, the video parameter may be the resolution in the video data, the first probability that a pixel belongs to the skin, and the number of pixels representing the skin.

On the one hand, the resolution of the video data can be read from the MediaRecorder object, which reflects the fineness of the details in the image data. Generally, the higher the resolution, the more pixels it contains. The clearer the image data is. On the contrary, the lower the resolution, the fewer pixels it contains, and the more blurred the image data is. In the case of a lower resolution, the image data is more blurred. If the detail enhancement processing is performed at this time, The artifact phenomenon is easy to appear in the blurred area, and the detail enhancement processing increases the occupation of resources such as CPU and memory in the mobile terminal, but there is no obvious actual benefit.

On the other hand, in live broadcast, short video and other business operations, video data is usually beautified, such as skin resurfacing, etc. If detail enhancement is performed at this time, artifact is prone to appear in the area where the skin is located. phenomenon, the detail enhancement processing increases the occupation of resources such as CPU and memory in the mobile terminal, but there is no obvious actual benefit.

In an embodiment, the video data has multiple frames of image data, and each frame of image data includes multiple pixels. For each frame of image data, skin color detection can be performed on each frame of image data by means of a parameterized model, a non-parametric model, or the like, Calculate the first probability of belonging to the skin for each pixel in each frame of image data, in response to determining that the first probability is greater than a preset skin color threshold, determine that the pixel represents the skin, and count the number of at least one pixel representing the skin , and the quantity of the at least one pixel is taken as the quantity of skin color.

Among them, the parametric model is based on the assumption that skin color can obey the Gaussian probability distribution model, and the non-parametric model estimates the skin color histogram and compares it with the histogram of the training set.

In an example of skin color detection, the video data collected by the camera is mostly YUV (“Y” represents the brightness (Luminance or Luma), that is, the grayscale value, and “U” and “V” represent the chromaticity. (Chrominance or Chroma), which is used to describe the color and saturation of the image, and is used to specify the color of the pixel) format. In order to facilitate skin color detection, the image data can be converted from the YUV color space to RGB (Red Green Blue, Red Green Blue) color space.

For example, each frame of image data can be converted from YUV color space to RGB color space by the following conversion relationship:

R=Y+1.4075*(V-128)

G=Y-0.3455*(U-128)-0.7169*(V-128)

B=Y+1.779*(U-128)

In the RGB color space, the R (red), G (green), and B (blue) components of each pixel in the image data are traversed to determine whether the pixel meets the preset threshold condition.

In response to determining that the pixel points in each frame of image data meet the preset threshold condition, the first probability that the pixel points belong to the skin is set to 1.

In response to determining that the pixel in each frame of image data does not meet the preset threshold condition, the first probability that the pixel belongs to the skin is set to 0.

Among them, the threshold conditions include:

The R component is greater than the first color threshold, the G component is greater than the second color threshold, the B component is greater than the third color threshold, the R component is greater than the G component, and the R component is greater than the B component;

The difference between the maximum value of the R component, the G component, and the B component and the minimum value of the R component, the G component, and the B component is greater than the fourth color threshold;

The difference between the R component and the G component is greater than the fifth color threshold.

Assuming that the first color threshold is 95, the second color threshold is 40, the third color threshold is 20, the fourth color threshold is 15, and the fifth color threshold is 5, the threshold conditions are expressed as follows:

R>95,G>40,B>20,R>G,R>B

(Max(R,G,B)-Min(R,G,B))>15

Abs(R-G)>5

It should be noted that, in addition to skin color detection in the RGB color space, the threshold condition can also be equivalent to the YUV color space through conversion formulas such as BT601.FullRange, and skin color detection is performed in the YUV color space. At this time, the threshold condition is Cb>77, Cb<127, 133<Cr<173, etc., which are not limited in this embodiment.

In this example, skin color detection is used as a condition for judging whether to perform detail enhancement processing on video data, and the accuracy requirements for skin color detection are not very high. Therefore, the threshold condition can meet the requirements, and the threshold condition is simple, using statistical The method is simple, the operation is simple, and the execution time of the skin color detection can be effectively reduced.

Of course, the above video parameters and their detection methods are only examples. When implementing this embodiment, other video parameters and their detection methods may be set according to actual conditions, such as the average brightness value of image data, etc. Unrestricted. In addition, in addition to the above video parameters and their detection methods, those skilled in the art may also adopt other video parameters and their detection methods according to actual needs, which are not limited in this embodiment.

Step 104: Determine the state of performing detail enhancement processing on the video data according to the environment parameter and the video parameter.

In this embodiment, as shown in FIG. 2, in S206, the environment parameters related to the detail enhancement processing are combined with the video parameters to estimate the difference between the current environment where the mobile terminal is located, the photographed object and the detail enhancement processing. Therefore, based on the degree of adaptation, a decision is made to determine the state of performing detail enhancement processing on the video data, that is, to decide whether to perform detail enhancement processing. When certain conditions are met, it is allowed to perform detail enhancement processing on the video data. , when this condition is not met, the detail enhancement processing on the video data is prohibited.

In an embodiment of the present application, step 104 includes the following steps:

Step 1041: For the multiple frames of image data in the video data, compare the environmental parameters with the preset environmental conditions, and compare the video parameters with the preset video conditions, respectively.

In this embodiment, on the one hand, corresponding environmental conditions can be preset for the environmental parameters, and the environmental conditions are used to indicate the degree of adaptation between the environment where the mobile terminal is located and the detail enhancement processing; on the other hand, for the video parameters A video condition may be preset, and the video condition is used to indicate the degree of adaptation between the object photographed by the mobile terminal and the detail enhancement processing.

In one example, the environmental parameter includes the brightness value of light, and the video parameter includes the resolution of the video data and the number of skin color, where the number of skin color is the number of pixels representing the skin.

Then in this example, the luminance value is compared to a preset luminance threshold.

In response to determining that the brightness value is greater than the preset brightness threshold, it means that the environment where the mobile terminal is located is not a dark light environment, and at this time, it can be determined that the environmental parameters meet the environmental conditions; in response to determining that the brightness value is less than or equal to the preset brightness threshold, it means that The environment where the mobile terminal is located belongs to a dark light environment, and at this time, it can be determined that the environmental parameters do not meet the environmental conditions.

In addition, the resolution of the video data is compared with a preset resolution threshold, and the ratio between the number of skin colors and the resolution of the video data is calculated as the skin color ratio, and the skin color ratio is compared with the preset skin color threshold.

In response to determining that the resolution of the video data is greater than the preset resolution threshold, and the skin color ratio is less than the preset skin color threshold, it means that the video data is relatively clear, and the object photographed by the mobile terminal is not mainly a person. At this time, the video parameters can be determined. Meet the video conditions; in response to determining that the resolution of the video data is less than or equal to the preset resolution threshold, and/or, the skin color ratio is greater than or equal to the preset skin color threshold, indicating that the video data is relatively blurred, and the object photographed by the mobile terminal is Characters are the main character. At this time, it may not be determined that the video parameters meet the video conditions.

In this example, the environmental condition f ₁ and the video condition f ₂ are expressed as follows:

in,

is the brightness value of light, S ₁ is the number of skin color, R ₁ is the resolution of video data, τ ₁ is the brightness threshold, τ ₂ is the resolution threshold, and τ ₃ is the skin color threshold.

Of course, the above environmental conditions and video conditions are only examples. When implementing this embodiment, other environmental conditions and video conditions may be set according to the actual conditions of the environmental parameters and video parameters. For example, in response to determining that the environmental parameter is the CPU occupancy rate, When the CPU occupancy rate is less than the preset occupancy threshold, it is determined that the environmental parameter satisfies the environmental condition, in response to determining that the video parameter is an average brightness value, when the average brightness value is greater than the preset video brightness threshold, it is determined that the video parameter satisfies the video condition, etc. , which is not limited in this embodiment. In addition, in addition to the above environmental conditions and video conditions, those skilled in the art can also adopt other environmental conditions and video conditions according to actual needs, which are not limited in this embodiment.

Step 1042: In response to determining that the environmental parameters satisfy the environmental conditions and the video parameters satisfy the video conditions, determine that the image data matches the detail enhancement processing.

If the environmental parameters of a certain frame of image data meet the environmental conditions and the video parameters meet the video conditions, it means that when the frame of image data is captured, the degree of adaptation between the environment where the mobile terminal is located and the detail enhancement processing is high, and the mobile terminal The degree of fit between the captured subject and the detail enhancement processing is high.

Step 1043: Based on the distribution of the image data matching the detail enhancement processing, set the state of performing the detail enhancement processing on the video data.

Since collecting video data is a continuous behavior, considering the continuity of video data, it is possible to identify and match the image data for detail enhancement processing for part of the image data (such as the first 80-120 frames of image data) when the video data is initially collected. distribution in this part of the image data, thereby setting the state of performing the detail enhancement processing on the video data, that is, determining whether to perform the detail enhancement processing on the video data.

In an embodiment, the proportion of the image data of the matching detail enhancement process in the part of the image data may be calculated, the proportion is taken as the matching ratio, and the matching ratio is compared with a preset matching threshold.

In response to determining that the matching ratio is greater than or equal to the preset matching threshold, it means that the environment where the mobile terminal is located is relatively stably adapted to the detail enhancement processing, and the object photographed by the mobile terminal is relatively stably adapted to the detail enhancement processing. Detail enhancement processing is performed on video data.

In response to determining that the matching ratio is less than the preset matching threshold, it indicates that the environment where the mobile terminal is located has a large fluctuation in the adaptation of the detail enhancement processing, and the object photographed by the mobile terminal has a large fluctuation in the adaptation of the detail enhancement processing. At this time, at this time , which disables detail enhancement processing on video data.

In addition, for business operations such as live broadcasting, the environment where the mobile terminal is located and the objects photographed by the mobile terminal are relatively stable. After setting the state of performing detail enhancement processing on video data, the environmental parameters and video parameters can no longer be detected. According to the environmental parameters and video parameters to determine the state of performing detail enhancement processing on video data, to avoid frequent switching of detail enhancement processing, resulting in sudden changes in the picture of the video data.

In this embodiment, for the case where the detail enhancement processing is prohibited, the video data may be subjected to subsequent processing according to the business scenario, which is not limited in this embodiment.

For example, as shown in FIG. 2, for the video data for which the detail enhancement processing is prohibited in S206, in S208, the video data can be displayed on the screen, and in S209, the video data can be encoded, such as according to H.264 Format encoding, and encapsulate it in FLV (Flash Video, streaming media) format, waiting to be transmitted to the device that plays the video data.

In this embodiment, video data is collected, and in the process of collecting video data, parameters related to detail enhancement processing are collected from the environment as environmental parameters, and parameters related to detail enhancement processing are collected from video data as video parameters. , according to the environmental parameters and video parameters to determine the state of performing detail enhancement processing on the video data. On the one hand, in the case where the environment where the mobile terminal is located and the object photographed by the mobile terminal match the detail enhancement processing, the normal execution of the detail enhancement processing is guaranteed. , so as to ensure the quality of the video data. On the other hand, in the case where the environment where the mobile terminal is located and the object shot by the mobile terminal do not match the detail enhancement processing, the execution of the detail enhancement processing is prohibited, the frequency of the detail enhancement processing is reduced, and the details are saved. Enhance the computing power of processing, reduce the occupation of CPU, memory and other resources, reserve more computing power to ensure the normal execution of business operations, and finally improve the flexibility of detail enhancement processing and improve the robustness of detail enhancement processing.

FIG. 4 is a flowchart of a video detail enhancement method provided by another embodiment of the present application. Based on the foregoing embodiment, this embodiment further adds detail enhancement processing operations, and deploys detail enhancement processing on a mobile terminal to record video In the process of data recording, users can perceive the effect of detail enhancement processing in real time, which can maximize the user's experience of shooting video data. A better input to improve the viewing experience, the method includes the following steps:

Step 401: Collect video data.

Step 402: In the process of collecting video data, parameters related to detail enhancement processing are collected from the environment as environment parameters.

Step 403: Collect parameters related to detail enhancement processing from the video data as video parameters.

Step 404: Determine the state of performing detail enhancement processing on the video data according to the environment parameter and the video parameter.

Step 405: In the process of echoing video data, parameters related to detail enhancement processing are collected as echo parameters.

In business operations such as live broadcast and short video, an echo operation can be performed, that is, the video data collected by the camera is displayed on the screen of the mobile terminal, as shown in Figure 2, in S205, during the process of echoing the video data, The parameters related to detail enhancement processing are collected for echo and used as echo parameters.

In an embodiment of the present application, the echo parameter includes the resolution of the screen. In this case, the resolution of the screen can be read by calling the DisplayMetries object or the like.

In addition, the environment parameters, video parameters, and echo parameters are passed to the code corresponding to the detail enhancement processing through JNI (Java Native Interface, Java Native Interface), etc., waiting for the detail enhancement processing to be performed on each frame of image data in the video data. .

Step 406: In response to determining that the state is to allow the detail enhancement processing to be performed on the video data, calculate the adjustment parameters according to the environmental parameters, the video parameters and the echo parameters.

As shown in FIG. 2, in S207, in the case of allowing the detail enhancement processing to be performed on the video data, the environmental parameters, video parameters and echo parameters can be comprehensively measured, thereby calculating the parameters for adjusting the detail enhancement processing, as the adjustment parameters, the The adjustment parameters can be used to adjust the intensity of the detail enhancement processing, that is, to adjust the intensity of the detail enhancement processing performed by each pixel in each frame of image data, thereby adjusting the detail enhancement processing, so that the detail enhancement processing can be used in the environment where the mobile terminal is located, and the mobile terminal. A better comprehensive performance is achieved between the photographed object and the echo effect of the mobile terminal.

In an embodiment of the present application, step 406 includes the following steps:

Step 4061: Convert the video parameters to the first target parameters participating in the detail enhancement process.

Step 4062: Convert the echo parameter to the second target parameter participating in the detail enhancement process.

In this embodiment, the environment parameters, video parameters, and echo parameters are normalized, the video parameters are converted into the first target parameters participating in the detail enhancement processing, and the echo parameters are converted into the second target parameters involved in the detail enhancement processing. target parameters, so as to perform linear fusion with the environmental parameters and the second target parameters at the same scale.

In an example, the video parameters include a first probability that a pixel belongs to skin and a resolution of the video data. In this example, the first probability can be subtracted from one to obtain a second probability that the pixel belongs to non-skin, the The second probability is the first target parameter.

In this example, since the skin color area is considered as smooth as possible from the user's subjective aesthetic point of view, if the details of the skin color area are enhanced, it is easy to enter the artifact. Although the clarity is improved, the quality of the video data is degraded. The pixel points belong to the second probability of non-skin, which can reduce the detail enhancement of the skin and increase the detail enhancement of the non-skin in the detail enhancement process.

In another example, the echo parameter includes the resolution of the screen, then in this example, the ratio between the resolution of the video data and the resolution of the screen is calculated, and the ratio is used as the display ratio, and the display ratio is The second target parameter.

In this example, if the display scale is small, a large-scale upsampling is usually performed during the echo operation, and the detail enhancement process is likely to lead to uneven detail performance.

Step 4063 , linearly fuse the environmental parameters, the first target parameters and the second target parameters to obtain adjustment parameters.

In this embodiment, the environmental parameter, the first target parameter and the second target parameter can be linearly fused to obtain the adjustment parameter, that is, the adjustment parameter is positively correlated with the environmental parameter, the first target parameter, and the second target parameter, that is, the environment The larger the parameter, the first target parameter, and the second target parameter, the larger the adjustment parameter, and the stronger the intensity of the detail enhancement processing. On the contrary, the smaller the environmental parameter, the first target parameter, and the second target parameter The intensity of the enhancement processing is weaker.

In an example, the environment parameter includes the brightness value of light, the first target parameter includes the second probability that the pixel belongs to non-skin, and the second target parameter includes the display ratio, then in this example, the first adjustment weight is configured for the brightness value , configure the second adjustment weight for the second probability, and configure the third adjustment weight for the display ratio.

It should be noted that the first adjustment weight, the second adjustment weight, and the third adjustment weight can be set by those skilled in the art according to the actual situation. For example, the second adjustment weight is dominant, that is, the second adjustment weight is larger than the first adjustment weight. The weight, the third adjustment weight, etc., are not limited in this embodiment.

Calculate the sum between the first adjustment value, the second adjustment value, and the third adjustment value, and use the sum as the fourth adjustment value, where the first adjustment value represents the product of the brightness value and the first adjustment weight, The second adjustment value represents the product between the second probability and the second adjustment weight, and the third adjustment value represents the product between the display scale rate and the third adjustment weight.

A preset second adjustment coefficient is added on the basis of the product between the fourth adjustment value and the preset first adjustment coefficient as an adjustment parameter.

In this example, the tuning parameters are as follows:

p ₂ =1-p ₁

Among them, the S _ti table is the adjustment parameter of the i-th pixel point of the t-th frame,

is the brightness value, p ₁ is the first probability, p ₂ is the second probability, R ₁ is the resolution of the video data, R ₂ is the resolution of the screen, r is the display ratio, α is the first adjustment weight, and β is the first adjustment weight. The second adjustment weight, γ is the third adjustment weight, k is the first adjustment coefficient, and b is the second adjustment coefficient.

Step 407: Perform detail enhancement processing on the video data according to the adjustment parameters.

After the intensity of the current detail enhancement process (ie, the adjustment parameter) is determined, the relevant parameters in the detail enhancement process may be adjusted according to the intensity (ie, the adjustment parameter), so as to perform the detail enhancement process on the video data.

In an embodiment of the present application, step 407 may include the following steps:

Step 4071: Extract details from the image data of the video data to obtain original detail data.

In an embodiment, the video data may be filtered by methods such as box filtering, mean filtering, etc., so as to extract details from the image data of the video data as original detail data.

Step 4072: Adjust the original detail data based on the adjustment parameter to obtain the target detail data.

The adjustment parameters are applied to the original detail data and adjusted to realize the intensity adjustment of the detail enhancement processing.

In one example, the adjustment weight can be queried, and the adjustment weight and the adjustment parameter are multiplied by the original detail data to obtain the target detail data.

Of course, in addition to adjusting the weight, the adjustment parameter can also be multiplied by the original detail data to obtain the target detail data, etc., which is not limited in this embodiment.

Step 4073, superimpose the target detail data on the image data.

The target detail data after adjusting the adjustment parameters can be superimposed on the original image data by means such as USM sharpening, and the image data after detail enhancement can be obtained. At this time, the detail enhancement processing is expressed as follows:

y _t (i,j)=y _t (i,j)+λ*S _ti *D _t (i,j)

Among them, y _t (i, j) is the image data after detail enhancement, y _t (i, j) is the image data before detail enhancement, λ is the adjustment weight, S _ti is the adjustment parameter, D _t (i, j) for the original detail data.

In order to reduce the computational complexity of the detail enhancement processing, the detail enhancement processing can be performed under a certain channel (such as the Y channel in the YUV color space and the G channel in the RGB color space) to which the user is sensitive, and the obtained detail enhancement channel (such as Y _t ) is fused with other channels (such as U _t V _t ) to obtain video data with improved definition.

As shown in FIG. 5 , the left side is the original image data, and the right side is the image data after the detail enhancement processing is performed by applying this embodiment. In contrast, the right image data is compared with the left image data in the background of plants. In the area where the character's collar area has a noticeable improvement in sharpness.

Of course, the above-mentioned detail enhancement processing is just an example. When implementing this embodiment, the detail enhancement processing can be set according to the actual situation. size, etc., which are not limited in this embodiment. In addition, in addition to the above-mentioned detail enhancement processing, those skilled in the art may also adopt other detail enhancement processing according to actual needs, which is not limited in this embodiment.

For the video data after the detail enhancement processing, subsequent processing may be performed according to the service scenario, which is not limited in this embodiment.

For example, as shown in FIG. 2, in S208, the video data after the detail enhancement processing is displayed on the screen, and in S209, the video data after the detail enhancement is encoded, for example, according to the H.264 format, and packaged as FLV format, waiting to be transferred to the device that plays the video data.

In this embodiment, in response to determining that the detail enhancement processing is allowed to be performed on the video data, in the process of echoing the video data, parameters related to the detail enhancement processing are collected for the echoing, and are used as the echoing parameters, according to the environmental parameters, video parameters The adjustment parameters are calculated with the echo parameters, and the detail enhancement processing is performed on the video data according to the adjustment parameters, so that the detail enhancement intensity is adapted to the environment where the mobile terminal is located, the objects photographed by the mobile terminal, and the echo effect of the mobile terminal as a whole. The efficiency of power utilization, due to the consideration of the effect of detail enhancement processing and the consideration of computational complexity, makes it possible to apply detail enhancement processing in the process of real-time video communication on mobile terminals or in the process of collecting short videos.

It should be noted that, for the sake of simple description, the method embodiments are expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the described action sequence, because According to the embodiments of the present application, certain steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all exemplary embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

FIG. 6 is a structural block diagram of a video detail enhancement apparatus provided by an embodiment of the present application. When applied to a mobile terminal, the apparatus may include the following modules:

A video data collection module 601, configured to collect video data;

The environment parameter collection module 602 is configured to collect parameters related to detail enhancement processing from the environment as environment parameters during the process of collecting the video data;

A video parameter collection module 603, configured to collect parameters related to detail enhancement processing from the video data as video parameters;

The enhancement state determining module 604 is configured to determine the state of performing the detail enhancement processing on the video data according to the environment parameter and the video parameter.

In an embodiment of the present application, the video data has multiple frames of image data, and each frame of image data includes multiple pixels; the environmental parameter collection module 602 includes:

A parameter acquisition sub-module, configured to acquire parameters related to detail enhancement processing collected between the image data of the current frame and the image data of the previous frame in the environment, as environmental parameters;

The smoothing processing sub-module is configured to perform smoothing processing on the environmental parameters, and use the smoothed environmental parameters as the environmental parameters corresponding to the image data of the current frame.

In an embodiment of the present application, taking video data as a reference, the environmental parameters are divided into multiple segments, and the smoothing sub-module includes:

The parameter weight configuration unit, which is set to configure the parameter weight for each environmental parameter;

The ratio calculation unit is configured to calculate the ratio between the first target value and the second target value, and use the ratio as the environmental parameter corresponding to the image data of the current frame, wherein the first target value is a plurality of environmental parameters corresponding to The sum value of the products between the parameter weights, the second target value is the sum value between the parameter weights corresponding to the multi-segment environment parameters respectively.

In an embodiment of the present application, the parameter weight configuration unit includes:

a timestamp determining subunit, configured to determine a timestamp for collecting the environmental parameters of each segment;

The parameter weight setting subunit is configured to set a parameter weight for each segment of environmental parameters based on the timestamp, where the parameter weight is positively correlated with the timestamp.

In an embodiment of the present application, the video data has multiple frames of image data, and each frame of image data includes multiple pixels; the video parameter collection module 603 includes:

a first probability calculation submodule, configured to calculate the first probability of belonging to the skin for each pixel in the image data of each frame;

a skin color pixel point determination submodule, configured to determine that the pixel point corresponding to the first probability represents the skin in response to determining that the first probability is greater than a preset skin color threshold;

The skin color quantity counting sub-module is configured to count the quantity of at least one pixel point representing the skin, and use the quantity of the at least one pixel point as the skin color quantity.

In an embodiment of the present application, the first probability calculation submodule includes:

The color space conversion unit is set to convert each frame of image data from the YUV color space to the RGB color space;

The first numerical value setting unit is set to in the RGB color space, in response to determining that the pixel points in each frame of image data meet the preset threshold condition, the first probability that the pixel points belong to the skin is set to 1 ;

a second numerical value setting unit, configured to set the first probability that the pixel belongs to the skin to 0 in response to determining that the pixel in each frame of image data does not meet the preset threshold condition;

Wherein, the threshold condition includes:

The R component is greater than the first color threshold, the G component is greater than the second color threshold, the B component is greater than the third color threshold, the R component is greater than the G component, and the R component is greater than the B component;

The difference between the maximum value of the R component, the G component, and the B component and the minimum value of the R component, the G component, and the B component is greater than the fourth color threshold;

The difference between the R component and the G component is greater than the fifth color threshold.

In an embodiment of the present application, the enhanced state determination module 604 includes:

Condition comparison submodule, is set to for the multi-frame image data in the described video data, compares described environment parameter and preset environment condition respectively, and described video parameter is compared with preset video condition;

an image matching submodule, configured to determine that the image data matches the detail enhancement processing in response to determining that the environmental parameter satisfies the environmental condition and the video parameter satisfies the video condition;

A distribution setting sub-module configured to set a state of performing the detail enhancement processing on the video data based on the distribution of the image data matching the detail enhancement processing.

In an embodiment of the present application, the environmental parameter includes a brightness value of light, and the video parameter includes the resolution of the video data and the number of skin colors, and the number of skin colors is the number of pixels representing skin;

The conditional comparison submodule includes:

an environmental condition satisfaction determination unit, configured to determine that the environmental parameter satisfies the environmental condition in response to determining that the brightness value is greater than a preset brightness threshold;

a resolution comparison unit, configured to compare the resolution of the video data with a preset resolution threshold;

a skin color ratio calculation unit, set to calculate the ratio between the skin color number and the resolution of the video data, as the skin color ratio;

The video condition satisfaction determination unit is configured to determine that the video parameter satisfies the video condition in response to determining that the resolution of the video data is greater than a preset resolution threshold and the skin color ratio is less than the preset skin color threshold.

In an embodiment of the present application, the distribution setting sub-module includes:

a matching ratio calculation unit, configured to calculate the ratio of the image data matching the detail enhancement processing in the video data, and use the ratio as a matching ratio;

an enhancement unit, configured to allow the detail enhancement processing to be performed on the video data in response to determining that the matching ratio is greater than or equal to a preset matching threshold;

A disabling enhancement unit, configured to disallow performing the detail enhancement processing on the video data in response to determining that the matching ratio is smaller than a preset matching threshold.

In an embodiment of the present application, it also includes:

The echo parameter acquisition module is configured to collect parameters related to the detail enhancement processing as echo parameters during the process of echoing the video data;

An adjustment parameter calculation module, configured to, in response to determining that the state is to allow the execution of detail enhancement processing on the video data, calculate an adjustment parameter according to the environmental parameter, the video parameter and the echo parameter, and the adjustment parameter uses for adjusting the intensity of the detail enhancement processing;

The detail enhancement processing execution module is configured to perform the detail enhancement processing on the video data according to the adjustment parameter.

In an embodiment of the present application, the adjustment parameter calculation module includes:

a first target parameter conversion submodule, configured to convert the video parameters into the first target parameters participating in the detail enhancement processing;

A second target parameter conversion submodule, configured to convert the echo parameter into a second target parameter participating in the detail enhancement processing;

The linear fusion sub-module is configured to perform linear fusion of the environmental parameter, the first target parameter and the second target parameter to obtain adjustment parameters.

In an embodiment of the present application, the video parameter includes a first probability that a pixel belongs to the skin, a resolution of the video data, and the echo parameter includes a screen resolution;

The first target parameter conversion submodule includes:

A second probability calculation unit, configured to subtract the first probability from one to obtain the second probability that the pixel point belongs to non-skin;

The second target parameter conversion submodule includes:

A display ratio calculation unit, configured to calculate the ratio between the resolution of the video data and the resolution of the screen, and the ratio between the resolution of the video data and the resolution of the screen will be used as a display ratio .

In an embodiment of the present application, the environmental parameter includes a brightness value of light;

The linear fusion submodule includes:

a first adjustment weight configuration unit, configured to configure a first adjustment weight for the brightness value;

a second adjustment weight configuration unit, configured to configure a second adjustment weight for the second probability;

a third adjustment weight configuration unit, configured to configure a third adjustment weight for the display scale;

an adjustment value calculation unit, configured to calculate a sum value among the first adjustment value, the second adjustment value, and the third adjustment value, and use the sum value as a fourth adjustment value, the first adjustment value representing the brightness value and the first adjustment weight, the second adjustment value represents the product of the second probability and the second adjustment weight, and the third adjustment value represents the display ratio and the the product between the third adjustment weights;

The adjustment value adjustment unit is configured to add a preset second adjustment coefficient on the basis of the product between the fourth adjustment value and the preset first adjustment coefficient as an adjustment parameter.

In an embodiment of the present application, the detail enhancement processing execution module includes:

An original detail data extraction submodule, configured to extract detail from the image data of the video data to obtain original detail data;

an original detail data adjustment submodule, configured to adjust the original detail data based on the adjustment parameter to obtain target detail data;

A target detail data superimposition sub-module, configured to overlay the target detail data to the image data.

In an embodiment of the present application, the original detail data adjustment sub-module includes:

The weight adjustment query unit is set to query adjustment weight;

The detail multiplication unit is configured to multiply the adjustment weight, the adjustment parameter and the original detail data to obtain target detail data.

The video detail enhancement apparatus provided by the embodiment of the present application can execute the video detail enhancement method provided by any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

FIG. 7 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application. FIG. 7 shows a block diagram of an exemplary mobile terminal 12 suitable for use in implementing embodiments of the present application. The mobile terminal 12 shown in FIG. 7 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 7, the mobile terminal 12 takes the form of a general-purpose computing device. Components of the mobile terminal 12 may include, but are not limited to, one or more processors or processing units 16 , a system memory 28 , and a bus 18 connecting various system components including the system memory 28 and the processing unit 16 .

Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.

Mobile terminal 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the mobile terminal 12, including volatile and non-volatile media, removable and non-removable media.

System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 . The mobile terminal 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. For example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 7, commonly referred to as a "hard drive"). Although not shown in Figure 7, a disk drive for reading and writing to removable non-volatile magnetic disks (eg "floppy disks") and removable non-volatile optical disks (eg CD-ROM, DVD-ROM) may be provided or other optical media) to read and write optical drives. In these cases, each drive may be connected to bus 18 through one or more data media interfaces. The memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.

A program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data , each or some combination of these examples may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.

The mobile terminal 12 may also communicate with one or more external devices 14 (eg, a keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the mobile terminal 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the mobile terminal 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 . Also, the mobile terminal 12 may communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network such as the Internet) through a network adapter 20 . As shown, the network adapter 20 communicates with other modules of the mobile terminal 12 via the bus 18 . It should be understood that, although not shown, other hardware and/or software modules may be used in conjunction with the mobile terminal 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives and data backup storage systems.

The processing unit 16 executes a variety of functional applications and data processing by running the programs stored in the system memory 28, for example, implementing the video detail enhancement method provided by the embodiments of the present application.

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, multiple processes of the above-mentioned video detail enhancement method can be achieved, and the same In order to avoid repetition, the technical effect will not be repeated here. The computer-readable storage medium may be a non-transitory computer-readable storage medium.

Wherein, the computer-readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

Claims (18)

1. A video detail enhancement method, applied in a mobile terminal, the method comprising:

   collect video data;

   In the process of collecting the video data, parameters related to detail enhancement processing are collected from the environment as environment parameters;

   Collect parameters related to detail enhancement processing from the video data as video parameters;

   The state in which the detail enhancement processing is performed on the video data is determined according to the environment parameter and the video parameter.
2. The method according to claim 1, wherein the video data has multiple frames of image data; the collecting parameters related to detail enhancement processing from the environment as the environment parameters, comprising:

   Acquiring from the environment parameters collected between the image data of the current frame and the image data of the previous frame and related to detail enhancement processing as environment parameters;

   Smoothing is performed on the environmental parameters, and the smoothed environmental parameters are used as environmental parameters corresponding to the image data of the current frame.
3. The method according to claim 2, wherein, taking the video data as a reference, dividing the environmental parameters into multiple segments, performing smoothing processing on the environmental parameters, and using the smoothed environmental parameters as The environmental parameters corresponding to the image data of the current frame include:

   Configure parameter weights for each environmental parameter;

   Calculate the ratio between the first target value and the second target value, and use the ratio as the environmental parameter corresponding to the image data of the current frame, wherein the first target value is the corresponding parameter of the plurality of environmental parameters respectively The sum value of the product of the weights, and the second target value is the sum value of the parameter weights corresponding to the multi-stage environmental parameters respectively.
4. The method according to claim 3, wherein the configuring parameter weights for each segment of environmental parameters comprises:

   determining the time stamp for collecting each of the environmental parameters;

   A parameter weight is set for each piece of environmental parameters based on the timestamp, and the parameter weight is positively correlated with the timestamp.
5. The method according to claim 1, wherein the video data includes multiple frames of image data; each frame of image data includes multiple pixels, and the parameters related to detail enhancement processing are collected from the video data as video parameters ,include:

   Calculate the first probability of belonging to the skin for each pixel in the each frame of image data;

   In response to determining that the first probability is greater than a preset skin color threshold, determining that the pixel corresponding to the first probability represents skin;

   The number of at least one pixel point representing the skin is counted, and the number of the at least one pixel point is used as the number of skin color.
6. The method according to claim 5, wherein the calculating the first probability of belonging to the skin for each pixel in each frame of image data comprises:

   Converting the image data of each frame from the YUV color space to the RGB color space;

   In the RGB color space, in response to determining that the pixels in each frame of image data meet a preset threshold condition, setting the first probability that the pixels belong to the skin to 1;

   In response to determining that the pixel in each frame of image data does not meet the preset threshold condition, setting the first probability that the pixel belongs to the skin to 0;

   Wherein, the threshold condition includes:

   The R component is greater than the first color threshold, the G component is greater than the second color threshold, the B component is greater than the third color threshold, the R component is greater than the G component, and the R component is greater than the B component;

   a difference between a maximum value of the R component, the G component, and the B component and a minimum value of the R component, the G component, and the B component greater than a fourth color threshold; and

   The difference between the R component and the G component is greater than a fifth color threshold.
7. The method according to claim 1, wherein the determining the state of performing the detail enhancement processing on the video data according to the environmental parameter and the video parameter comprises:

   For multiple frames of image data in the video data, compare the environmental parameters with preset environmental conditions, and compare the video parameters with preset video conditions;

   In response to determining that the environmental parameter satisfies the environmental condition and the video parameter satisfies the video condition, determining that the image data matches the detail enhancement process;

   A state in which the detail enhancement processing is performed on the video data is set based on the distribution of the image data matching the detail enhancement processing.
8. The method according to claim 7, wherein the environmental parameter includes a brightness value of light, and the video parameter includes a resolution of the video data and a skin color number, and the skin color number is the number of pixels representing skin; Wherein, the video data includes multiple frames of image data, and each frame of image data includes multiple pixels;

   The step of comparing the environmental parameters with preset environmental conditions and comparing the video parameters with preset video conditions, respectively, includes:

   In response to determining that the brightness value is greater than a preset brightness threshold, determining that the environmental parameter satisfies an environmental condition;

   comparing the resolution of the video data with a preset resolution threshold;

   Calculate the ratio between the number of the skin color and the resolution of the video data, as the skin color ratio;

   In response to determining that the resolution of the video data is greater than a preset resolution threshold, and the skin color ratio is less than the preset skin color threshold, it is determined that the video parameter satisfies a video condition.
9. The method of claim 7, wherein the setting the state of performing the detail enhancement processing on the video data based on the distribution of the image data matching the detail enhancement processing comprises:

   Calculate the proportion of the image data matching the detail enhancement processing in the video data, and use the proportion as a matching proportion;

   in response to determining that the matching ratio is greater than or equal to a preset matching threshold, allowing the detail enhancement processing to be performed on the video data;

   In response to determining that the matching ratio is less than a preset matching threshold, the detail enhancement processing is prohibited from being performed on the video data.
10. The method according to any one of claims 1-9, further comprising:

    In the process of echoing the video data, collecting parameters related to the detail enhancement processing as echo parameters;

    In response to determining that the state allows detail enhancement processing to be performed on the video data, an adjustment parameter is calculated from the environmental parameter, the video parameter, and the echo parameter, the adjustment parameter used to adjust the detail enhancement processing Strength of;

    The detail enhancement process is performed on the video data according to the adjustment parameter.
11. The method according to claim 10, wherein the calculating the adjustment parameter according to the environmental parameter, the video parameter and the echo parameter comprises:

    converting the video parameters into first target parameters participating in the detail enhancement process;

    converting the echo parameter into a second target parameter participating in the detail enhancement process;

    Linearly fuse the environment parameter, the first target parameter and the second target parameter to obtain an adjustment parameter.
12. The method according to claim 11, wherein the video data includes multiple frames of image data, and each frame of image data includes multiple pixels; the video parameters include a first probability that a pixel belongs to the skin, a first probability of the video data Resolution, the echo parameter includes the resolution of the screen;

    The converting the video parameters into the first target parameters participating in the detail enhancement processing includes:

    Subtracting the first probability from one to obtain the second probability that the pixel belongs to non-skin;

    The converting the echo parameter into the second target parameter participating in the detail enhancement processing includes:

    Calculate the ratio between the resolution of the video data and the resolution of the screen, and use the ratio between the resolution of the video data and the resolution of the screen as a display scale.
13. The method of claim 12, wherein the environmental parameter includes a brightness value of light;

    The linear fusion of the environmental parameter, the first target parameter and the second target parameter to obtain an adjustment parameter includes:

    configuring a first adjustment weight for the brightness value;

    configuring a second adjustment weight for the second probability;

    configuring a third adjustment weight for the display scale;

    Calculate the sum of the first adjustment value, the second adjustment value, and the third adjustment value, and use the sum as the fourth adjustment value, where the first adjustment value represents the brightness value and the first adjustment value The product of weights, the second adjustment value represents the product between the second probability and the second adjustment weight, and the third adjustment value represents the difference between the display ratio and the third adjustment weight product;

    On the basis of the product between the fourth adjustment value and the preset first adjustment coefficient, a preset second adjustment coefficient is added as an adjustment parameter.
14. The method according to claim 10, wherein the performing the detail enhancement processing on the video data according to the adjustment parameter comprises:

    Extract details from the image data of the video data to obtain original detail data;

    Adjust the original detail data based on the adjustment parameter to obtain target detail data;

    The target detail data is superimposed on the image data.
15. The method according to claim 14, wherein the adjusting the original detail data based on the adjustment parameter to obtain the target detail data comprises:

    query adjustment weight;

    Multiplying the adjustment weight, the adjustment parameter and the original detail data to obtain target detail data.
16. A video detail enhancement device, applied in a mobile terminal, the device includes:

    The video data acquisition module is set to collect video data;

    an environment parameter collection module, configured to collect parameters related to detail enhancement processing from the environment as environment parameters during the process of collecting the video data;

    A video parameter collection module, configured to collect parameters related to detail enhancement processing from the video data as video parameters;

    An enhancement state determination module, configured to determine the state of performing the detail enhancement processing on the video data according to the environment parameter and the video parameter.
17. A mobile terminal, comprising:

    at least one processor;

    memory, arranged to store one or more programs,

    When the at least one program is executed by the at least one processor, the at least one processor is caused to implement the video detail enhancement method according to any one of claims 1-15.
18. A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the video detail enhancement method according to any one of claims 1-15.

Images