WO2023060762A1

WO2023060762A1 - Image processing method and apparatus, and computer-readable storage medium

Info

Publication number: WO2023060762A1
Application number: PCT/CN2021/139608
Authority: WO
Inventors: 王鹏; 余明火; 张光宇; 刘翔章; 赵盖
Original assignee: 深圳创维－Rgb电子有限公司
Priority date: 2021-10-12
Filing date: 2021-12-20
Publication date: 2023-04-20
Also published as: CN114007020A; CN114007020B

Abstract

An image processing method and apparatus, and a computer-readable storage medium. The method comprises: acquiring an overexposed image, and acquiring a portrait target area in the overexposed image (S100); acquiring an average pixel effective brightness value of each portrait target area (S200); acquiring through calculation a secondary exposure time, on the basis of a current exposure time, an overexposed brightness threshold, and the average pixel effective brightness value (S300); acquiring a secondary exposure image on the basis of the secondary exposure time, and on the basis of the overexposed image and the secondary exposure image, acquiring a target image after overexposure correction processing is completed (S400). In the present method, photographed objects having a large brightness difference in the same scene are well recorded in the same target image, so that the degree of clarity of the obtained target image is increased, and the image recording effect is improved.

Description

Image processing method, device and computer-readable storage medium

technical field

The present application relates to the technical field of image processing, for example, to an image processing method, device, and computer-readable storage medium.

Background technique

With the development of science and technology, camera equipment is also rapidly developed and applied in various scenarios. For example, in application scenarios such as smart classrooms and remote conferences, images can be collected through cameras. However, in the process of collecting images through the camera, the collected images may not be able to record the current scene normally due to the high brightness of local areas in the scene. For example, in a smart classroom scenario, when the brightness of the classroom is not high and the brightness of the display device is too high, when the camera adjusts the automatic exposure based on the teacher, the part of the display screen that captures the image will be overexposed, making the content of this part unclear.

In the prior art, the exposure time is usually adjusted when the picture is overexposed, so as to obtain an image with better display effect. The problem with the existing technology is that simply adjusting the exposure time, no matter how you adjust it, can’t record the subjects with large brightness differences in the same scene well in the same frame, which is not conducive to improving the clarity of the obtained image, and affects Image recording effect.

Therefore, the prior art still needs to be improved and developed.

application content

The main purpose of the present application is to provide an image processing method, device, and computer-readable storage medium, aiming at solving the problem that the solution of adjusting the exposure time in the prior art cannot record objects with large brightness differences in the same scene well in the same scene. In the picture, it is not conducive to improving the clarity of the obtained image, and affects the image recording effect.

In order to achieve the above purpose, the first aspect of the present application provides an image processing method, wherein the above method includes:

Obtain an overexposed image, and obtain the portrait target area in the above overexposed image;

Respectively obtain the pixel effective luminance mean value of each of the above-mentioned portrait target areas;

Calculate and obtain the secondary exposure time based on the current exposure time, the overexposure brightness threshold and the average value of the effective brightness of the above-mentioned pixels;

A secondary exposure image is acquired based on the secondary exposure time, and a target image after overexposure correction processing is acquired based on the overexposure image and the secondary exposure image.

Optionally, the above-mentioned acquisition of the over-exposed image, and acquisition of the portrait target area in the above-mentioned over-exposure image, including:

Get the image to be processed;

Obtaining the overexposed pixels in the image to be processed based on the above-mentioned overexposure brightness threshold, wherein the brightness value of the above-mentioned overexposed pixels is greater than the above-mentioned overexposure brightness threshold;

When the ratio of the number of overexposed pixels in the image to be processed is higher than the preset ratio of overexposed pixels, the image to be processed is used as an overexposed image;

Get the portrait target area in the above overexposed image.

Optionally, the acquiring the image to be processed includes: acquiring the image to be processed by using a camera automatic exposure technology.

Optionally, the acquisition of the overexposed pixels in the image to be processed based on the above overexposure brightness threshold includes:

The color space of the image to be processed is converted into an HSL model, and pixels whose luminance values are greater than the overexposed brightness threshold in the converted image to be processed are used as the overexposed pixels.

Optionally, the above-mentioned overexposure brightness threshold is 0.9.

Optionally, the aforementioned acquisition of the portrait target area in the aforementioned overexposed image includes:

Perform face recognition on the above-mentioned overexposed image;

The regions corresponding to the recognized faces are respectively used as the above-mentioned portrait target regions.

Optionally, the above method respectively obtains the pixel effective brightness mean value of each of the above portrait target areas, including:

For each of the above-mentioned portrait target areas, calculate the average value of the brightness values of the effective pixels in the above-mentioned portrait target area, as the pixel effective brightness average value of the above-mentioned portrait target area, wherein, the brightness value of the above-mentioned effective pixels is not less than the first brightness threshold and not For pixels greater than the second brightness threshold, the first brightness threshold is equal to the regional brightness mean minus the regional standard deviation, the second brightness threshold is equal to the regional brightness mean plus the regional standard deviation, and the regional brightness mean and the regional standard deviation are respectively the above The mean and standard deviation of the brightness values of all pixels within the portrait target area.

Optionally, the above-mentioned calculation and acquisition of the secondary exposure time based on the current exposure time, the over-exposure brightness threshold and the above-mentioned pixel effective brightness average value include:

Calculate the mean value of the pixel effective brightness mean value of all the above-mentioned portrait target regions as the image effective brightness mean value;

The secondary exposure time is calculated and obtained, wherein the secondary exposure time is equal to the product of the effective average brightness of the image and the current exposure time divided by the over-exposure brightness threshold.

Optionally, the above calculation of the mean value of the mean value of the pixel effective brightness of all the above-mentioned portrait target regions is used as the mean value of the effective brightness of the image, including:

calculate

Among them, k is a positive integer between 1 and N, L(k) represents the kth element of the pixel effective brightness average value array, N represents the total number of the above-mentioned portrait target areas, L' represents the effective brightness average value of the above-mentioned image, and the effective brightness of the above-mentioned pixels The brightness mean value array is an array composed of the pixel effective brightness mean values of all the above-mentioned portrait target regions.

Optionally, the above calculation obtains the secondary exposure time, including:

calculate

Wherein, T' is the above-mentioned secondary exposure time, T is the current exposure time corresponding to the above-mentioned image to be processed, _L1 is the above-mentioned overexposure brightness threshold, and L' is the average value of the effective brightness of the above-mentioned image.

Optionally, the aforementioned acquisition of a secondary exposure image based on the aforementioned secondary exposure time, and acquisition of a target image after completion of overexposure correction processing based on the aforementioned overexposed image and the aforementioned secondary exposure image, including:

Taking pictures based on the above-mentioned secondary exposure time to obtain a secondary exposure image;

The above-mentioned overexposed image is corrected based on the above-mentioned double-exposure image to obtain a target image.

Optionally, the above-mentioned shooting is performed based on the above-mentioned secondary exposure time to obtain a secondary exposure image, including:

The camera is controlled to shoot based on the above-mentioned secondary exposure time, and the image obtained by shooting is used as the above-mentioned secondary exposure image.

Optionally, the above-mentioned overexposed image is corrected based on the above-mentioned secondary exposure image to obtain a target image, including:

A target image is obtained by replacing the overexposed pixels in the overexposed image with corresponding pixels in the double exposure image, wherein the brightness value of the overexposed pixel is greater than the overexposed brightness threshold.

The second aspect of the present application provides an image processing device, wherein the above-mentioned device includes:

An over-exposed image acquisition module, configured to acquire an over-exposed image, and obtain the portrait target area in the above-mentioned over-exposed image;

Mean value acquisition module, used to respectively obtain the pixel effective luminance mean value of each above-mentioned portrait target area;

The secondary exposure time acquisition module is used to calculate and obtain the secondary exposure time based on the current exposure time, the overexposure brightness threshold and the above-mentioned pixel effective brightness average;

A processing module, configured to acquire a secondary exposure image based on the secondary exposure time, and acquire a processed target image based on the overexposed image and the secondary exposure image.

A third aspect of the present application provides a computer-readable storage medium, where an image processing program is stored on the computer-readable storage medium, and when the image processing program is executed by a processor, any one of the steps of the above-mentioned image processing method is implemented.

It can be seen from the above that in the scheme of this application, the overexposed image is obtained, and the portrait target area in the above overexposed image is obtained; the pixel effective brightness average value of each of the above portrait target areas is obtained respectively; based on the current exposure time, the overexposure brightness threshold and the above-mentioned The second exposure time is obtained by calculating the mean value of the effective brightness of the pixels; a second exposure image is obtained based on the above second exposure time, and a target image after overexposure correction processing is obtained based on the above overexposure image and the above second exposure image. Compared with the scheme of simply adjusting the exposure time in the prior art, in the scheme of the present application, the secondary exposure time corresponding to the current overexposed image is obtained, and after the secondary exposure image is obtained based on the secondary exposure time, the current overexposed image and the Obtained double exposure image Acquisition of a target image after overexposure correction processing is conducive to recording objects with large brightness differences in the same scene in the same target image, which is conducive to improving the quality of the obtained target image. Clearness, improve image recording effect.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only for the present application For some embodiments, those skilled in the art can also obtain other drawings according to these drawings without paying creative efforts.

FIG. 1 is a schematic flow diagram of an image processing method provided in an embodiment of the present application;

FIG. 2 is a schematic flowchart of step S100 in FIG. 1 of the embodiment of the present application;

FIG. 3 is a schematic flowchart of step S104 in FIG. 2 of the embodiment of the present application;

FIG. 4 is a schematic diagram of the brightness value distribution corresponding to a portrait target area provided by the embodiment of the present application;

Fig. 5 is a specific flow diagram of step S300 in Fig. 1 of the embodiment of the present application;

Fig. 6 is a specific flow diagram of step S400 in Fig. 1 of the embodiment of the present application;

FIG. 7 is a schematic flow chart of an image processing method provided in an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

FIG. 9 is a functional block diagram of an internal structure of a smart terminal provided by an embodiment of the present application.

Detailed ways

In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude one or more other features. , whole, step, operation, element, component and/or the presence or addition of a collection thereof.

It should also be understood that the terminology used in the specification of the present application is for the purpose of describing specific embodiments only and is not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural referents unless the context clearly dictates otherwise.

It should also be further understood that the term "and/or" used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .

As used in this specification and the appended claims, the term "if" may be construed as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be construed, depending on the context, to mean "once determined" or "in response to the determination" or "once detected [the described condition or event]" event]" or "in response to detection of [described condition or event]".

The technical solutions in the embodiments of the present application are clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In the following description, a lot of specific details are set forth in order to fully understand the application, but the application can also be implemented in other ways different from those described here, and those skilled in the art can do it without violating the content of the application. By analogy, the present application is therefore not limited by the specific embodiments disclosed below.

With the development of science and technology, camera equipment is also rapidly developed and applied in various scenarios. For example, in application scenarios such as smart classrooms and remote conferences, images can be collected through cameras. However, in the process of collecting images through the camera, the collected images may not be able to record the current scene normally due to the high brightness of local areas in the scene. For example, a camera is an essential and important device in a smart classroom system. The camera in a smart classroom captures the teaching process of the teacher and is used for teaching recording and remote streaming. However, the display device of the smart classroom sometimes cannot record normally because the brightness is too high. Specifically, in the smart classroom scenario, when the brightness of the classroom is not high and the brightness of the display device is too high, when the camera adjusts the automatic exposure based on the teacher, the part of the display screen that captures the image will be overexposed, making the content of this part unclear.

In the prior art, the exposure time is usually adjusted when the picture is overexposed, so as to obtain an image with better display effect. The problem with the existing technology is that simply adjusting the exposure time, no matter how you adjust it, can’t record the subjects with large brightness differences in the same scene well in the same frame, which is not conducive to improving the clarity of the obtained image, and affects Image recording effect. In some application scenarios, the convolution method is also used for image restoration, but the convolution method consumes a lot of exposure time and cannot be applied to real-time recording in dynamic scenes. The failure to record the display screen information will cause incomplete recording of the teaching content, remote students cannot observe the teaching content on the display screen, etc., which will affect the normal teaching function and user experience.

In order to solve the problems of the prior art, in the scheme of the present application, the overexposed image is obtained, and the portrait target area in the above-mentioned overexposed image is obtained; the pixel effective brightness mean value of each of the above-mentioned portrait target areas is obtained respectively; based on the current exposure time, the overexposure The brightness threshold and the average value of the effective brightness of the above pixels are calculated to obtain the secondary exposure time; the secondary exposure image is obtained based on the secondary exposure time, and a target after the overexposure correction process is obtained based on the above overexposure image and the above secondary exposure image image. Compared with the scheme of simply adjusting the exposure time in the prior art, in the scheme of the present application, the secondary exposure time corresponding to the current overexposed image is obtained, and after the secondary exposure image is obtained based on the secondary exposure time, the current overexposed image and the Obtained double exposure image Acquisition of a target image after overexposure correction processing is conducive to recording objects with large brightness differences in the same scene in the same target image, which is conducive to improving the quality of the obtained target image. Clearness, improve image recording effect.

Example 1

As shown in Figure 1, the embodiment of the present application provides an image processing method, specifically, the above method includes the following steps:

Step S100, acquiring an overexposed image, and acquiring a portrait target area in the overexposed image.

Wherein, the above-mentioned overexposed image is an image obtained by photographing a target object with partial area overexposure, and the target object is an object that needs to be recorded through the image. In this embodiment, the application scenario of a smart classroom is taken as an example for illustration, and the target objects include teachers and students in the classroom and display screens, and may also include areas containing teaching information such as blackboards. In other application scenarios, the target object may also be other objects. For example, in a smart conference scenario, the target object may include participants and a conference playback display screen, which is not specifically limited here.

The aforementioned portrait target area is an area corresponding to a human face in the overexposed image. The current camera usually adjusts the automatic exposure based on the face. The exposure corresponding to the face area is usually normal, and there will not be too strong overexposure. Therefore, obtaining the target area of the portrait in the overexposed image is helpful to quickly Pixels with normal brightness in the over-exposed image are obtained, so as to facilitate knowing the degree of over-exposure of the over-exposed image, thereby facilitating adjustment of the secondary exposure time, and compensation and correction of the over-exposed area. In this embodiment, in the application scenario of the smart classroom, the faces in the overexposed image include the faces of the teacher and the faces of the students.

In step S200, obtain the pixel effective luminance mean value of each of the above-mentioned portrait target areas respectively.

Wherein, a portrait target area corresponds to a pixel effective mean brightness, which is the mean value of the brightness values of all effective pixels in the corresponding portrait target area, wherein an effective pixel is a pixel that is neither too dark nor too bright. For example, in an application scenario, valid pixels are pixels whose luminance values are within a preset range in the corresponding portrait target area. In the above step S200, the mean value of effective brightness of pixels corresponding to each portrait target area is calculated and acquired respectively.

Step S300 , calculating and obtaining a secondary exposure time based on the current exposure time, the overexposure brightness threshold, and the above average pixel effective brightness.

Specifically, the exposure time is the time for the shutter to be opened in order to project light onto the photosensitive surface of the photographic photosensitive material. Wherein, the above-mentioned current exposure time is the exposure time corresponding to the current over-exposed image, that is, the above-mentioned over-exposed image is an image obtained by shooting based on the current exposure time. The overexposure brightness threshold is a preset brightness threshold. Considering the situation of overexposure, when the brightness value corresponding to a certain pixel exceeds the above-mentioned overexposure brightness threshold, it is considered that the point is overexposed. The above-mentioned overexposure brightness threshold may be set and adjusted according to actual needs. In one application scenario, the current exposure time can be directly reduced as the second exposure time, so as to acquire information of partially overexposed objects in the scene by means of the second lower exposure time. In this embodiment, the secondary exposure time is accurately calculated based on the current exposure time, the overexposure brightness threshold, and the above average pixel effective brightness, so as to ensure that the information of locally overexposed objects in the scene can be fully obtained, and at the same time reduce The required waiting and calculation time can improve real-time performance and meet the real-time performance requirements of the classroom.

In step S400, a secondary exposure image is acquired based on the secondary exposure time, and a target image after overexposure correction processing is acquired based on the overexposure image and the secondary exposure image.

Specifically, after the second exposure time is obtained, the second exposure image is acquired based on the second exposure time. The secondary exposure time is calculated according to the brightness value of the current overexposed image, which can ensure that in the obtained secondary exposure image, the area corresponding to the overexposed area in the current overexposed image is normal (not overexposed ), so that the over-exposure image and the double-exposure image can be combined to obtain a clearer target image that has completed the over-exposure correction. In the above target image, all target objects in the current scene (including the faces of teachers and students and the content displayed on the display screen) are clearly visible, and some content will not be unclear due to overexposure.

It can be seen from the above that in the image processing method provided by the embodiment of the present application, the overexposed image is obtained, and the portrait target area in the above-mentioned overexposed image is obtained; the pixel effective brightness average value of each of the above-mentioned portrait target areas is obtained respectively; based on the current exposure time, Calculate the over-exposure brightness threshold and the average value of the effective brightness of the above pixels to obtain the secondary exposure time; obtain the secondary exposure image based on the above secondary exposure time, and obtain an image based on the above-mentioned over-exposure image and the above-mentioned secondary exposure image after the over-exposure correction process is completed target image. Compared with the scheme of simply adjusting the exposure time in the prior art, in the scheme of the present application, the secondary exposure time corresponding to the current overexposed image is obtained, and after the secondary exposure image is obtained based on the secondary exposure time, the current overexposed image and the Obtained double exposure image Acquisition of a target image after overexposure correction processing is conducive to recording objects with large brightness differences in the same scene in the same target image, which is conducive to improving the quality of the obtained target image. Clearness, improve image recording effect.

Specifically, in this embodiment, as shown in FIG. 2, the above step S100 includes:

Step S101, acquiring an image to be processed.

Step S102, acquiring overexposed pixels in the image to be processed based on the overexposure brightness threshold, where the brightness value of the overexposed pixels is greater than the overexposure brightness threshold.

Step S103, when the ratio of the number of overexposed pixels in the image to be processed is higher than the preset proportion of overexposed pixels, the image to be processed is used as an overexposed image.

Step S104, acquiring the portrait target area in the above-mentioned overexposed image.

Wherein, the image to be processed is an image that requires image processing, and the image to be processed is an image obtained by shooting a target object in a target scene (such as a smart classroom scene) through a camera. The image to be processed may be an image with an overexposure problem, or an image without an overexposure problem. Therefore, in this implementation, it is necessary to determine whether the image to be processed is an overexposed image. Specifically, in this embodiment, the first image of the current scene, that is, the image to be processed, is first obtained through the camera automatic exposure technology, and the image resolution is X×Y, that is, the image to be processed includes X×Y pixels.

In this embodiment, in order to analyze the brightness of the image, the color space of the image to be processed is transformed into an HSL model, where L is a brightness value. The L brightness value range in the HSL model is fixed at 0-1.0, regardless of the color depth, and is suitable for the exposure degree analysis in this embodiment.

In this embodiment, the above-mentioned overexposure brightness threshold is preset to be 0.9. For a certain pixel in the image to be processed, when its brightness value L exceeds 0.9, the pixel is considered to be overexposed and is regarded as an overexposed pixel. Traversing X×Y pixels, obtaining all the overexposed pixels and accumulating their number, calculating the proportion of overexposed pixels in all pixels in the image to be processed (ie, the proportion of the number), when the number of overexposed pixels accounts for When the ratio is higher than the preset overexposure pixel ratio, the image to be processed is processed as an overexposed image; otherwise, the image to be processed is considered normal and no processing is required, and the image to be processed is directly used as the target image. In this embodiment, the preset ratio of overexposed pixels is 10%. When the overexposed pixels exceed 10% of the total pixels in the image to be processed, it is considered that some areas of the image to be processed are overexposed, and image correction processing is required. .

Specifically, in this embodiment, as shown in FIG. 3, the above step S104 includes:

Step S1041, perform face recognition on the above-mentioned overexposed image.

In step S1042, the regions corresponding to the recognized faces are respectively used as the above-mentioned target regions of the portrait.

The current camera usually adjusts the automatic exposure based on the face. The exposure corresponding to the face area is usually normal, and there will not be too strong overexposure. Therefore, in this embodiment, the overexposure image obtained through face recognition face target area to facilitate subsequent image correction. Specifically, the above-mentioned portrait target area includes the area corresponding to the teacher and the area corresponding to the student, assuming that the total number of the portrait target area in the above-mentioned overexposed image is N, the brightness value L in the HSL model corresponding to each pixel in the N portrait target area Record sequentially to obtain the queue ROI[N].

Specifically, in this embodiment, the above-mentioned step S200 includes: for each of the above-mentioned portrait target areas, calculating the average value of the brightness values of the effective pixels in the above-mentioned portrait target area, as the pixel effective brightness average value of the above-mentioned portrait target area, wherein the above-mentioned A valid pixel is a pixel whose luminance value is not less than the first luminance threshold and not greater than the second luminance threshold, the first luminance threshold is equal to the regional luminance mean minus the regional standard deviation, and the above second luminance threshold is equal to the regional luminance mean plus the regional standard deviation , the above-mentioned area brightness mean value and the above-mentioned area standard deviation are respectively the mean value and the standard deviation of the brightness values of all pixels in the above-mentioned portrait target area.

In each portrait target area in the ROI queue, most of the image pixels are valid data, but also contain a small amount of overexposed or dark background or edge pixels, and the brightness distribution in each portrait target area is approximately normal distribution. Specifically, in this embodiment, the effective pixels are pixels whose luminance values are within an effective range (that is, within the range formed by the first luminance threshold and the second luminance threshold). Specifically, for each portrait target area, the brightness values corresponding to all the pixels in the area are obtained. For example, in this embodiment, the queue head data in the ROI queue are processed, and the brightness values of all pixels in each portrait target area are respectively obtained. For the brightness value, for each portrait target area, the brightness values of all its pixels can be respectively formed into a brightness value array ROI_L[n], where n represents the total number of pixels in the portrait target area.

In this embodiment, the luminance values corresponding to all the pixels in each portrait target area are put into the model of approximately normal distribution, specifically, for each portrait target area, the luminance values in the luminance value array ROI_L[n] are respectively put into All elements were placed into a model with an approximately normal distribution. Fig. 4 is a schematic diagram of the brightness value distribution corresponding to a portrait target area provided by the embodiment of the present application, where p is the number of pixels of each brightness value L, μ is the mean value of the normal distribution, and σ is the standard deviation of the normal distribution. The mean value of the above-mentioned regional brightness is the mean value μ of the normal distribution, and the above-mentioned regional standard deviation is the standard deviation σ of the normal distribution. Specifically, in this embodiment, the mean value μ and standard deviation σ are calculated and obtained based on the following formulas (1) and (2):

Among them, Lmax is the maximum value of the brightness value in the target area of the current portrait, i represents the current brightness value, from 0 to Lmax, represents the value of all brightness values traversed, p(i) represents the number of pixels with brightness value i , n represents the total number of pixels in the current portrait target area. j is used for counting, j is taken from 1 to n, ROI_L(j) represents the jth element in the brightness value array ROI_L, the first element of the array ROI_L in this embodiment is ROI_L(1), that is, counting starts from 1 .

In this embodiment, the first brightness threshold is μ-σ, and the second brightness threshold is μ+σ. When the brightness value L is within [μ-σ, μ+σ], the brightness value is an effective brightness value, and the corresponding pixel It is an effective pixel, and the luminance value not in this interval is the interference luminance value. In this embodiment, only effective luminance values are used to calculate the average pixel effective luminance in each portrait target area, thereby improving calculation accuracy, avoiding interference from disturbing luminance values, and obtaining significant luminance values of each portrait target area. Specifically, calculate the average value of the pixel effective brightness in each portrait target area based on the following formula (3):

Among them, A represents the effective average brightness of pixels in a portrait target area, i represents the current brightness value, p(i) represents the number of pixels with brightness value i, and n represents the total number of pixels in the current portrait target area. Combining the average pixel effective luminance values of all portrait target areas to construct an array L[N] of effective pixel luminance average values containing N elements, where N is the total number of portrait target areas.

In an application scenario, the calculated luminance data of the portrait target area may be released, and the corresponding head of the queue ROI[N] may be removed from the queue to save storage space. At the same time, it can also be judged according to the queue ROI[N] whether all the portrait target areas have been calculated. When the queue ROI[N] in the memory is empty, it is considered that the pixel effective brightness mean array L[ N].

Specifically, in this embodiment, as shown in FIG. 5, the above step S300 includes:

Step S301 , calculating the mean value of the average pixel effective brightness of all the above-mentioned portrait target areas, as the average effective brightness of the image.

Step S302, calculating and obtaining a secondary exposure time, wherein the secondary exposure time is equal to the product of the effective image brightness average and the current exposure time divided by the over-exposure brightness threshold.

Specifically, calculate the mean value of the pixel effective luminance mean value corresponding to all N portrait target areas, as the image effective luminance mean value L', in an application scenario, calculate the image effective luminance mean value based on the following formula (4):

Among them, k is used for counting, k is taken from 1 to N, L(k) represents the kth element of the pixel effective brightness mean array, and the first element of the pixel effective brightness mean value array is L(1), that is, starting from 1 Count, N represents the total number of portrait object regions. After obtaining the average value L' of the effective brightness of the above image, based on the relationship between the brightness of the image and the exposure time

A suitable secondary exposure time can be calculated:

Wherein, T′ is the second exposure time, T is the current exposure time, L ₁ is the overexposure brightness threshold, and in this embodiment, the overexposure brightness threshold is set to 0.9.

Specifically, in this embodiment, as shown in FIG. 6, the above step S400 includes:

Step S401, shooting based on the above-mentioned secondary exposure time, and acquiring a secondary exposure image.

Step S402, correcting the above-mentioned over-exposed image based on the above-mentioned double-exposure image, and acquiring a target image.

Specifically, the camera is controlled to take a second shot of the current scene based on the second exposure time obtained by the above calculation, so that a partially overexposed area can be clearly displayed in the second exposure image. Further, the target image is obtained by replacing the overexposed pixels in the overexposed image with corresponding pixels in the double exposure image, wherein the brightness value of the overexposed pixel is greater than the overexposed brightness threshold. In this embodiment, by replacing the pixels with brightness values greater than 0.9 in the above-mentioned overexposed image with the corresponding pixels in the double-exposure image, the correct image after image correction (i.e. the target image) can be obtained as the correction of a frame of image in the video . It should be noted that the time complexity of the method in this embodiment is O(X×Y), which can meet the requirements of real-time processing. After the correction of one frame of image is completed, the next frame of image can be processed, where X×Y is the image resolution of the overexposed image.

In this embodiment, the above image processing method is also specifically described based on a specific application scenario. FIG. 7 is a schematic flowchart of an image processing method provided in an embodiment of the present application. As shown in Figure 7, after acquiring the auto-exposure image (that is, the image to be processed), the color space is converted to determine whether the auto-exposure image is overexposed (whether q% is greater than 10%, and q% is the proportion of overexposed pixels ). When the automatic exposure image is overexposed, the process is performed based on the specific process shown in Figure 7 to obtain the second exposure time, and then the second exposure image is obtained, and the current image is corrected based on the second exposure image, thereby obtaining the correct image ( i.e. the target image). In the prior art, the scene information is obtained by continuously adjusting the exposure time through scene brightness analysis, but when there are objects with significant brightness differences in the same scene, this exposure method cannot capture objects with different brightness in the same frame at the same time . For image inpainting techniques using convolution methods, multiple convolutions will actually result in more frequent exposures, which can only be applied to image inpainting of static scenes. However, the image processing method in this embodiment uses the HSL model to analyze the brightness of the picture. The time complexity of the algorithm is O(X×Y), and the analysis work can be completed with imperceptible time delay. This low-delay algorithm can satisfy Real-time image inpainting for dynamic scenes.

The purpose of using cameras in smart classrooms is different from that of cameras in other fields. Compared with the smoothness of video and the degree of color restoration, information recording is more emphasized. The image processing method proposed in this embodiment reduces the frame rate, obtains significant exposure values (i.e., effective brightness values) based on face recognition technology, and simultaneously converts the entire picture information into an HSL mathematical model to facilitate the analysis of the brightness of each part of the image. After analyzing the brightness value of the over-exposed area of the image, the information of the partially over-exposed object in the scene is obtained by means of the second low exposure time, and finally the two images are stitched together to record the object with different brightness in the same scene. on screen. It should be noted that the above image processing method can also be used in other camera shooting scenes with a display, such as smart conference scenes, etc., and the above image processing method can be used to solve the problem that the display is too bright in the scene and the information on the display cannot be captured. question.

Example 2

As shown in FIG. 8, corresponding to the above image processing method, an embodiment of the present application further provides an image processing device, and the above image processing device includes:

The overexposure image acquisition module 510 is configured to acquire the overexposure image, and acquire the portrait target area in the overexposure image.

The mean value acquisition module 520 is configured to respectively acquire the mean value of the pixel effective brightness of each of the above-mentioned portrait target areas.

Wherein, a portrait target area corresponds to a pixel effective mean brightness, which is the mean value of the brightness values of all effective pixels in the corresponding portrait target area, wherein an effective pixel is a pixel that is neither too dark nor too bright. For example, in an application scenario, valid pixels are pixels whose luminance values are within a preset range in the corresponding portrait target area. In this embodiment, the average pixel effective brightness corresponding to each portrait target area is calculated and acquired respectively.

The second exposure time acquiring module 530 is configured to calculate and acquire the second exposure time based on the current exposure time, the overexposure brightness threshold and the above-mentioned pixel effective brightness average.

The processing module 540 is configured to acquire a secondary exposure image based on the secondary exposure time, and acquire a processed target image based on the overexposed image and the secondary exposure image.

It can be seen from the above that the image processing device provided in the embodiment of the present application acquires the overexposed image through the overexposed image acquisition module 510, and acquires the target area of the portrait in the above-mentioned overexposed image; the average value acquisition module 520 respectively acquires the values of the target areas of the above-mentioned portraits. Pixel effective luminance mean value; the second exposure time is obtained by calculating and obtaining the second exposure time based on the current exposure time, the overexposure brightness threshold and the above-mentioned pixel effective luminance mean value through the second exposure time acquisition module 530; the second exposure image is obtained based on the above-mentioned second exposure time through the processing module 540 , and acquire a processed target image based on the above-mentioned over-exposure image and the above-mentioned double-exposure image. Compared with the scheme of simply adjusting the exposure time in the prior art, in the scheme of the present application, the secondary exposure time corresponding to the current overexposed image is obtained, and after the secondary exposure image is obtained based on the secondary exposure time, the current overexposed image and the Obtained double exposure image Acquisition of a target image after overexposure correction processing is conducive to recording objects with large brightness differences in the same scene in the same target image, which is conducive to improving the quality of the obtained target image. Clearness, improve image recording effect.

Specifically, in this embodiment, for the specific functions of the image processing apparatus and its modules, reference may be made to the corresponding description in the above image processing method, which will not be repeated here.

Example 3

Based on the foregoing embodiments, the present application also provides an intelligent terminal, the functional block diagram of which may be shown in FIG. 9 . The above intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Wherein, the processor of the smart terminal is used to provide calculation and control capabilities. The memory of the smart terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and an image processing program. The internal memory provides an environment for the operation of the operating system and the image processing program in the non-volatile storage medium. The network interface of the smart terminal is used to communicate with external terminals through a network connection. When the image processing program is executed by the processor, the steps of any one of the above image processing methods are implemented. The display screen of the smart terminal may be a liquid crystal display screen or an electronic ink display screen.

Those skilled in the art can understand that the functional block diagram shown in Figure 9 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the smart terminal to which the solution of this application is applied. The specific smart terminal More or fewer components than shown in the figures may be included, or certain components may be combined, or have a different arrangement of components.

In one embodiment, an intelligent terminal is provided. The above-mentioned intelligent terminal includes a memory, a processor, and an image processing program stored on the above-mentioned memory and operable on the above-mentioned processor. When the above-mentioned image processing program is executed by the above-mentioned processor Carry out the following operation instructions:

Example 4

An embodiment of the present application also provides a computer-readable storage medium, where an image processing program is stored on the above-mentioned computer-readable storage medium, and when the above-mentioned image processing program is executed by a processor, any image processing method provided by the embodiment of the present application is implemented. step.

It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units, Module completion means that the internal structure of the above-mentioned device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit, and the above-mentioned integrated units may adopt hardware It can also be implemented in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the above system, reference may be made to the corresponding process in the foregoing method embodiments, and details will not be repeated here.

In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality in different ways for each particular application, but such implementation should not be considered beyond the scope of the present application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are only illustrative. For example, the division of the above-mentioned modules or units is only a logical function division. In actual implementation, other division methods may be used, such as multiple units or Components may be combined or integrated into another system, or some features may be omitted, or not implemented.

If the above-mentioned integrated modules/units are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing related hardware through computer programs. The above computer programs can be stored in a computer-readable storage medium. When executed by the processor, the steps in the above-mentioned various method embodiments can be realized. Wherein, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code may be in the form of source code, object code, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random Access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand; Modifications to the technical solutions described in the examples, or equivalent replacement of some of the technical features; and these modifications or replacements do not mean that the essence of the corresponding technical solutions deviates from the spirit and scope of the technical solutions of the various embodiments of the application, and should be included in this application. within the scope of the application.

Industrial Applicability

The image processing method, device, and computer-readable storage medium provided in the embodiments of the present application acquire the secondary exposure time corresponding to the current overexposed image, and after acquiring the secondary exposure image based on the secondary exposure time, combine the current overexposed image and the obtained Obtain a target image after the over-exposure correction processing of the double-exposure image, and realize the correction processing of the over-exposure image, so that the subjects with large brightness differences in the same scene can be well recorded in the same target image In the actual effect, the clarity of the image has been significantly improved, and the image recording effect has been improved.

Claims

An image processing method, wherein the method includes:

Obtain an overexposed image, and obtain a portrait target area in the overexposed image;

Respectively obtain the average pixel effective brightness of each of the portrait target areas;

calculating and obtaining a second exposure time based on the current exposure time, the overexposure brightness threshold and the average value of the effective brightness of the pixel;

A secondary exposure image is acquired based on the secondary exposure time, and a target image after overexposure correction processing is acquired based on the overexposure image and the secondary exposure image.
The image processing method according to claim 1, wherein said acquiring the overexposed image, and acquiring the portrait target area in the overexposed image comprises:

Get the image to be processed;

Acquiring an overexposed pixel in the image to be processed based on the overexposure brightness threshold, wherein the brightness value of the overexposed pixel is greater than the overexposure brightness threshold;

When the ratio of the number of overexposed pixels in the image to be processed is higher than a preset ratio of overexposed pixels, the image to be processed is used as an overexposed image;

Acquiring a portrait target area in the overexposed image.
The image processing method according to claim 2, wherein said acquiring the image to be processed comprises: acquiring the image to be processed by a camera automatic exposure technique.
The image processing method according to claim 2, wherein said acquiring overexposed pixels in said image to be processed based on said overexposure brightness threshold comprises:

Converting the color space of the image to be processed into an HSL model, and using pixels whose luminance values are greater than the overexposed luminance threshold in the converted image to be processed as the overexposed pixels.
The image processing method according to claim 4, wherein the overexposure brightness threshold is 0.9.
The image processing method according to claim 2, wherein said obtaining the portrait target area in the overexposed image comprises:

Perform face recognition on the overexposed image;

The regions corresponding to the recognized faces are respectively used as the portrait target regions.
The image processing method according to claim 1, wherein said obtaining the pixel effective brightness mean value of each said portrait target area respectively comprises:

For each of the portrait target areas, calculate the mean value of the brightness values of the effective pixels in the portrait target area as the pixel effective brightness mean value of the portrait target area, wherein the effective pixels have a brightness value not less than the first A pixel whose brightness threshold is not greater than the second brightness threshold, the first brightness threshold is equal to the regional brightness mean minus the regional standard deviation, the second brightness threshold is equal to the regional brightness mean plus the regional standard deviation, the regional brightness mean and The regional standard deviations are respectively the mean value and standard deviation of brightness values of all pixels in the portrait target area.
The image processing method according to claim 2, wherein said calculating and obtaining the secondary exposure time based on the current exposure time, the overexposure brightness threshold and the average value of the pixel effective brightness comprises:

Calculate the mean value of the pixel effective brightness mean value of all the portrait target regions as the image effective brightness mean value;

Calculate and obtain a secondary exposure time, wherein the secondary exposure time is equal to the product of the effective image brightness average value and the current exposure time divided by the overexposure brightness threshold.
The image processing method according to claim 8, wherein the calculation of the mean value of the pixel effective brightness mean value of all the portrait target regions, as the image effective brightness mean value, includes:

calculate
Wherein, k is a positive integer between 1 and N, L(k) represents the kth element of the array of pixel effective brightness averages, N represents the total number of the portrait target area, and L' represents the effective brightness average value of the image, so The pixel effective brightness mean value array is an array composed of pixel effective brightness mean values of all the portrait target regions.
The image processing method according to claim 9, wherein said calculating and obtaining the secondary exposure time comprises:

calculate
Wherein, T' is the secondary exposure time, T is the current exposure time corresponding to the image to be processed, L 1 is the overexposure brightness threshold, and L' is the effective average brightness of the image.
The image processing method according to claim 1, wherein the second exposure image is acquired based on the second exposure time, and an overexposure correction process is completed based on the overexposure image and the second exposure image acquisition After the target image, include:

Taking pictures based on the secondary exposure time to obtain a secondary exposure image;

Correcting the overexposed image based on the double exposure image to acquire a target image.
The image processing method according to claim 11, wherein said taking pictures based on said secondary exposure time and obtaining a secondary exposure image comprises:

The camera is controlled to shoot based on the secondary exposure time, and the image obtained by shooting is used as the secondary exposure image.
The image processing method according to claim 11, wherein said correcting said overexposed image based on said secondary exposure image to acquire a target image comprises:

A target image is obtained by replacing overexposed pixels in the overexposed image with corresponding pixels in the double-exposure image, wherein the luminance value of the overexposed pixel is greater than the overexposed luminance threshold.
An image processing device, wherein the device includes:

An overexposed image acquisition module, configured to acquire an overexposed image, and acquire a portrait target area in the overexposed image;

Mean value acquisition module, used to respectively obtain the pixel effective brightness mean value of each described portrait target area;

A secondary exposure time acquisition module, configured to calculate and obtain a secondary exposure time based on the current exposure time, the overexposure brightness threshold, and the average value of the effective brightness of the pixel;

A processing module, configured to acquire a secondary exposure image based on the secondary exposure time, and acquire a processed target image based on the overexposed image and the secondary exposure image.
A computer-readable storage medium, wherein an image processing program is stored on the computer-readable storage medium, and when the image processing program is executed by a processor, the image processing method according to any one of claims 1-13 is implemented step.