WO2022246663A1

WO2022246663A1 - Image processing method, device and system, and storage medium

Info

Publication number: WO2022246663A1
Application number: PCT/CN2021/095871
Authority: WO
Inventors: 席迎来
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-05-25
Filing date: 2021-05-25
Publication date: 2022-12-01

Abstract

An image processing method, comprising: acquiring an image to be processed (S110); dividing said image into a plurality of image blocks (S120); determining a flat area in said image (S130); calculating a histogram of the flat area (S140); and acquiring a target image of each image block in the flat area according to the histogram of the flat area (S150). According to the present application, an image can be processed according to a histogram of the image. Also provided are a device, a system, and a storage medium.

Description

Image processing method, device, system and storage medium

technical field

The present application relates to the technical field of image processing, and in particular to an image processing method, device, system and storage medium.

Background technique

With the development of Internet technology, image and video content tends to be popularized, and users' demands for editing images and videos are increasing day by day. Image/video editors have become a popular software APP on mobile terminals, providing personalized tools for image/video lovers to freely edit. Due to the variety of image/video material acquisition devices and shooting environments are also very different, usually the visual effect of image/video material cannot be 100% satisfactory, and basic quality such as lighting, contrast, color temperature, and saturation need to be adjusted.

Existing image adjustment methods usually stretch the whole image directly, or divide it into multiple image blocks in a fixed way and stretch each divided image block, but stretch the whole image directly. It will change the contrast of the whole image. In many cases, the effect of doing so is not good, and after block processing, it is easy to stretch the local contrast of the image block too much, resulting in image distortion; although some methods divide the image into multiple After the smaller image blocks, some smaller image blocks are classified into larger image blocks, and then the larger image blocks are stretched, but the larger image blocks will include different objects. After that, the adjacency of different objects will produce distortion, which will also lead to image distortion and other phenomena.

Contents of the invention

The present application provides an image processing method, device, system and storage medium, aiming to solve technical problems such as image distortion easily caused by existing image/video adjustment methods.

In the first aspect, the embodiment of the present application provides an image processing method, including:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

determining a flat area in the image to be processed, the flat area includes at least two image blocks, and in the flat area, each image block has at least one adjacent image block, The sum of the information entropies of all the image blocks in the flat area is less than or equal to a preset threshold;

Computing a histogram of said flat area;

The target image of each image block in the flat area is acquired according to the histogram of the flat area.

Optionally, the determining the flat area in the image to be processed includes:

Determine the information entropy of each image block according to the histogram of each image block in the image to be processed;

If the sum of the information entropies of at least two adjacent image blocks is less than or equal to the preset threshold, it is determined that the at least two adjacent image blocks are located in the same flat area.

Optionally, the calculating the histogram of the flat area includes:

The histogram of each image block in the flat area is superimposed to obtain the histogram of the flat area.

Optionally, the acquiring the target image of each image block in the flat area according to the histogram of the flat area includes:

determining the histogram of each image block in the flat area according to the histogram of the flat area;

According to the histogram of each image block in the flat area, the target image of each image block in the flat area is acquired.

Optionally, the determining the histogram of each image block in the flat area according to the histogram of the flat area includes:

The histogram of the flat area is used as the histogram of each image block in the flat area.

According to the number of image blocks in the flat area, the histogram of the flat area is divided to obtain the histogram of each image block in the flat area.

Optionally, the method also includes:

calculating the histogram of each image block outside the flat area in the image to be processed;

According to the histogram of each image block outside the flat area, the target image of each image block outside the flat area is acquired.

Optionally, acquiring the target image of the image block according to the histogram of the image block includes:

performing clipping processing on the histogram according to the clipping threshold corresponding to the histogram of the image block;

performing equalization processing on the cropped histogram to obtain a mapping function corresponding to the image block;

The target image of the image block is determined according to the mapping function corresponding to the image block.

Optionally, the clipping thresholds corresponding to the histograms of at least two image blocks are different.

Optionally, the clipping threshold corresponding to the histogram of the image blocks outside the flat area is greater than the clipping threshold corresponding to the histogram of the image blocks in the flat area.

Optionally, the method also includes:

determining information entropy of the image block and/or distribution parameters of pixel values in the image block according to the histogram of the image block;

A clipping threshold corresponding to the histogram is determined according to information entropy of the image block and/or a distribution parameter of pixel values in the image block.

Optionally, the clipping threshold is positively correlated with the information entropy of the image block.

Optionally, the distribution parameter of pixel values in the image block includes a standard deviation of pixel values in the image block, and the clipping threshold is positively correlated with the standard deviation; and/or

The distribution parameters of the pixel values in the image include the mean value of the pixel values in the image block, and the clipping threshold is negatively correlated with the mean value.

Optionally, the determining the clipping threshold corresponding to the histogram according to the information entropy of the image block and/or the distribution parameters of pixel values in the image block includes:

The quotient of the standard deviation and the mean value of the pixel values in the image block is multiplied by a preset weight coefficient, and the information entropy of the image block is added to obtain the clipping threshold corresponding to the histogram.

Optionally, the method also includes:

Smooth the clipping thresholds corresponding to the histograms of adjacent image blocks.

Optionally, the smoothing the clipping threshold corresponding to the histogram of the adjacent image block includes:

The clipping thresholds corresponding to the histograms of adjacent image blocks are smoothed by mean filtering.

Optionally, performing clipping processing on the histogram according to the clipping threshold corresponding to the histogram of the image block includes:

determining an area in the histogram of the image block whose magnitude is higher than the clipping threshold;

Determine the rising height of the amplitude in the histogram according to the quotient of the area and the horizontal width of the histogram;

If the magnitude in the histogram is lower than the difference between the clipping threshold and the rise height, increase the magnitude according to the rise height; and/or, if the magnitude in the histogram is higher than the The difference between the clipping threshold and the rising height, and set the amplitude as the clipping threshold.

Optionally, the determining the target image of the image block according to the mapping function corresponding to the image block includes:

Determine the equalized pixel value of the center pixel of the image block according to the mapping function corresponding to the image block;

determining a preset number of image blocks adjacent to non-central pixels of the image block;

determining the preset number of pixel values corresponding to the non-central pixel according to the mapping function corresponding to the preset number of image blocks;

Interpolation processing is performed on the preset number of pixel values to obtain the equalized pixel values of the non-central pixels.

Optionally, the acquisition of images to be processed includes:

Get the image to be processed in the HSL domain or the image to be processed in the HSV domain.

Optionally, the image processing method processes at least one of the following components of the image to be processed: a saturation component and a brightness component.

In the second aspect, the embodiment of the present application provides an image processing device, including one or more processors, working individually or jointly, for performing the following steps:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

Computing a histogram of said flat area;

Optionally, when the processor executes the determining the flat area in the image to be processed, it is configured to:

Optionally, when the processor executes the calculation of the histogram of the flat area, it is used to execute:

Optionally, when the processor executes the step of obtaining the target image of each image block in the flat area according to the histogram of the flat area, it is configured to:

Optionally, when the processor performs the determining the histogram of each image block in the flat area according to the histogram of the flat area, it is configured to:

Optionally, the processor is also used for:

Optionally, when the processor acquires the target image of the image block according to the histogram of the image block, it is configured to:

Optionally, the processor is also used for:

Optionally, when the processor executes determining the clipping threshold corresponding to the histogram according to the information entropy of the image block and/or the distribution parameters of the pixel values in the image block, it is used to execute:

Optionally, the processor is also used for:

Optionally, when the processor executes the smoothing processing on the clipping thresholds corresponding to the histograms of adjacent image blocks, it is used to execute:

Optionally, the processor executes the clipping threshold corresponding to the histogram of the image block, and when clipping the histogram, executes:

determining the rising height of the amplitude in the histogram according to the quotient of the area and the horizontal width of the histogram;

Optionally, when the processor executes the mapping function corresponding to the image block to determine the target image of the image block, it is used to execute:

Optionally, when the processor executes the acquisition of the image to be processed, it is used to execute:

In a third aspect, the embodiment of the present application provides a terminal device, which is characterized in that it includes one or more processors, working individually or jointly, to perform the following steps:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

Computing a histogram of said flat area;

Optionally, when the processor executes the acquisition of the target image of each image block in the flat area according to the histogram of the flat area, it is used to execute:

Optionally, the processor is also used for:

Interpolation processing is performed on the preset number of pixel values to obtain the pixel values after equalization processing of the non-central pixels.

In a fourth aspect, the embodiment of the present application provides a movable platform, the movable platform is equipped with a photographing device, and the photographing device is used to acquire images;

Also included is one or more processors, working individually or jointly, for performing the following steps:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

Computing a histogram of said flat area;

Optionally, when the processor executes the determining the histogram of each image block in the flat area according to the histogram of the flat area, it is used to execute:

Optionally, the processor is also used for:

In the fifth aspect, the embodiment of the present application provides an image processing system, including a shooting device and a display device, and also includes one or more processors, working individually or jointly, for performing the following steps to process the shooting The image acquired by the device is processed by:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

Computing a histogram of said flat area;

acquiring a target image of each image block in the flat area according to the histogram of the flat area;

The display device can display the image processed by the processor.

Optionally, the processor is also used for:

In a sixth aspect, the embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the above method.

Embodiments of the present application provide an image processing method, device, system, and storage medium, by using adjacent image blocks whose sum of information entropy is less than or equal to a preset threshold as a class, that is, a flat area, to realize image processing in an image Blocks are classified more intelligently to prevent different objects from being divided into the same area, and then according to the histogram of the flat area, such image blocks with certain similarities are processed, and the image blocks in the flat area are stretched more evenly , to prevent distortion, can reduce image distortion, and the processing effect is better.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the disclosure content of the embodiments of the present application.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a schematic flow diagram of an image processing method provided in an embodiment of the present application;

Fig. 2 is a schematic diagram of an application scene of an image processing method in an embodiment;

Fig. 3 is a schematic diagram of dividing an image to be processed into a plurality of image blocks in an embodiment;

Fig. 4 is a schematic diagram of a flat area in an image to be processed in an embodiment;

Fig. 5 is a schematic diagram of an image to be processed in an embodiment;

Fig. 6 is a schematic diagram of an image to be processed after image processing in one embodiment;

Fig. 7 is a schematic diagram of a processed image of an image to be processed in another embodiment;

Fig. 8 is a schematic diagram of clipping a histogram in an embodiment;

Fig. 9 is a schematic diagram of clipping thresholds corresponding to different image blocks in an embodiment;

Fig. 10 is a schematic diagram of image blocks adjacent to different pixels in an embodiment;

Fig. 11 is a schematic block diagram of an image processing device provided by an embodiment of the present application;

Fig. 12 is a schematic block diagram of a mobile platform provided by an embodiment of the present application;

Fig. 13 is a schematic block diagram of an image processing system provided by an embodiment of the present application;

Fig. 14 is a schematic block diagram of a terminal device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

The flow charts shown in the drawings are just illustrations, and do not necessarily include all contents and operations/steps, nor must they be performed in the order described. For example, some operations/steps can be decomposed, combined or partly combined, so the actual order of execution may be changed according to the actual situation.

Some implementations of the present application will be described in detail below in conjunction with the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of an image processing method provided in an embodiment of the present application. The image processing method can be applied in an image processing device or an image processing system, and is used for processing an image according to a histogram of the image, and the like.

Wherein, the image processing equipment includes, for example, at least one of an image signal processor (Image Signal Processor, ISP), a digital signal processor (Digital Signal Processor, DSP), and an application processor (Application Processor, AP), and of course it is not limited to Therefore, for example, a SoC (System on Chip, System on Chip) chip may be included. It should be noted that the image processing device may or may not include an image sensor.

Wherein, the image processing system includes, for example, at least one of a terminal device, a mobile platform, and a server. For example, a terminal device may include at least one of a mobile phone, a camera, a video camera, a tablet computer, a notebook computer, a personal digital assistant, a wearable device, a remote control, etc.; At least one of people and vehicles. Furthermore, the unmanned aerial vehicle may be a rotor-type drone, such as a quad-rotor drone, a hexacopter drone, an octo-rotor drone, or a fixed-wing drone. Exemplarily, the server may be an individual server, or may be a server cluster.

In some application scenarios, as shown in Figure 2, the camera device 110 carried by the movable platform 100 acquires images in real time, and processes the images according to the image processing method; terminal device 200. The terminal device 200 may be, for example, a mobile phone, a computer, FPV (First Person View, first-person perspective) glasses, and the like. The display device 210 included in the terminal device 200 can display images received from the mobile platform 100 for viewing by the user.

In other application scenarios, as shown in FIG. 2 , the camera device 110 mounted on the mobile platform 100 acquires images in real time, and sends the acquired images to the terminal device 200 communicatively connected to the mobile platform 100 . The terminal device 200 processes the image received from the mobile platform 100 according to the image processing method, and can also display the processed image for the user to watch.

In some other implementations, the terminal device acquires images through the equipped camera, or acquires locally stored images, or acquires images from the server, and processes the acquired images according to the image processing method, and can also display the processed images, for users to view.

As shown in FIG. 1 , the image processing method of the embodiment of the present application includes step S110 to step S130.

S110. Acquire an image to be processed.

It should be noted that the image to be processed can be stored locally, or collected in real time by the camera, or obtained from other devices through a communication link, and the image to be processed can be an independent image or a video stream The images in , of course, are not limited to this.

Optionally, the acquiring the image to be processed includes: acquiring the image to be processed in the HSL (Hue, Saturation, Lightness) domain or HSV (Hue, Saturation, Value, hue, saturation, lightness) The pending image for the domain. Of course it is not limited to this, the image to be processed can be RGB (Red, Green, Blue, red, green, blue) domain, CMY/CMYK (Cyan, Magenta, Yellow, Black, cyan, magenta, yellow, black) domain, Lab (Lightness means brightness, a is from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value), b is from bright blue (low brightness value) to gray (medium brightness value) ) to yellow (high brightness value)) domain or YUV/YCbCr (Y represents brightness, U represents chroma, V represents density, Cb represents the degree of offset to blue, Cr represents the degree of offset to red) domain image.

Optionally, the image processing method processes at least one of the following components of the image to be processed: a saturation component and a brightness component. For example, the image processing method may process the saturation component of the image to be processed, or the image processing method may process the saturation component of the image to be processed, or the image processing method may Process the saturation component and brightness component of the image to be processed. For example, the brightness component may be the brightness (Lightness) component of the image to be processed in the HSL domain, or may be the value (Value) component of the image to be processed in the HSV domain, and of course it is not limited thereto.

Optionally, the image to be processed is converted into an image to be processed in the HSL domain or an image to be processed in the HSV domain, and then at least one component is processed; after the processing is completed, it can also be converted to the original color space (domain) Image. For example, converting the image to be processed to the HSL domain and processing the brightness and saturation components can ensure that the contrast and color of the image are well stretched, corrected to an appropriate range, and avoid the RGB domain histogram Chromatic aberration caused by equalization.

S120. Divide the image to be processed into multiple image blocks.

Optionally, the image to be processed is divided into multiple image blocks with multiple rows and multiple columns, for example, 8 rows and 8 columns with 64 image blocks in total, as shown in FIG. 3 .

Optionally, the sizes of the divided image blocks are all the same, that is, the image to be processed is evenly divided into a plurality of image blocks. Optionally, if the length of the image to be processed cannot be divisible by the preset number of columns of the image block, the image to be processed can be extended horizontally to an integer multiple of the number of columns, if the width of the image to be processed cannot be divided by the preset If the number of rows of the image block is evenly divided, the image to be processed can be extended to an integer multiple of the number of rows in the vertical direction. Of course, in some other implementation manners, the sizes of the multiple divided image blocks may also be different, for example, the sizes of the multiple divided image blocks may be determined according to the composition of the image to be processed.

S130. Determine a flat area in the image to be processed.

For example, the object distances corresponding to the pixels in the flat area are approximately the same. The flat area in the image to be processed may include sky, water, road, wall, etc. In some implementations, the flat area is, for example, a background area.

Specifically, the flat area includes at least two image blocks, and in the flat area, each image block has at least one adjacent image block, and all the image blocks in the flat area The sum of the information entropies of the image blocks is less than or equal to the preset threshold. The image blocks in the image are classified more intelligently, and the adjacent image blocks whose sum of information entropy is less than or equal to the preset threshold are regarded as a class, that is, the flat area, and the image blocks in the same flat area have similarity, grayscale Image characteristics such as color, texture, etc. have no or small mutations, such as the grayscale of each pixel in an image block in the same flat area is in the same or similar grayscale range. It can prevent image blocks with different image properties from being divided into one flat area, thereby preventing distortion phenomena such as distortion between areas with different image properties in the processed image.

The information entropy of an image is a statistical form of a feature, which reflects the average amount of information in the image. The information entropy of the image block can be one-dimensional entropy or two-dimensional entropy, wherein the one-dimensional entropy represents the amount of information contained in the aggregation characteristics of the grayscale distribution of the image block, based on the one-dimensional entropy The two-dimensional entropy of the image block can be obtained by introducing a feature quantity that can reflect the spatial characteristics of the gray distribution.

Optionally, according to the histogram of each image block in the image to be processed, determine the information entropy of each image block; if the sum of the information entropy of at least two adjacent image blocks is less than or equal to the preset threshold , determining that the at least two adjacent image blocks are located in the same flat area.

Optionally, a preset component of each image block is calculated, such as a histogram of a brightness component, and information entropy of each image block is determined according to the histogram. For example, the information entropy H of the image block is determined according to the following formula:

Wherein, 0 to 255 is the range of the preset component, and of course it is not limited thereto, i represents any value within the range, p _i is the probability that the value i within the range appears in the image block , p _i can be determined according to the histogram of the image block. According to the information entropy of the image block, multiple adjacent image blocks can be combined to obtain the flat area, for example, if the sum of the information entropy of adjacent image blocks is less than or equal to the preset threshold, then These image blocks can be merged, if the sum of the information entropy of these image blocks and the information entropy of adjacent image blocks of these image blocks is still less than or equal to the preset threshold, then these image blocks and these image blocks Adjacent image blocks are merged until the information entropy is equal to or greater than the preset threshold, and it can be determined that the flat area is obtained from the merged image blocks.

Please refer to Fig. 4, house, tower and sky are included in the image to be processed, it is determined that the flat area in the image to be processed includes the sky on the left side of the upper half of the tower and the sky on the right side, and the image blocks in the two sky areas are all Not adjacent, can be determined as two flat areas.

It should be noted that there may be one or more flat areas in the image to be processed, and different flat areas may or may not be adjacent. For example, the image to be processed includes the wall and the sky. If the image characteristics of the wall and the sky are different, they can be divided into two flat areas. If the wall and the sky are adjacent, the two flat areas can also be adjacent. If the wall and the sky are not adjacent, then the two flat areas divided are not adjacent. Of course, if the image properties of the walls and the sky are the same and the walls and the sky are adjacent, the adjacent walls and the sky can be divided into a flat region.

S140. Calculate the histogram of the flat area.

Optionally, the histogram of each image block in the flat area is superimposed to obtain the histogram of the flat area. Of course it is not limited to this, for example, the histogram of each image block in the flat area is superimposed, and the superimposed histogram is divided by the number of image blocks in the flat area to obtain the histogram of the flat area.

For a single image block in the flat area, the histogram of the image block is prone to peaks. The histogram of the flat area in the embodiment of the present application can be obtained by superimposing the histograms of multiple image blocks in the flat area, which can prevent The histogram is spiked, so image processing works better based on the histogram of said flat areas.

S150. Acquire a target image of each image block in the flat area according to the histogram of the flat area.

Optionally, each image block in the flat area is processed through histogram equalization to obtain a corresponding target image.

Optionally, the acquiring the target image of each image block in the flat area according to the histogram of the flat area includes: determining the histogram of each image block in the flat area according to the histogram of the flat area Figure: Acquiring the target image of each image block in the flat area according to the histogram of each image block in the flat area.

Optionally, a mapping function corresponding to the image block is determined according to the histogram of the image block in the flat area, and a target image of the image block is determined according to the mapping function corresponding to the image block.

Optionally, the histogram of the image block is clipped, and the mapping function corresponding to the image block is determined according to the clipped histogram, and the mapping function is used to indicate the preset component of the image block and the image The mapping relationship between the preset components of the target image of the block, so the target image of the image block can be determined according to the mapping function.

In some embodiments, the histograms of different image blocks in the same flat area determined according to the histograms of the flat area are all the same.

Optionally, the histogram of the flat area is used as the histogram of each image block in the flat area. It can be understood that the histogram of each image block in the flat area may be the histogram of the flat area.

Optionally, according to the number of image blocks in the flat area, the histogram of the flat area is divided to obtain the histogram of each image block in the flat area.

Generally, the histogram of a single image block in a flat area tends to have peaks, and processing the image block according to the histogram of a single image block may easily cause image distortion, for example, an excessive increase in local contrast may be caused. In the embodiment of the present application, the histogram of each image block in the flat area is determined according to the histogram of the flat area, and the histogram of the entire flat area can be obtained from the histograms of multiple image blocks in the flat area The images are superimposed, so the histogram is not prone to peaks, which can reduce image distortion and improve the processing effect.

Optionally, the method further includes: calculating a histogram of each image block outside the flat area in the image to be processed; obtaining the flat area according to the histogram of each image block outside the flat area. The target image for each image patch outside the region.

It can be understood that the mapping function corresponding to the image block outside the flat area can also be determined according to the histogram of each image block outside the flat area, and according to the mapping function corresponding to the image block outside the flat area , determine the target image of the image block outside the flat area.

The information entropy of the image blocks outside the flat area is larger, or the information entropy of adjacent image blocks is larger although the information entropy is smaller. By processing each image block outside the flat area according to its respective histogram, the details of the target image of the image blocks outside the flat area can be preserved.

Optionally, according to the target image of each image block in the flat area and the target image of the image blocks outside the flat area, the processed image of the image to be processed can be obtained, as shown in FIG. 5 The image to be processed, as shown in Figure 6 and Figure 7, is the processed image of the image to be processed, the contrast and color of the image have been stretched in a better range, corrected to a suitable range, and the image distortion has been reduced, such as reducing localized image distortion in flat areas.

Optionally, performing clipping processing on the histogram according to the clipping threshold corresponding to the histogram of the image block; and performing equalization processing on the clipped histogram to obtain a mapping function corresponding to the image block. By clipping the histogram of the image block so that the amplitudes of the clipped histograms are all equal to or smaller than the clipping threshold, it is possible to prevent the image block from being processed according to the uncropped histogram, such as performing When the histogram is equalized, the local contrast is increased too much, resulting in image distortion and noise amplification.

Optionally, performing clipping processing on the histogram according to a clipping threshold corresponding to the histogram of the image block includes: determining an area in the histogram of the image block whose magnitude is higher than the clipping threshold; According to the quotient of the area and the horizontal width of the histogram, determine the rising height of the amplitude in the histogram; if the amplitude in the histogram is lower than the difference between the clipping threshold and the rising height, according to The ascent height increases the magnitude; and/or, if the magnitude in the histogram is higher than the difference between the clipping threshold and the ascent height, setting the magnitude as the clipping threshold.

Please refer to FIG. 8 , the area in the histogram of the image block whose magnitude is higher than the clipping threshold can be uniformly distributed on the horizontal width of the histogram, and the horizontal width is determined according to the range of the preset component , to determine the rising height of the partial amplitude before the clipping process, and the rising height is used to lift the amplitude in the histogram lower than the difference between the clipping threshold and the rising height during the clipping process, so that the clipping process The total area of the front and back histograms is constant, but of course it is not limited to this.

Exemplarily, the clipping threshold corresponding to the histogram of the image block is expressed as ClipLimit, and the area of the part in the histogram higher than ClipLimit is expressed as totalExcess, and the area totalExcess is divided by the horizontal width of the histogram to obtain the Rising height L=totalExcess÷N, the histogram is clipped with the difference between the clipping threshold and the rising height upper=ClipLimit−L as a limit. For example, if the amplitude in the histogram is greater than ClipLimit, set the amplitude directly to ClipLimit; if the amplitude in the histogram is greater than or equal to upper and less than or equal to ClipLimit, set the amplitude to ClipLimit; if the amplitude in the histogram is smaller than upper, increase the amplitude to the rising height L.

Please refer to Fig. 6 and Fig. 7, the clipping threshold of the histogram in Fig. 6 is smaller than the clipping threshold of the histogram in Fig. 7, when the clipping threshold is small, the contrast and The stretching of the color is not enough, and the image toning effect is not obvious enough; when the clipping threshold is large, the contrast and color stretching of the flat area of the image, such as the box in Figure 7, is excessive, and the image toning effect is somewhat abrupt, such as As shown in Figure 7, the contrast stretching to the sky is overly exaggerated. It can be seen that the size of the clipping threshold has a great influence on the effect of image processing. By determining different histogram clipping thresholds for different image blocks in the image to be processed, it can be ensured that each region in the image to be processed can be properly processed, such as contrast stretching, and the effect of image processing is better .

In some implementations, the clipping threshold corresponding to the histogram of the image block outside the flat area is greater than the clipping threshold corresponding to the histogram of the image block in the flat area, so the stretching degree of the non-flat area can be guaranteed. And image toning effect, it can also prevent the contrast and color stretching in flat areas, and ensure that each area gets the appropriate contrast stretching.

Optionally, the method further includes: determining information entropy of the image block and/or distribution parameters of pixel values in the image block according to the histogram of the image block; Or the distribution parameters of the pixel values in the image block, and determine the clipping threshold corresponding to the histogram. Therefore, the adaptive adjustment of the clipping thresholds of different image blocks can be realized.

Optionally, the cropping threshold is positively correlated with the information entropy of the image block, which can improve the contrast and color stretching of the image block with larger information entropy, and the effect of image toning is more obvious, and reduces the impact on image blocks with smaller information entropy. The contrast and color stretching of image blocks prevents excessive stretching.

Optionally, the distribution parameter of pixel values in the image block includes a standard deviation of pixel values in the image block, and the clipping threshold is positively correlated with the standard deviation. The clipping threshold corresponding to the histogram of the image block may be adjusted according to the degree of dispersion of pixel values in the image block, and image blocks with a higher degree of dispersion may be more stretched.

Optionally, the distribution parameter of pixel values in the image includes an average value of pixel values in the image block, and the clipping threshold is negatively correlated with the average value. An image block with a lower average value of pixel values in the image block can be more stretched.

Optionally, the quotient of the standard deviation and the mean value of the pixel values in the image block may be multiplied by a preset weight coefficient and added to the information entropy of the image block to obtain the clipping threshold corresponding to the histogram.

For example, the clipping threshold ClipLimit corresponding to the histogram of the image block can be determined according to the following formula:

Among them, σ represents the standard deviation of the pixel values in the image block, μ represents the mean value of the pixel values in the image block, the quotient of σ and μ can be used to indicate the image dynamic range of the image block, and α represents the preset coefficient , used to adjust the weight of the information entropy of the image block and the dynamic range of the image, for example, the value of α is 1 to 5, such as 3.

FIG. 9 is a schematic diagram of clipping thresholds corresponding to the histograms of different image blocks in the image to be processed, and image blocks with richer details have higher clipping thresholds.

Optionally, the method further includes: smoothing the clipping thresholds corresponding to the histograms of adjacent image blocks. For example, the clipping thresholds corresponding to the histograms of adjacent image blocks may be smoothed by mean filtering, but of course it is not limited thereto. Prevent certain block effects in the processed image when the clipping thresholds of adjacent image blocks differ greatly.

The mapping function of the corresponding image block can be determined according to the clipped histogram, and the processed image of the image to be processed can be determined according to the mapping function of each image in the image to be processed. Optionally, the determining the target image of the image block according to the mapping function corresponding to the image block includes: determining the equalized pixel of the central pixel of the image block according to the mapping function corresponding to the image block value; determine the preset number of image blocks adjacent to the non-central pixel of the image block; determine the preset number of pixels corresponding to the non-central pixel according to the mapping function corresponding to the preset number of image blocks value; performing interpolation processing on the preset number of pixel values to obtain the equalized pixel value of the non-central pixel. Through interpolation processing, the computational load of image processing can be reduced.

According to the mapping function of the image block, histogram equalization can be performed on the central pixel of the image block, and the image to be processed can be determined by interpolating the mapping function of the local neighborhood of any pixel in the image to be processed The mapping function of any pixel in the image to perform histogram equalization on the remaining pixels except the central pixel of each image block.

Referring to Fig. 10, the pixel value after equalization processing of the central pixel 11 of each image block is determined according to the pixel value of the central pixel 11 and the mapping function of the image block; the image blocks adjacent to the non-central pixel 12 include Four, four pixel values can be determined according to the pixel values of the non-central pixel 12 and the four mapping functions corresponding to the image blocks adjacent to the non-central pixel 12, and bilinear interpolation can be performed on the four pixel values to determine The non-central pixel 12 equalizes the pixel value after processing; the image block adjacent to the non-central pixel 13 in the image block at the edge of the image to be processed and not in the four corners includes two, which can be based on the non-central pixel 13 The pixel value and the two mapping functions corresponding to the image blocks adjacent to the non-central pixel 13 determine two pixel values, and performing linear interpolation on the two pixel values can determine the equalized pixel value of the non-central pixel 13; The image blocks adjacent to the non-central pixel 14 in the image blocks at the four corners of the image to be processed include one, that is, the image block diagonally adjacent to the non-central pixel 14, according to the non-central pixel 14 The pixel value and the mapping function corresponding to the image block can determine the equalized pixel value of the non-central pixel 14 .

A processed image of the image to be processed can be obtained according to the target image after equalization processing of all image blocks. Exemplarily, image processing is performed on the brightness component of the image to be processed, and image processing is performed on the saturation component of the image to be processed, and the color space is changed according to the brightness component and saturation component after image processing to obtain RGB Image in color space, get exposure, contrast, color corrected image. Perform histogram equalization operations on the S (saturation) and L (brightness) channels respectively to adjust the image color and contrast.

Optionally, both the brightness component and the saturation component of the image to be processed are mapped to an integer in the interval [0, 255], and then histogram equalization processing is performed, and the brightness component and saturation component after the histogram equalization processing are A float renormalized to the interval [0, 1].

The image processing method provided by the embodiment of the present application realizes more intelligent classification of the image blocks in the image by taking the adjacent image blocks whose sum of information entropy is less than or equal to the preset threshold as a class, that is, the flat area, to prevent Divide different objects in the same area, and then process such image blocks with certain similarities according to the histogram of the flat area, stretch the image blocks in the flat area more evenly, prevent distortion, and reduce image distortion. Handling is better.

In some embodiments, the image processing method can provide simple and fast intelligent image color adjustment, and can quickly generate a primary color adjustment effect with only a very simple interaction. For example, the user only needs to click the color adjustment button to directly correct the exposure and color of the screen to a normal level, realize one-click adjustment of image quality, and achieve basically satisfactory results. This method can not only process images quickly, but also be suitable for real-time processing of high-definition video. The exemplary image processing method can be conveniently configured on the editor APP of the mobile terminal, and can realize high-resolution video, such as real-time grading processing of 1080P video, so that users can edit images or videos as they like anytime, anywhere. Smart color editing. The interaction is simpler, the implementation is more convenient, and the flexibility is stronger, which greatly reduces the user's threshold for use.

Please refer to FIG. 11 . FIG. 11 is a schematic block diagram of an image processing device 400 provided by an embodiment of this specification. The image processing device 400 includes one or more processors 401 working individually or jointly for executing the steps of the image processing method of the foregoing embodiments.

Exemplarily, the image processing device includes at least one of an image signal processor (Image Signal Processor, ISP), a digital signal processor (Digital Signal Processor, DSP), an application processor (Application Processor, AP), and of course Not limited thereto, for example, a SoC (System on Chip, System on Chip) chip may be included. It should be noted that the image processing device may or may not include an image sensor.

Specifically, the processor 401 is used for:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

Computing a histogram of said flat area;

Optionally, when the processor 401 executes the determining the flat area in the image to be processed, it is configured to:

Optionally, when the processor 401 executes the calculation of the histogram of the flat area, it is configured to:

Optionally, when the processor 401 executes the acquisition of the target image of each image block in the flat area according to the histogram of the flat area, it is configured to:

Optionally, when the processor 401 executes the determining the histogram of each image block in the flat area according to the histogram of the flat area, it is configured to:

Optionally, the processor 401 is also used for:

Optionally, when the processor 401 executes acquiring the target image of the image block according to the histogram of the image block, it is used to execute:

Optionally, the processor 401 is also used for:

Optionally, when the processor 401 determines the clipping threshold corresponding to the histogram according to the information entropy of the image block and/or the distribution parameters of the pixel values in the image block, it is used to execute:

Optionally, the processor 401 is also used for:

Optionally, when the processor 401 executes the smoothing processing on the clipping threshold corresponding to the histogram of the adjacent image block, it is used to execute:

Optionally, the processor 401 executes the clipping threshold corresponding to the histogram of the image block, and when clipping the histogram, executes:

Optionally, when the processor 401 executes the mapping function corresponding to the image block to determine the target image of the image block, it is used to execute:

Optionally, when the processor 401 executes the acquisition of the image to be processed, it is used to execute:

The specific principle and implementation manner of the image processing device provided in the embodiment of this specification are similar to the image processing method in the foregoing embodiment, and will not be repeated here.

Please refer to FIG. 12 . FIG. 12 is a schematic block diagram of a mobile platform 500 provided in another embodiment of this specification. The mobile platform 500 may include at least one of an unmanned aerial vehicle, a cloud platform, an unmanned vehicle, and the like. Furthermore, the unmanned aerial vehicle may be a rotor-type drone, such as a quad-rotor drone, a hexacopter drone, an octo-rotor drone, or a fixed-wing drone. The photographing device 510 can be mounted on the movable platform 500 , and the photographing device 510 is used to acquire images, and the movable platform 500 can process the images acquired by the photographing device 510 .

The mobile platform 500 also includes one or more processors 501, and the one or more processors 501 can work individually or jointly to execute the steps of the image processing method of the foregoing embodiments.

Specifically, the processor 501 is used for:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

Computing a histogram of said flat area;

Optionally, when the processor 501 executes the determining the flat area in the image to be processed, it is configured to:

Optionally, when the processor 501 executes the calculation of the histogram of the flat area, it is configured to:

Optionally, when the processor 501 executes the acquisition of the target image of each image block in the flat area according to the histogram of the flat area, it is configured to:

Optionally, when the processor 501 executes the determining the histogram of each image block in the flat area according to the histogram of the flat area, it is configured to:

Optionally, the processor 501 is also used for:

Optionally, when the processor 501 acquires the target image of the image block according to the histogram of the image block, it is configured to:

Optionally, the processor 501 is also used for:

Optionally, when the processor 501 executes determining the clipping threshold corresponding to the histogram according to the information entropy of the image block and/or the distribution parameters of the pixel values in the image block, it is used to execute:

Optionally, the processor 501 is also used for:

Optionally, when the processor 501 executes the smoothing processing on the clipping threshold corresponding to the histogram of the adjacent image block, it is used to execute:

Optionally, the processor 501 executes the clipping threshold corresponding to the histogram of the image block, and when clipping the histogram, executes:

Optionally, when the processor 501 executes the mapping function corresponding to the image block to determine the target image of the image block, it is used to execute:

Optionally, when the processor 501 executes the acquisition of the image to be processed, it is used to execute:

The specific principles and implementations of the movable platform provided in the embodiments of this specification are similar to the image processing methods in the foregoing embodiments, and will not be repeated here.

Please refer to FIG. 13 . FIG. 13 is a schematic block diagram of an image processing system 600 provided in another embodiment of this specification.

The image processing system 600 includes, for example, at least one of a terminal device, a mobile platform, and a server. For example, a terminal device may include at least one of a mobile phone, a camera, a video camera, a tablet computer, a notebook computer, a personal digital assistant, a wearable device, a remote control, etc.; At least one of people and vehicles. Furthermore, the unmanned aerial vehicle may be a rotor-type drone, such as a quad-rotor drone, a hexacopter drone, an octo-rotor drone, or a fixed-wing drone. Exemplarily, the server may be an individual server, or may be a server cluster.

The image processing system 600 includes a photographing device 610 and a display device 620 , and also includes one or more processors 601 . In some implementations, one or more processors 601 may be mounted on the camera device 610 and/or the display device 620 .

One or more processors 601 may work individually or jointly to execute the steps of the image processing method of the foregoing embodiments, so as to process the image acquired by the photographing device 610 . The display device 620 can display the image processed by the processor 601 .

The specific principles and implementations of the image processing system provided in the embodiments of this specification are similar to the image processing methods in the foregoing embodiments, and will not be repeated here.

Please refer to FIG. 14 , which is a schematic block diagram of a terminal device 700 provided in another embodiment of this specification.

The terminal device 700 may include at least one of a mobile phone, a camera, a video camera, a tablet computer, a notebook computer, a personal digital assistant, a wearable device, a remote controller, and the like.

The terminal device 700 includes one or more processors 701, and the one or more processors 701 can work individually or jointly to execute the steps of the image processing method of the foregoing embodiments.

An embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image processing method provided in the above-mentioned embodiment step.

Wherein, the computer-readable storage medium may be an internal storage unit of the system, device, or removable platform described in any of the foregoing embodiments, such as a hard disk or memory of the removable platform. The computer-readable storage medium can also be an external storage device of the system, device or removable platform, such as a plug-in hard disk equipped on the removable platform, a smart memory card (Smart Media Card, SMC), a security Digital (Secure Digital, SD) card, flash memory card (Flash Card), etc.

It should be understood that the terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the application.

It should also be understood that the term "and/or" used in this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

The above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the scope of the technology disclosed in the application. Modifications or replacements, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

An image processing method, characterized in that, comprising:

Get the image to be processed;

dividing the image to be processed into a plurality of image blocks;

determining a flat area in the image to be processed, the flat area includes at least two image blocks, and in the flat area, each image block has at least one adjacent image block, The sum of the information entropies of all the image blocks in the flat area is less than or equal to a preset threshold;

Computing a histogram of said flat area;

The target image of each image block in the flat area is acquired according to the histogram of the flat area.
The image processing method according to claim 1, wherein said determining the flat area in the image to be processed comprises:

Determine the information entropy of each image block according to the histogram of each image block in the image to be processed;

If the sum of the information entropies of at least two adjacent image blocks is less than or equal to the preset threshold, it is determined that the at least two adjacent image blocks are located in the same flat area.
The image processing method according to claim 2, wherein said calculating the histogram of said flat region comprises:

The histogram of each image block in the flat area is superimposed to obtain the histogram of the flat area.
The image processing method according to any one of claims 1-3, wherein said obtaining the target image of each image block in the flat area according to the histogram of the flat area comprises:

determining the histogram of each image block in the flat area according to the histogram of the flat area;

According to the histogram of each image block in the flat area, the target image of each image block in the flat area is acquired.
The image processing method according to claim 4, wherein said determining the histogram of each image block in the flat area according to the histogram of the flat area comprises:

The histogram of the flat area is used as the histogram of each image block in the flat area.
The image processing method according to claim 4, wherein said determining the histogram of each image block in the flat area according to the histogram of the flat area comprises:

According to the number of image blocks in the flat area, the histogram of the flat area is divided to obtain the histogram of each image block in the flat area.
The image processing method according to claim 4, wherein the method further comprises:

calculating the histogram of each image block outside the flat area in the image to be processed;

According to the histogram of each image block outside the flat area, the target image of each image block outside the flat area is acquired.
The image processing method according to any one of claims 4-7, wherein acquiring the target image of the image block according to the histogram of the image block comprises:

performing clipping processing on the histogram according to the clipping threshold corresponding to the histogram of the image block;

performing equalization processing on the cropped histogram to obtain a mapping function corresponding to the image block;

The target image of the image block is determined according to the mapping function corresponding to the image block.
The image processing method according to claim 8, wherein the clipping thresholds corresponding to the histograms of at least two image blocks are different.
The image processing method according to claim 9, wherein the clipping threshold corresponding to the histogram of the image block outside the flat area is greater than the clipping threshold corresponding to the histogram of the image block in the flat area.
The image processing method according to any one of claims 8-10, wherein the method further comprises:

determining information entropy of the image block and/or distribution parameters of pixel values in the image block according to the histogram of the image block;

A clipping threshold corresponding to the histogram is determined according to information entropy of the image block and/or a distribution parameter of pixel values in the image block.
The image processing method according to claim 11, wherein the clipping threshold is positively correlated with the information entropy of the image block.
The image processing method according to claim 11, wherein the distribution parameter of pixel values in the image block comprises a standard deviation of pixel values in the image block, and the clipping threshold is positively correlated with the standard deviation; and /or

The distribution parameters of the pixel values in the image include the mean value of the pixel values in the image block, and the clipping threshold is negatively correlated with the mean value.
The image processing method according to claim 11, wherein the determining the clipping threshold corresponding to the histogram according to the information entropy of the image block and/or the distribution parameters of the pixel values in the image block includes :

The quotient of the standard deviation and the mean value of the pixel values in the image block is multiplied by a preset weight coefficient, and the information entropy of the image block is added to obtain the clipping threshold corresponding to the histogram.
The image processing method according to any one of claims 8-14, wherein the method further comprises:

Smooth the clipping thresholds corresponding to the histograms of adjacent image blocks.
The image processing method according to claim 15, wherein said smoothing the clipping thresholds corresponding to the histograms of adjacent image blocks comprises:

The clipping thresholds corresponding to the histograms of adjacent image blocks are smoothed by mean filtering.
The image processing method according to any one of claims 8-16, wherein the clipping processing of the histogram according to the clipping threshold corresponding to the histogram of the image block includes:

determining an area in the histogram of the image block whose magnitude is higher than the clipping threshold;

determining the rising height of the amplitude in the histogram according to the quotient of the area and the horizontal width of the histogram;

If the magnitude in the histogram is lower than the difference between the clipping threshold and the rise height, increase the magnitude according to the rise height; and/or, if the magnitude in the histogram is higher than the The difference between the clipping threshold and the rising height, and set the amplitude as the clipping threshold.
The image processing method according to any one of claims 8-16, wherein the determining the target image of the image block according to the mapping function corresponding to the image block includes:

Determine the equalized pixel value of the center pixel of the image block according to the mapping function corresponding to the image block;

determining a preset number of image blocks adjacent to non-central pixels of the image block;

determining the preset number of pixel values corresponding to the non-central pixel according to the mapping function corresponding to the preset number of image blocks;

Interpolation processing is performed on the preset number of pixel values to obtain the equalized pixel values of the non-central pixels.
The image processing method according to any one of claims 1-18, wherein said acquiring the image to be processed comprises:

Get the image to be processed in the HSL domain or the image to be processed in the HSV domain.
The image processing method according to claim 19, characterized in that the image processing method processes at least one of the following components of the image to be processed: a saturation component and a brightness component.
An image processing device, characterized by comprising one or more processors working individually or jointly to execute the steps of the image processing method according to any one of claims 1-20.
A terminal device, characterized by comprising one or more processors working individually or jointly to execute the steps of the image processing method according to any one of claims 1-20.
An image processing system, characterized in that it includes a photographing device and a display device;

It also includes one or more processors, working individually or jointly, for executing the steps of the image processing method according to any one of claims 1-20, so as to process the image acquired by the shooting device;

The display device can display the image processed by the processor.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements any one of claims 1-20. The image processing method described above.