WO2022151054A1

WO2022151054A1 - Image processing method and apparatus, and storage medium

Info

Publication number: WO2022151054A1
Application number: PCT/CN2021/071514
Authority: WO
Inventors: 周长波
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-01-13
Filing date: 2021-01-13
Publication date: 2022-07-21

Abstract

An image processing method and apparatus, and a storage medium. The image processing method (100) comprises: acquiring a first image of a first dynamic range of a photographing scenario (S110); analyzing the first image to determine an exposure abnormality region in the first image (S120); processing at least the image of the exposure abnormality region to obtain a processed image (S130), wherein the processed image has a second dynamic range, and the second dynamic range is higher than the first dynamic range; and displaying the first image and the processed image (S140). In this way, a user can intuitively observe the exposure state under a current photographing condition, and whether the effect, after processing, of an image photographed under the current photographing condition conforms to the requirement of the user, thereby assisting the user to perform exposure control, achieving accurate exposure, and improving user experience.

Description

Image processing method and device, and storage medium

manual

technical field

The present invention generally relates to the technical field of data processing, and more particularly to an image processing method and device, and a storage medium.

Background technique

Most cameras have a digital viewfinder, which can be used to observe the frame and brightness, color, etc. of a picture or video captured under the current shooting conditions. Under some shooting conditions, including when the exposure time is too long or the shooting scene is too bright, parts of the image can be so bright that they saturate to pure white. Usually some cameras have the function of overexposure prompting, but prompting that a specific area has been overexposed cannot meet the needs of users.

Therefore, in view of the existence of the above problems, the present application provides a new image processing method, device, and storage medium.

SUMMARY OF THE INVENTION

The present invention has been made to solve at least one of the above-mentioned problems. Specifically, one aspect of the present invention provides an image processing method, the method includes: acquiring a first image of a first dynamic range of a shooting scene; analyzing the first image to determine exposure in the first image an abnormal area, the abnormal exposure area includes an overexposed area and/or an underexposed area; at least an image of the abnormally exposed area is processed to obtain a processed image, wherein the processed image has a second dynamic range, and the first 2. The dynamic range is higher than the first dynamic range; the first image and the processed image are displayed.

Another aspect of the present invention provides an image processing apparatus, the image processing apparatus includes: a memory for storing executable program instructions; one or more processors for executing the program instructions stored in the memory , causing the processor to perform the following steps:

acquiring a first image of a first dynamic range of the shooting scene;

Analyzing the first image to determine an abnormally exposed area in the first image, where the abnormally exposed area includes an overexposed area and/or an underexposed area;

processing at least the image of the abnormally exposed area to obtain a processed image, wherein the processed image has a second dynamic range, and the second dynamic range is higher than the first dynamic range;

The display unit is used for displaying the first image and the processed image.

Another aspect of the present application provides a computer storage medium on which a computer program is stored, the computer program implements the aforementioned image processing method when executed by a processor.

According to the image processing method of the present application, at least the image of the overexposed area and/or the underexposed area is processed, and the first image and the processed image are displayed, so that the user can intuitively observe the current shooting conditions The exposure state of the camera, and whether the processed effect of the image captured under the current shooting conditions meets the user's requirements, so as to assist the user in exposure control, achieve accurate exposure, and improve the user's experience.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 shows a schematic flowchart of an image processing method in an embodiment of the present application;

2 shows a schematic diagram of a standard dynamic range image of a shooting scene displayed on a camera screen;

Figure 3 shows a schematic diagram when only overexposed areas are identified with prompt information;

FIG. 4 shows a schematic diagram of a prompting manner of an overexposed area in an embodiment of the present application;

FIG. 5 shows a schematic block diagram of an image processing apparatus in an embodiment of the present application.

Detailed ways

In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of the embodiments of the present invention, and it should be understood that the present invention is not limited by the example embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without one or more of these details. In other instances, some technical features known in the art have not been described in order to avoid obscuring the present invention.

It should be understood that the present invention may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the/the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the terms "compose" and/or "include", when used in this specification, identify the presence of stated features, integers, steps, operations, elements and/or components, but do not exclude one or more other The presence or addition of features, integers, steps, operations, elements, parts and/or groups. As used herein, the term "and/or" includes any and all combinations of the associated listed items.

For a thorough understanding of the present invention, detailed structures will be presented in the following description in order to explain the technical solutions proposed by the present invention. Alternative embodiments of the present invention are described in detail below, however, the invention is capable of other embodiments in addition to these detailed descriptions.

Some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described below and features in the embodiments may be combined with each other without conflict.

Most cameras have a digital viewfinder, which can be used to observe the frame and brightness, color, etc. of a picture or video captured under the current shooting conditions. Under some shooting conditions, including when the exposure time is too long or the shooting scene is too bright, parts of the image can be so bright that they saturate to pure white. Many cameras have the function of overexposure prompting, the most common one is "zebra crossing", that is, displaying oblique black and white alternating lines in the area where the brightness exceeds a specified threshold to remind the user that this area has been overexposed (that is, overexposed). Excessive areas show some line-like patterns to help achieve accurate exposure.

When using High Dynamic Range (HDR) shooting technology, the saturated areas in some images can be recovered through the additional information retained in the file. In this shooting mode, it is not enough to only prompt that a specific area has been overexposed, and the user also needs to know whether this area can be recovered through the additionally retained information in the file and what effect the recovery can achieve.

Therefore, in view of the existence of the above problems, an image processing method is proposed in an embodiment of the present application. The method includes: acquiring a first image of a first dynamic range of a shooting scene; analyzing the first image to determine the first image an abnormally exposed area in an image, the abnormally exposed area includes an overexposed area and/or an underexposed area; at least the image of the abnormally exposed area is processed to obtain a processed image, wherein the processed image has a second dynamic range, the second dynamic range is higher than the first dynamic range; displaying the first image and the processed image.

Below, the image processing method of the present application will be described with reference to FIGS. 1 to 4 , wherein FIG. 1 shows a schematic flowchart of the image processing method in an embodiment of the present application; A schematic diagram of a standard dynamic range image of a shooting scene; FIG. 3 shows a schematic diagram when only the overexposed area is marked with prompt information; FIG.

As an example, as shown in FIG. 1 , the image processing method 100 according to the embodiment of the present application includes the following steps:

First, in step S110, a first image of a first dynamic range of the shooting scene is acquired.

The first image may be an image or a video, etc., and a still image, a dynamic image, or a video of a shooting scene may be captured by a shooting device. The first image may also be a live view image or the like that is continuously and real-time collected by a photographing device during viewfinder.

In an example, the acquiring the first image of the first dynamic range of the shooting scene includes: acquiring image code stream data, wherein the image code stream data includes a base layer code stream and an enhancement layer image code stream; decoding the image code stream base layer code stream to obtain the first image. That is, the image code stream data may be data encoded based on the encoder, which may include the base layer code stream. When the base layer code stream is used to obtain the first image with the first dynamic range, the enhancement layer image code stream is used to obtain the first image with the first dynamic range. Two dynamic range processed images, wherein the second dynamic range is higher than the first dynamic range, for example, the first image is a standard dynamic range image (including but not limited to JPEG images), and the processed image is a high dynamic range image .

In another example, the acquiring the first image of the first dynamic range of the shooting scene includes: acquiring original image data, and acquiring the first image of the first dynamic range of the shooting scene based on the original image data. The raw image data may be unencoded image data, such as RAW data collected by an image sensor.

In other examples, the base layer image code stream may also be directly obtained, and the base layer image code stream may be decoded to obtain the first image. The base layer code stream may be an independent code stream, and the encoding process may not involve a series of processing processes on the enhancement layer image data.

Next, in step S120, the first image is analyzed to determine an abnormally exposed area in the first image, and the abnormally exposed area includes an overexposed area and/or an underexposed area.

Overexposed areas and/or underexposed areas in the first image can be determined based on any suitable method known to those skilled in the art.

In one example, analyzing the first image to determine an abnormally exposed area in the first image includes: determining the exposure in the first image based on pixel information of pixels of the first image abnormal area. For example, comparing the brightness information of each area in the first image with a preset brightness threshold to determine an abnormally exposed area in the first image, where the preset brightness threshold includes a first preset brightness threshold or a second preset brightness threshold, the first preset brightness threshold is greater than the second preset brightness threshold, the overexposed area includes an area whose brightness information is greater than the first preset brightness threshold, the underexposed area Including an area whose brightness information is less than the second preset brightness threshold. The first preset brightness threshold and the second preset brightness threshold may be preset parameters in the photographing device, or may be suitable parameters specified according to user input, for example, the first preset brightness threshold may be saturated brightness 70%, 80%, 90%, 100%, etc. The second preset brightness threshold may be, for example, 40%, 30%, 20%, 10%, etc. of the saturated brightness. The pixels included in the overexposed area and the pixels included in the underexposed area can be determined by comparing the luminance value of each pixel in the first image with the luminance threshold.

It is worth mentioning that the first image may include both an overexposed area and an underexposed area, or may include only one of the overexposed area and the underexposed area.

Next, in step S130, at least the image of the abnormally exposed area is processed to obtain a processed image, wherein the processed image has a second dynamic range, and the second dynamic range is higher than the first dynamic range.

The image of the abnormally exposed area may be processed by any suitable method known to those skilled in the art to obtain a processed image. The processed image has a second dynamic range, the second dynamic range is higher than the first dynamic range, the second dynamic range is higher than the first dynamic range, for example, the first image is a standard dynamic range image (including but not limited to JPEG images), the processed images are high dynamic range images.

In an example, the processing at least the image of the abnormally exposed area to obtain the processed image includes: acquiring image code stream data, wherein the image code stream data includes a base layer code stream and an enhancement layer image code stream ; Decode the base layer code stream and the enhancement layer image code stream to obtain the base layer image data and the enhancement layer image data; fuse the base layer image data and the enhancement layer image data to obtain a second image; extracting the image corresponding to the abnormal exposure area in the second image to obtain the processed image. Compared with the abnormally exposed area in the first image, the processed image has better brightness and better display effect.

In one example, the processing at least the image of the abnormally exposed area to obtain the processed image includes: when the first image includes an overexposed area, processing the second image to reduce For the brightness of the second image, for example, the brightness of the second image is reduced by applying a preset mapping function to the pixel values of the second image. Linear curve) or linear mapping (for example, applying a linear mapping similar to y=x/2 to pixel values), or, it can also be any other suitable processing method that can reduce the brightness of the second image.

After the brightness reduction processing is performed on the second image, extracting the image corresponding to the overexposed area in the second image to obtain the processed image, comprising: extracting the processed image corresponding to the overexposed area in the second image. The image corresponding to the overexposed area is obtained to obtain the processed image. The processed image extracted at this time has both high dynamic range and better brightness than the image corresponding to the overexposed area in the first image.

In one example, the processing at least the image of the abnormally exposed area to obtain the processed image includes: when the first image includes an underexposed area, processing the second image to improve the the brightness of the second image. For example, the brightness of the second image is improved by applying a preset mapping function to the pixel values of the second image, and the preset mapping function may be, for example, a nonlinear mapping (eg, a preset nonlinear curve) or a linear mapping (eg A linear mapping similar to y=2x is applied to the pixel values), or it can be any other suitable processing method that can improve the brightness of the second image.

After the brightness enhancement process is performed on the second image, extracting the image corresponding to the overexposed area in the second image to obtain the processed image, comprising: extracting the processed image corresponding to the overexposed area in the second image. image corresponding to the underexposed area to obtain the processed image. The processed image extracted at this time has both high dynamic range and better brightness than the image corresponding to the overexposed area in the first image.

In other examples, the processed image may also be extracted first, and then the processed image may be subjected to brightness reduction processing or brightness enhancement processing in substantially the same manner as described above.

In another example, an area may be determined first, and only the corresponding base layer image data and enhancement layer image data of the abnormally exposed area may be fused. For example, at least the image of the abnormally exposed area is processed to Obtaining a processed image includes: acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream; decoding the base layer image code stream to obtain base layer image data; The enhancement layer code stream is decoded to obtain enhancement layer image data; the base layer image data corresponding to the abnormal exposure area and the enhancement layer image data corresponding to the abnormal exposure area are fused to obtain a processed image. Further, brightness processing can also be performed on the processed image, for example, brightness reduction processing is performed on the processed image corresponding to the overexposed area, and brightness enhancement processing is performed on the processed image corresponding to the underexposed area, so as to obtain a more ideal brightness. For the processing method, reference may be made to the foregoing processing method for the brightness of the second image, which will not be repeated here.

In this embodiment of the present application, the image code stream data may be data encoded by an encoder, and obtaining the obtained image code stream data includes the following steps S1 to S6:

First, in step S1, the original image data is acquired, and the image data to be processed is obtained by performing nonlinear image transformation processing on the original image data.

The original image data is high dynamic range image data. The original image data includes linear image data or nonlinear image data. The linear image data may be image data acquired by an image sensor. The nonlinear image data may be high dynamic range image data generated based on multiple images, or may be image data acquired by an image sensor.

Wherein, the original image data may refer to the original image data collected by the photographing device of the image processing apparatus, the original image data may be linear image data, and the image data may be linear HDR image data. Wherein, the photographing device may be a camera, a video camera, a camera, etc., for example, the original image data may be obtained through a camera of a mobile phone, and the original image data may be 16-bit (bit) linear image data.

After the original image data is acquired, the image data to be processed may be acquired from the original image data, which may be non-linear image data, and the size may be 12 bits. For example, according to the foregoing, the camera of the mobile phone captures an image of the external environment, and the image is 16-bit linear image data. After the encoder in the mobile phone obtains the image data, it can obtain 12-bit nonlinear image data through nonlinear image transformation, which can be used as the intermediate layer image data, and can be directly used as the image data to be processed. The image data to be processed can be non-linear image data. Linear HDR image data.

In another embodiment, original image data is acquired, and image transformation processing is performed on the original image data to obtain image data to be processed. The image transformation processing may include linear transformation processing, nonlinear transformation processing, or a combination of linear transformation processing and nonlinear transformation processing.

Next, in step S2, extract the base layer image data of a preset data amount from the to-be-processed image data or the original image data;

In addition, the base layer image data refers to the image data of the base data bit depth, that is, the low bit depth image data, and the dynamic range thereof is also the base dynamic range. Such as 8bit image data. The base layer image data is compliant with the displayable image data of conventional display devices.

Wherein, extracting the base layer image data of the preset data amount from the to-be-processed image data or the original image data includes: performing linear image transformation processing on the to-be-processed image data or the original image data to obtain the base layer image data of the preset data amount.

The preset data amount may be determined according to the requirements of the electronic device associated with or where the decoder is located, such as 8 bits.

For example, according to the foregoing, the encoder in the mobile phone can perform linear cropping after acquiring the above-mentioned 12-bit image data, and directly obtain the linearly cropped 8-bit base layer image data.

It should be noted that, the base layer image data of the preset data amount can also be obtained by performing linear image transformation processing from the original image data. It is not repeated here, but it should be understood that the dynamic range of the image data of the base layer is smaller than the dynamic range of the image data to be processed.

Next, in step S3, the base layer image data is encoded based on the first preset encoding rule to obtain a base layer image code stream;

The first preset encoding rule is an encoding rule determined according to the JPEG encoding rule. In addition, it can also be an encoding rule determined according to the encoding rule of the current mainstream image format, such as TIFF (Tag Image File Format, Tag Image File Format), BMP (Bitmap, Bitmap), etc. The following description may be given by taking JPEG as an example: For example, an encoder in a mobile phone may acquire the above-mentioned base layer image data, and encode it according to JPEG encoding rules to obtain a base layer image code stream.

Next, in step S4, a residual is calculated based on the to-be-processed image data and the base layer image data, and enhancement layer image data is determined based on the residual;

In this application, residual refers to image data obtained by removing base layer image data from image data to be processed. The residual can be directly used as enhancement layer image data. The enhancement layer image data refers to the image data between the high bit depth image data and the base layer image data. Its dynamic range is also high dynamic range, which is higher than the dynamic range of the base layer image data.

In an example, calculating the residual based on the image data to be processed and the base layer image data includes: acquiring preprocessed base layer image data, wherein the preprocessed base layer image data includes base layer image data obtained after decoding the base layer image code stream; calculating residuals based on the image data to be processed and the preprocessed base layer image data, for example, subtracting the image data from the image data to be processed The residual is obtained by preprocessing the base layer image data.

For another example, the calculating the residual based on the to-be-processed image data and the preprocessed base layer image data includes:

First, align the dynamic range represented by the image data to be processed and the dynamic range represented by the preprocessed base layer image data, for example, the alignment may be: hdr/16, where hdr is the image to be processed data, that is, the above-mentioned nonlinear HDR image data. When the hdr is 12bit and the base layer image data is 8bit image data, hdr/16 can obtain 8bit integer and 4bit decimal, so as to align with the dynamic range of the base layer image data.

Then, according to the ratio of the dynamic range of the image data to be processed to the dynamic range of the base layer image data, the aligned image data to be processed is compensated; the preprocessing is subtracted from the compensated image data to be processed The base layer image data is obtained, and the residual is obtained. The compensation method may be, for example, compensating hdr/16 data, and converting hdr/16*drccoef, where *drccoef is the ratio of the dynamic range of the base layer image data to the dynamic range of the HDR image (ie, hdr) data countdown. After the compensation, the preprocessed base layer image data can be subtracted to obtain the enhancement layer image data.

In order to optimize the calculation method, reduce the calculation amount, and improve the calculation efficiency and time, the data of the enhancement layer image can be selected. Exemplarily, the determining the enhancement layer image data based on the residual includes: obtaining data to be retained based on the residual; selecting retained data from the data to be retained, and using the data as the enhancement layer image data.

The way to choose to keep the data can be, for example, (hdr/16*drc _coef -base)*2 ⁿ , where n is the number of bits of data you want to keep. According to the previous text, hdr/16 can get 4 decimal places, so the maximum value of n is 4, and the minimum value is 1. Here you can choose n to be 3, then you can keep three decimal places for a long time. (hdr/16*drc _coef -base) is the residual. Random noise data can be added to this residual. Since in the encoding process of the subsequent images, the calculation process is performed on positive integers, it is also necessary to convert the decimals into positive integers.

For example, the reserved data is adjusted to obtain enhancement layer image data with positive integer data.

It can be adjusted by the following formula (1):

Among them, res is the image data of the enhancement layer. It can be understood that it is adjusted to a positive integer through 2048. Base can be the preprocessed base layer image data.

In addition, it is also possible to directly adjust the residual to obtain a residual with positive integer data, which is directly used as the enhancement layer image data.

Next, in step S5, the enhancement layer image data is encoded based on the second preset encoding rule to obtain an enhancement layer image code stream. Wherein, the second preset encoding rule refers to the encoding rule for the enhancement layer image data, and is encoded accordingly. The encoding process may include: dividing into slices (slice image data), DCT (Discrete Cosine Transform, discrete cosine transform) transformation, quantization, calculation of Qp map (quantization parameter map, quantization parameter table), coefficient scanning, entropy encoding, etc. The encoding process may be any suitable encoding manner well known to those skilled in the art, which will not be described further here.

Finally, in step S6, the image code stream data is generated based on the enhancement layer image code stream and the base layer image code stream, for example, the enhancement layer image code stream is written into a preset in the base layer image code stream field to generate the video stream data. Among them, the preset field can select the corresponding preset field according to different encoding rules. For example, for the encoding rule corresponding to JPEG, it may be the position immediately following the APP0 application segment. For example, according to the above, the encoder in the mobile phone finds the position of the APP0 application segment of the base layer image code stream (code stream marker character FF E0), then reads the length of the application segment, and moves the pointer back by this length, It is the insertion position of the enhancement layer image code stream, and the enhancement layer image code stream is inserted into the base layer image code stream, and the final image code stream data is output.

If the APP0 application segment does not exist in the base layer image code stream, the enhancement layer image code stream is inserted at the last or other custom position of the base layer image code stream.

The above-mentioned enhancement layer image code stream retained in the image code stream data can obtain enhancement layer image data by decoding the enhancement layer image code stream, and the enhancement layer image data can be used to restore the saturated area in part of the image. The image data of the enhancement layer with the dynamic range of the basic layer image data (such as JPEG) can be used to restore the saturated areas in part of the image when adjusting the brightness, contrast and other parameters of the image in image post-processing, such as reducing the image brightness to make part of the image white The sky is restored to its original blue color.

Continuing to refer to FIG. 1, in step S140, the first image and the processed image are displayed.

By displaying the first image and the processed image, the user can be shown the effect that the processed image can achieve. The first image and the processed image may be displayed in a preset display manner, including: for the abnormally exposed area in the first image, alternately displaying the image corresponding to the abnormally exposed area in the first image and all the images corresponding to the abnormally exposed area in the first image. The processing image is described, and through the alternate display, the user is prompted by the jumping of the image that the area covered by the processed image in the first image is an abnormal exposure area. The alternate display also means that by controlling the display timing of the processed image, it is displayed at a preset frequency. When it is displayed, it will cover the abnormally exposed area in the first image, and when it is not displayed, it will be displayed at a preset frequency. Displays abnormally exposed areas in the first image. Alternatively, other preset display manners can also be used to realize that while displaying the processed image, the user can also be prompted that there is an abnormally exposed area in the first image.

In one example, the method further includes: based on the abnormal exposure area, generating prompt information for prompting the abnormal exposure area; displaying the prompt information, the prompt information identifying the Expose abnormal areas. For example, the prompt information may include a preset pattern, such as a pattern similar to a zebra crossing. As shown in FIG. 3 , the abnormally exposed area in the first image is identified by the preset pattern; for another example, the prompt information Including a specific color, the abnormally exposed area in the first image is identified with a specific color, and the specific color can be any color that can highlight the abnormally exposed area, such as red, black, green and so on. Alternatively, abnormally exposed areas can be highlighted in a flashing manner.

It is worth mentioning that when the first image includes an overexposed area and an underexposed area, the same prompt information can be used to prompt the user, or different prompt information can be used to differentiate the prompting to the user, such as Overexposed and underexposed areas are identified in different preset patterns, or overexposed and underexposed areas are identified in different specific colors.

In the embodiment of the present application, as shown in FIG. 4 , the first image 201 , the processed image 203 and the prompt information 202 , such as a preset pattern (such as a zebra crossing), may also be displayed simultaneously, wherein the prompt information identifies the The exposed abnormal area in the first image, and the processed image can cover the prompt information such as a pattern similar to zebra crossing.

In other examples, the processed image can also be displayed according to the user's instruction. For example, when the user judges that there is an overexposed area and/or an underexposed area in the first image by observing the first image, or obtains the first image through the displayed prompt information When there are overexposed areas and/or underexposed areas in an image, the processed image can be manually controlled to display.

It is worth mentioning that, in this embodiment of the present application, there may be one or more abnormal exposure areas in the first image.

When there is no abnormal exposure area in the displayed first image, or, there is an abnormal exposure area in the displayed first image, but the processing effect of the processed image meets the user's needs, the current shooting parameters can be used for shooting, and when the displayed first image has an abnormal exposure area If there is an abnormal exposure area in an image, and the processing effect of the processed image does not meet the user's needs, the current shooting parameters can be corrected, for example, if there is an overexposed area, appropriately increase the shutter speed, reduce the aperture and/or Reduce the ISO sensitivity; if there is an underexposed area, appropriately reduce the shutter speed, increase the aperture and/or increase the ISO sensitivity, etc., so as to assist the user in exposure control, achieve accurate exposure, and improve the user experience.

So far, the description of the image processing method in the embodiments of the present application has been completed. It can be understood that the steps in the above-mentioned method of the present application can be exchanged or performed alternately in order on the premise that there is no contradiction.

To sum up, according to the image processing method of the present application, at least the image of the abnormally exposed area is processed, and the first image and the processed image are displayed in a preset display mode, so that the user can intuitively observe The exposure state under the current shooting conditions, and whether the processed effect of the image captured under the current shooting conditions meets the user's requirements, so as to assist the user in exposure control, achieve accurate exposure, and improve the user's experience.

Next, the image processing apparatus of the present application will be described with reference to FIG. 5 , wherein FIG. 5 shows a schematic block diagram of the image processing apparatus in the embodiment of the present application. The image processing apparatus can be used to process the aforementioned image processing method.

The image processing apparatus may be a handheld or wearable terminal device, for example, a smart phone, a tablet computer, a desktop computer, a computer, a camera, a video camera, a wearable device (such as a smart watch), etc., or any other An image processing device with a photographing device such as a camera, such as a movable platform, the movable platform includes but is not limited to aircraft (including but not limited to drones), robots, vehicles or ships.

The photographing device may be a device with an image sensor, such as a camera, a video camera, a camera, or other devices. An image sensor (also referred to herein as a camera sensor) is a device for capturing image data, which may be, for example, a CMOS image sensor, a CCD image sensor, or the like.

In one example, as shown in FIG. 5, the image processing apparatus 500 includes one or more processors 501, a memory 502, a communication interface 503, a display unit 504, and the like. These components are interconnected by a bus system and/or other form of connection mechanism (not shown).

The memory 502 is used for storing executable instructions, for example, for storing corresponding steps and program instructions for implementing the image processing method 100 according to the embodiment of the present application. The memory 502 can store an operating system (including programs for processing various basic system services and for performing hardware-related tasks), an application program required for at least one function (such as a control function of a photographing device, a sound playback function, an image playback function) and many more. Some of these applications can be downloaded from external servers via wireless communication. Other applications may be installed in memory 502 at the time of manufacture or at the factory, and may also include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and /or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like. The non-volatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory (eg, an SD card), and the like.

The communication interface 503 may be used for communication between the image processing apparatus 500 and other external devices, including wired or wireless communication. The communication interface 503 may be for accessing a wireless network based on a communication standard, such as WiFi (eg WiFi6), software-defined radio (SDR for short), 2G, 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication interface 503 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication interface 503 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

Optionally, the image processing apparatus 500 further includes an input device (not shown), which may be a device used by the user to input various operations, for example, may include function keys (such as touch keys, keys (such as volume control keys, switch keys, etc.). etc.), mechanical keys, soft keys, etc., one or more of joysticks, microphones, touchscreens, etc., or other user input devices for enabling a user to enter information. Data (eg, audio, video, images, etc.) is passed through input device to obtain.

The processor 501 may be a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other form of processing with data processing capabilities and/or instruction execution capabilities unit, and can control other components in the camera to perform desired functions. The processor 501 can execute the instructions stored in the memory 502 to execute the corresponding steps of the image processing method 100 described herein. For details, refer to the foregoing method description. For example, processor 501 can include one or more embedded processors, processor cores, microprocessors, logic circuits, hardware finite state machines (FSMs), digital signal processors (DSPs), ISPs, encoders, decoders or their combination.

In one example, the processor 501 is configured to execute program instructions stored in the memory, so that one or more processors perform the following steps: acquiring a first image of a first dynamic range of a shooting scene; performing analysis to determine abnormally exposed areas in the first image, the abnormally exposed areas including overexposed areas and/or underexposed areas; at least processing the images of the abnormally exposed areas to obtain a processed image, wherein the The processed image has a second dynamic range that is higher than the first dynamic range.

In an example, the processor 501 is further configured to: generate prompt information for prompting the abnormal exposure area based on the abnormal exposure area.

In an example, the processor 501 is configured to analyze the first image and determine the abnormally exposed area in the first image, including: determining, based on pixel information of pixel points of the first image, the Describe the abnormally exposed area in the first image. For example, comparing the brightness information of each area in the first image with a preset brightness threshold to determine an abnormally exposed area in the first image, where the preset brightness threshold includes a first preset brightness threshold or a second preset brightness threshold, the first preset brightness threshold is greater than the second preset brightness threshold, the overexposed area includes an area whose brightness information is greater than the first preset brightness threshold, and the underexposed area Including an area whose brightness information is less than the second preset brightness threshold.

In an example, the processor 501 is configured to acquire a first image of a first dynamic range of a shooting scene, including: acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream; decoding the base layer image code stream to obtain the first image; or, acquiring original image data, and acquiring a first image of a first dynamic range of a shooting scene based on the original image data.

In an example, the processor 501 is configured to process at least the image of the abnormally exposed area to obtain the processed image, including: acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and the enhancement layer code stream; decode the base layer image code stream and the enhancement layer code stream to obtain the base layer image data and the enhancement layer image data; fuse the base layer image data and the enhancement layer image data to obtain the A second image with a second dynamic range; extracting an image corresponding to the abnormally exposed area in the second image to obtain the processed image.

In one example, the processor 501 is configured to process at least the image of the abnormally exposed area to obtain a processed image, including: when the first image includes an overexposed area, processing the second image processing to reduce the brightness of the second image; or, when the first image includes an underexposed area, processing the second image to improve the brightness of the second image; the extracting The image corresponding to the abnormal exposure area in the second image to obtain the processed image includes: extracting the image corresponding to the abnormal exposure area in the processed second image to obtain the processed image image.

In one example, the processor 501 is configured to process at least the image of the abnormally exposed area to obtain a processed image, including: when the first image includes an overexposed area, based on a preset mapping function , process the second image to reduce the brightness of the second image; or when the first image includes an underexposed area, process the second image based on a preset mapping function to improve the brightness of the second image brightness.

In an example, the processor 501 is configured to process at least the image of the abnormally exposed area to obtain the processed image, including: acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and enhancement layer code stream; decode the base layer image code stream to obtain base layer image data; decode the enhancement layer code stream to obtain enhancement layer image data; The image data of the base layer and the image data of the enhancement layer corresponding to the abnormally exposed area are fused to obtain a processed image.

In an example, the processor 501 is configured to acquire image code stream data, including: acquiring original image data, performing nonlinear image transformation processing on the original image data to obtain image data to be processed; or extracting a preset amount of base layer image data from the original image data; encoding the base layer image data based on the first preset coding rule to obtain a base layer image code stream; based on the to-be-processed image data and Calculate residuals from the base layer image data, and determine enhancement layer image data based on the residuals; encode the enhancement layer image data based on a second preset coding rule to obtain an enhancement layer image code stream; The video code stream data is generated from the layer image code stream and the base layer image code stream.

In an example, the processor 501 is configured to calculate a residual based on the image data to be processed and the base layer image data, including: acquiring preprocessed base layer image data, wherein the preprocessed base layer The image data includes base layer image data obtained after decoding the base layer image code stream; a residual is calculated based on the to-be-processed image data and the preprocessed base layer image data.

In an example, the processor 501 is configured to calculate the residual based on the to-be-processed image data and the preprocessed base layer image data, including: subtracting the pre-processed image data from the to-be-processed image data Process the base layer image data to obtain the residual.

In one example, the processor 501 is configured to calculate the residual based on the to-be-processed image data and the preprocessed base layer image data, including: comparing the dynamic range represented by the to-be-processed image data with the The dynamic range represented by the preprocessed base layer image data is aligned; the aligned image data to be processed is compensated according to the ratio of the dynamic range of the image data to be processed to the dynamic range of the base layer image data; The preprocessed base layer image data is subtracted from the compensated image data to be processed to obtain a residual.

In an example, the processor 501 is configured to determine the enhancement layer image data based on the residual, including: obtaining data to be retained based on the residual; selecting retained data from the data to be retained, and using the data as the retained data the enhancement layer image data. In an example, the processor 501 is configured to select the reserved data from the to-be-retained data and use it as the enhancement layer image data, including: adjusting the reserved data to obtain a data with positive integer data. Enhancement layer image data.

Optionally, the image processing apparatus 500 further includes a display unit 504, a device for displaying images or videos generated by the processor component. The display unit 504 may include, for example, a liquid crystal display (LCD) device or an organic light emitting diode (OLED) device. Based on data acquired from the processor, the display may display various images, such as menus, selected operating parameters, a capture mode selection interface, images captured by an image sensor, received by the processor, etc. and/or received from a user interface of the device other information, etc. When the image processing device is a movable platform, the movable platform is connected to a control terminal such as a mobile phone, a remote controller, glasses, a tablet computer, etc., and the display unit may be a display screen of these control terminals.

Exemplarily, the display unit 504 is configured to: display the first image and the processed image, for example, for the abnormal exposure area in the first image, alternately display the corresponding abnormal exposure area in the first image. an image and the processed image. The prompt information includes a preset pattern, and the abnormal exposure area in the first image is identified by the preset pattern; or the prompt information includes a specific color, and the abnormal exposure area in the first image is marked with the preset pattern. identified by a specific color.

The display unit is further configured to display the prompt information, and the prompt information identifies an abnormally exposed area in the first image, for example, the first image, the processed image, and the prompt information are displayed at the same time.

Since the image processing apparatus in the embodiment of the present application can implement the aforementioned image processing method, it also has the advantages of the aforementioned method.

The image processing apparatus 500 in this application may be any device or apparatus that can process images, such as a camera, a mobile phone, a PTZ camera, a drone, a camera on a drone, a computer, or the like.

In addition, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions stored in the storage device to implement (implemented by the processor) the embodiments of the present invention described herein. function and/or other desired functions, for example, to execute the corresponding steps of the image processing method 100 according to the embodiment of the present application, various application programs and various data may also be stored in the computer-readable storage medium, such as the Various data used and/or generated by the application, etc.

For example, the computer storage medium may include, for example, a memory card for a smartphone, a storage unit for a tablet computer, a hard disk for a personal computer, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk Read only memory (CD-ROM), USB memory, or any combination of the above storage media. The computer-readable storage medium can be any combination of one or more computer-readable storage media.

Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above-described example embodiments are exemplary only, and are not intended to limit the scope of the invention thereto. Various changes and modifications can be made therein by those of ordinary skill in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or May be integrated into another device, or some features may be omitted, or not implemented.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the description of the exemplary embodiments of the invention, various features of the invention are sometimes grouped together , or in its description. However, this method of the invention should not be interpreted as reflecting the intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the corresponding claims reflect, the invention lies in the fact that the corresponding technical problem may be solved with less than all features of a single disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

It will be understood by those skilled in the art that all features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or apparatus so disclosed may be used in any combination, except that the features are mutually exclusive. Processes or units are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the invention within and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules according to the embodiments of the present invention. The present invention may also be implemented as apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

It should be noted that the above-described embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

Claims

An image processing method, characterized in that the method comprises:

acquiring a first image of a first dynamic range of the shooting scene;

Analyzing the first image to determine an abnormally exposed area in the first image, where the abnormally exposed area includes an overexposed area and/or an underexposed area;

processing at least the image of the abnormally exposed area to obtain a processed image, wherein the processed image has a second dynamic range, and the second dynamic range is higher than the first dynamic range;

The first image and the processed image are displayed.
The method according to claim 1, wherein the method further comprises: based on the abnormal exposure area, generating prompt information for prompting the abnormal exposure area;

The prompt information is displayed, where the prompt information is used to identify an abnormally exposed area in the first image.
The method of claim 1, wherein the analyzing the first image to determine an abnormally exposed area in the first image comprises:

Based on the pixel information of the pixel points of the first image, an abnormally exposed area in the first image is determined.
The method of claim 3, wherein the pixel information includes luminance information, and the determining an abnormally exposed area in the first image based on the pixel information of the pixel points of the first image comprises:

The brightness information of each area in the first image is compared with a preset brightness threshold to determine an overexposed area and/or an underexposed area in the first image, wherein the preset brightness threshold includes a first preset brightness threshold. Set a brightness threshold or a second preset brightness threshold, the first preset brightness threshold is greater than the second preset brightness threshold, and the overexposed area includes an area with brightness information greater than the first preset brightness threshold, so The underexposed area includes an area whose brightness information is less than the second preset brightness threshold.
The method of claim 1, wherein the acquiring the first image of the first dynamic range of the shooting scene comprises:

acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream;

decoding the base layer image code stream to obtain the first image; or,

Obtain original image data, and obtain a first image of a first dynamic range of a shooting scene based on the original image data.
The method of claim 1, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream;

Decoding the base layer image code stream and the enhancement layer code stream to obtain base layer image data and enhancement layer image data;

fusing the base layer image data and the enhancement layer image data to obtain a second image having the second dynamic range;

Extracting the image corresponding to the abnormally exposed area in the second image to obtain the processed image.
The method according to claim 6, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

When the first image includes an overexposed area, processing the second image to reduce the brightness of the second image; or,

When the first image includes an underexposed area, processing the second image to improve the brightness of the second image;

The extracting the image corresponding to the abnormally exposed area in the second image to obtain the processed image includes: extracting the image corresponding to the overexposed area and/or the underexposed area in the processed second image corresponding images to obtain the processed images.
The method of claim 7, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

When the first image includes an overexposed area, processing the second image based on a preset mapping function to reduce the brightness of the second image; or

When the first image includes an underexposed area, the second image is processed based on a preset mapping function to improve the brightness of the second image.
The method of claim 1, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream;

Decoding the base layer image code stream to obtain base layer image data;

Decoding the enhancement layer code stream to obtain enhancement layer image data;

The base layer image data corresponding to the abnormal exposure area and the enhancement layer image data corresponding to the abnormal exposure area are fused to obtain the processed image.
The method of claim 1, wherein the displaying the first image and the processed image comprises:

For the abnormally exposed area in the first image, the image corresponding to the abnormally exposed area in the first image and the processed image are alternately displayed.
The method of claim 2, wherein the prompt information includes a preset pattern, and an abnormally exposed area in the first image is identified by the preset pattern; or

The prompt information includes a specific color, and the abnormally exposed area in the first image is marked with a specific color.
The method of claim 1, wherein the processed image is a high dynamic range image, and the first image is a standard dynamic range image.
The method according to claim 5, 6 or 9, wherein the acquiring image code stream data comprises:

Obtaining original image data, and performing nonlinear image transformation processing on the original image data to obtain image data to be processed;

Extracting a preset amount of base layer image data from the to-be-processed image data or the original image data;

encoding the base layer image data based on the first preset encoding rule to obtain a base layer image code stream;

calculating a residual based on the to-be-processed image data and the base layer image data, and determining enhancement layer image data based on the residual;

encoding the enhancement layer image data based on the second preset encoding rule to obtain an enhancement layer image code stream;

The video code stream data is generated based on the enhancement layer image code stream and the base layer image code stream.
The method of claim 13, wherein the calculating a residual based on the to-be-processed image data and the base layer image data comprises:

acquiring preprocessed base layer image data, wherein the preprocessed base layer image data includes base layer image data obtained after decoding the base layer image code stream;

A residual is calculated based on the image data to be processed and the preprocessed base layer image data.
The method according to claim 14, wherein the calculating the residual based on the to-be-processed image data and the preprocessed base layer image data comprises:

The preprocessed base layer image data is subtracted from the to-be-processed image data to obtain the residual.
The method according to claim 14, wherein the calculating the residual based on the to-be-processed image data and the preprocessed base layer image data comprises:

Aligning the dynamic range represented by the image data to be processed and the dynamic range represented by the preprocessed base layer image data;

Compensate the aligned to-be-processed image data according to the ratio of the dynamic range of the to-be-processed image data to the dynamic range of the base layer image data;

The preprocessed base layer image data is subtracted from the to-be-processed image data to be compensated to obtain a residual.
The method according to any one of claims 13 to 16, wherein the determining the enhancement layer image data based on the residual comprises:

Based on the residual, obtain the data to be retained;

Retain data is selected from the to-be-retained data and used as the enhancement layer image data.
The method according to claim 17, wherein the selecting the reserved data from the to-be-reserved data as the enhancement layer image data comprises:

The reserved data is adjusted to obtain enhancement layer image data with positive integer data.
An image processing device, characterized in that the image processing device comprises: a memory for storing executable program instructions;

one or more processors for executing the program instructions stored in the memory, causing the processors to perform the following steps:

acquiring a first image of a first dynamic range of the shooting scene;

Analyzing the first image to determine an abnormally exposed area in the first image, where the abnormally exposed area includes an overexposed area and/or an underexposed area;

processing at least the image of the abnormally exposed area to obtain a processed image, wherein the processed image has a second dynamic range, and the second dynamic range is higher than the first dynamic range;

The display unit is used for displaying the first image and the processed image.
The image processing apparatus according to claim 19, wherein the processor is further configured to: generate prompt information for prompting the abnormal exposure area based on the abnormal exposure area;

The display unit is further configured to display the prompt information, where the prompt information identifies an abnormally exposed area in the first image.
The image processing apparatus according to claim 19, wherein the analyzing the first image to determine the abnormal exposure area in the first image comprises:

Based on the pixel information of the pixel points of the first image, an abnormally exposed area in the first image is determined.
21. The image processing device according to claim 21, wherein the pixel information includes luminance information, and the abnormal exposure area in the first image is determined based on the pixel information of the pixel points of the first image, include:

comparing the brightness information of each area in the first image with a preset brightness threshold to determine an abnormally exposed area in the first image, wherein the preset brightness threshold includes a first preset brightness threshold or a second preset brightness threshold A preset brightness threshold, the first preset brightness threshold is greater than the second preset brightness threshold, the overexposed area includes an area whose brightness information is greater than the first preset brightness threshold, and the underexposed area includes brightness The information is less than the area of the second preset brightness threshold.
The image processing apparatus according to claim 19, wherein the acquiring the first image of the first dynamic range of the shooting scene comprises:

acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream;

decoding the base layer image code stream to obtain the first image; or,

Obtain original image data, and obtain a first image of a first dynamic range of a shooting scene based on the original image data.
The image processing apparatus according to claim 19, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream;

Decoding the base layer image code stream and the enhancement layer code stream to obtain base layer image data and enhancement layer image data;

fusing the base layer image data and the enhancement layer image data to obtain a second image with the second dynamic range;

Extracting the image corresponding to the abnormally exposed area in the second image to obtain the processed image.
The image processing apparatus according to claim 24, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

When the first image includes an overexposed area, processing the second image to reduce the brightness of the second image; or,

When the first image includes an underexposed area, processing the second image to improve the brightness of the second image;

The extracting the image corresponding to the abnormal exposure area in the second image to obtain the processed image includes: extracting the image corresponding to the abnormal exposure area in the processed second image to obtain the processed image The processed image.
The image processing apparatus according to claim 25, wherein the processing at least the image of the abnormally exposed area to obtain the processed image comprises:

When the first image includes an overexposed area, processing the second image based on a preset mapping function to reduce the brightness of the second image; or

When the first image includes an underexposed area, the second image is processed based on a preset mapping function to improve the brightness of the second image.
The image processing apparatus according to claim 19, wherein:

The processing at least the image of the abnormally exposed area to obtain the processed image includes:

acquiring image code stream data, wherein the image code stream data includes a base layer image code stream and an enhancement layer code stream;

Decoding the base layer image code stream to obtain base layer image data;

Decoding the enhancement layer code stream to obtain enhancement layer image data;

The base layer image data corresponding to the abnormal exposure area and the enhancement layer image data corresponding to the abnormal exposure area are fused to obtain a processed image.
19. The image processing apparatus according to claim 19, wherein the display unit is further configured to: for the abnormally exposed areas in the first image, alternately display the images corresponding to the abnormally exposed areas in the first image. images and the processed images and.
The image processing apparatus according to claim 20, wherein the prompt information includes a preset pattern, and an abnormally exposed area in the first image is identified by the preset pattern; or

The prompt information includes a specific color, and the abnormally exposed area in the first image is marked with a specific color.
The image processing apparatus according to claim 19, wherein the processed image is a high dynamic range image, and the first image is a standard dynamic range image.
The image processing apparatus according to claim 23 or 24 or 27, wherein the acquiring the video code stream data comprises:

Obtaining original image data, and performing nonlinear image transformation processing on the original image data to obtain image data to be processed;

Extracting a preset amount of base layer image data from the to-be-processed image data or the original image data;

encoding the base layer image data based on the first preset encoding rule to obtain a base layer image code stream;

calculating a residual based on the to-be-processed image data and the base layer image data, and determining enhancement layer image data based on the residual;

encoding the enhancement layer image data based on the second preset encoding rule to obtain an enhancement layer image code stream;

The video code stream data is generated based on the enhancement layer image code stream and the base layer image code stream.
The image processing apparatus according to claim 31, wherein the calculating a residual based on the to-be-processed image data and the base layer image data comprises:

acquiring preprocessed base layer image data, wherein the preprocessed base layer image data includes base layer image data obtained after decoding the base layer image code stream;

A residual is calculated based on the image data to be processed and the preprocessed base layer image data.
The image processing apparatus according to claim 32, wherein the calculating the residual based on the to-be-processed image data and the preprocessed base layer image data comprises:

The preprocessed base layer image data is subtracted from the to-be-processed image data to obtain the residual.
The image processing apparatus according to claim 33, wherein the calculating the residual based on the to-be-processed image data and the preprocessed base layer image data comprises:

Aligning the dynamic range represented by the image data to be processed and the dynamic range represented by the preprocessed base layer image data;

Compensate the aligned to-be-processed image data according to the ratio of the dynamic range of the to-be-processed image data to the dynamic range of the base layer image data;

The preprocessed base layer image data is subtracted from the to-be-processed image data to be compensated to obtain a residual.
The image processing apparatus according to any one of claims 31 to 34, wherein the determining the enhancement layer image data based on the residual comprises:

Based on the residual, obtain the data to be retained;

Retain data is selected from the to-be-retained data and used as the enhancement layer image data.
The image processing apparatus according to claim 35, wherein the selection of reserved data from the to-be-reserved data as the enhancement layer image data comprises:

The reserved data is adjusted to obtain enhancement layer image data with positive integer data.
A computer storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the image processing method described in any one of 1 to 18 is implemented.