WO2023125467A1

WO2023125467A1 - Image processing method and apparatus, electronic device and readable storage medium

Info

Publication number: WO2023125467A1
Application number: PCT/CN2022/142142
Authority: WO
Inventors: 何思羽
Original assignee: 北京字跳网络技术有限公司
Priority date: 2021-12-30
Filing date: 2022-12-27
Publication date: 2023-07-06
Also published as: US20240357200A1; EP4439467A1; CN116416344A

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device and a readable storage medium. The method comprises: traversing a plurality of multimedia materials, and according to a target color display standard, determining a multimedia material of a non-target color display standard included in the plurality of multimedia materials; converting the multimedia material of the non-target color display standard into a multimedia material of the target color display standard; and performing image rendering on the plurality of multimedia materials according to the target color display standard, so as to display the plurality of multimedia materials according to the image rendering result or export the plurality of multimedia materials as a video file in a specific format according to the image rendering result.

Description

Image processing method, device, electronic device and readable storage medium

Cross References to Related Applications

This application is based on the Chinese application with the application number 202111658426.2 and the filing date is December 30, 2021, and claims its priority. The disclosure content of the Chinese application is hereby incorporated into this application as a whole.

technical field

The present disclosure relates to the technical field of image processing, and in particular to an image processing method, device, electronic equipment, and readable storage medium.

Background technique

When performing video editing, multiple multimedia materials are often edited to generate a video that meets expected display effects.

Contents of the invention

The disclosure provides an image processing method, device, electronic equipment and readable storage medium.

In a first aspect, the present disclosure provides an image processing method, including:

According to the target color display standard, determine the non-target color display standard multimedia materials included in the plurality of multimedia materials;

converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard; and

Perform image rendering on the multiple multimedia materials according to the target color display standard, so as to display the multiple multimedia materials on the display screen of the electronic device according to the image rendering result, or, according to the image rendering result, render the The multiple multimedia materials are exported as video files in a specific format.

In some embodiments, before determining the non-target color display standard multimedia materials included in the plurality of multimedia materials according to the target color display standard, the method further includes:

The target color display standard is determined.

In some embodiments, said determining the target color display standard includes:

The target color display standard is determined according to the color display standards respectively corresponding to the plurality of multimedia materials.

In some embodiments, the determining the target color display standard according to the color display standards respectively corresponding to the plurality of multimedia materials includes:

The target color display standard is determined according to the color display standard with the widest color space among the color display standards respectively corresponding to the plurality of multimedia materials.

In some embodiments, the determining the target color display standard includes: determining the target color display standard according to a pre-specified color display standard.

In some embodiments, the converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard includes:

If the multimedia material of the non-target color display standard is generated according to the video frame image of another multimedia material of the non-target color display standard in the plurality of multimedia materials at the freeze frame position, then the non-target color display converting the standard multimedia material to display the standard multimedia material in said target color; or

If the multimedia material of the non-target color display standard is generated according to the video frame image of the multimedia material of the target color display standard in the plurality of multimedia materials at the fixed position, then the non-target color display standard is not displayed. multimedia material for conversion.

In some embodiments, the target color display standard is a perceptual quantization PQ color display standard, and the non-target color display standard includes a standard dynamic range SDR color display standard and/or a mixed logarithmic gamma curve HLG color display standard; or,

The target color display standard is the SDR color display standard, and the non-target color display standard includes the HLG color display standard and/or the PQ color display standard; or,

The target color display standard is the HLG color display standard, and the non-target color display standard includes the SDR color display standard and/or the PQ color display standard.

In some embodiments, before performing image rendering on the plurality of multimedia materials according to the target color display standard, the method further includes:

The plurality of multimedia materials are processed according to clipping operation modes respectively corresponding to one or more clip types.

In a second aspect, the present disclosure provides an image processing device, including:

An identification module, configured to determine non-target color display standard multimedia materials included in the plurality of multimedia materials according to the target color display standard;

A conversion module, configured to convert the multimedia material of the non-target color display standard into the multimedia material of the target color display standard;

a rendering module, configured to perform image rendering on the plurality of multimedia materials; and

A control module, configured to control displaying the multiple multimedia materials on the display screen of the electronic device according to the image rendering result, or control exporting the multiple multimedia materials into a video file in a specific format according to the image rendering result .

In a third aspect, the present disclosure provides an electronic device, including: a memory and a processor;

the memory is configured to store computer program instructions;

The processor is configured to execute the computer program instructions, so that the electronic device implements the image processing method as described in the first aspect.

In a fourth aspect, the present disclosure provides a readable storage medium, including: computer program instructions;

At least one processor of the electronic device executes the computer program instructions to implement the image processing method as described in the first aspect.

In a fifth aspect, the present disclosure provides a computer program product, which, when executed by a computer, causes the computer to implement the image processing method as described in the first aspect.

In a sixth aspect, the present disclosure provides a computer program, including instructions, which, when executed by a processor, cause the processor to implement the image processing method as described in the first aspect.

The present disclosure provides an image processing method, device, electronic equipment, and readable storage medium, wherein the method includes: traversing multiple multimedia materials, and determining non-target color display standards included in the multiple multimedia materials according to the target color display standard Multimedia materials; convert non-target color display standard multimedia materials into target color display standard multimedia materials; and perform image rendering on multiple multimedia materials according to the target color display standard, and display or export to a specific format according to the image rendering results video file.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following will briefly introduce the drawings that need to be used in the descriptions of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, Other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure;

2A-2B are schematic flowcharts of an image processing method provided by another embodiment of the present disclosure;

FIG. 3 is a schematic flowchart of an image processing method provided by another embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure;

Fig. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, in the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways than described here; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, and Not all examples.

If the color display standards of multiple multimedia materials are the same, the edited video can meet the expected display effect. However, since different color display standards have different parameters such as color space, brightness range, etc., if the color display standards of multiple multimedia materials are not completely the same, the synthesized video obtained after editing and the preview during the editing process will be different if the color display standards are different. If the original color display standard of each multimedia material is displayed, the expected display effect cannot be achieved. For example, the first multimedia material is a standard dynamic range (Standard Dynamic Range, SDR) color display standard multimedia material, and the second multimedia material is a high dynamic range (High Dynamic Range, HDR) color display standard multimedia material. The standard color space of color display is bt.709, and the brightness range is 0-100nit, while the standard color space of HDR color display is bt.2020, and the brightness range is 0-10000nit. After editing, if it is displayed according to the SDR color display standard If the first multimedia material is displayed according to the HDR color display standard and the second multimedia material is displayed, the visual difference between the two multimedia materials before and after is relatively large, and the expected display effect cannot be achieved, and the user experience is greatly reduced.

Based on this, the present disclosure provides an image processing method, device, electronic equipment, readable storage medium, and computer program product, wherein the method converts non-target color display standard multimedia materials among a plurality of multimedia materials into target color display Standard multimedia materials make the color display standards of multiple multimedia materials consistent. In the composite video obtained based on this, each multimedia material is displayed according to the target color display standard, and the visual effect is consistent, so that the display effect of multimedia materials can be achieved. expected.

Exemplarily, the image processing method provided in the present disclosure may be executed by the image processing device provided in the present disclosure, and the image processing device may be implemented by any software and/or hardware. Exemplarily, the image processing device may include but not limited to: tablet computer, mobile phone (such as folding screen mobile phone, large-screen mobile phone, etc.), notebook computer, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook, personal digital Assistant (personal digital assistant, PDA), smart TV, smart screen, high-definition TV, 4K TV, smart projector and other Internet of things (the internet of things, IOT) equipment, server, server cluster, cloud server, etc. The specific type of electronic device is not limited in any way.

The embodiment of the present disclosure will take an electronic device as an example, and a video editing application (referred to as application 1) is installed in the electronic device, and multiple multimedia materials can be edited through the application 1 as an example. Combined with the drawings and application scenarios, this disclosure provides The image processing method is described in detail.

The image processing method provided by the present disclosure can be applied to the following scenarios.

Scenario 1: Start application 1, when importing multimedia materials for the first time.

Scenario 2: In the process of editing multiple multimedia materials, when adding a new multimedia material.

Scenario 3: Using the freeze function provided by application 1, when inserting a specified multimedia material at the freeze position. Among them, the "freezing function" refers to saving the video frame image at the specified position as an image, and then inserting a corresponding number of the image at the specified position according to the playback rate.

Of course, the method provided in the present disclosure is not limited to be applied to the above-mentioned scenarios, and the above-mentioned scenarios 1 to 3 are only examples.

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure. Referring to Figure 1, the method provided in this embodiment includes:

S101. Determine, according to the target color display standard, multimedia materials that are not included in the target color display standard included in the plurality of multimedia materials.

The present disclosure does not limit the types of multimedia materials. For example, the multimedia material may be any type of material such as a photo, a picture, or a video. In addition, the present disclosure does not limit the number of multimedia materials.

In addition, the color display standards corresponding to the multiple multimedia materials may not be completely the same, or may also be the same, which is not limited in the present disclosure. For example, there are 5 multimedia materials, among which 3 are multimedia materials with PQ color display standard, 1 of them is HLG color display standard multimedia material, and the other is SDR color display standard multimedia material.

The present disclosure makes no limitation on the target color display standard and the non-target color display standard. Exemplarily, the target color display standard is the PQ color display standard, and the non-target color display standard includes the SDR color display standard and/or the HLG color display standard; or, the target color display standard is the SDR color display standard, and the non-target color display standard includes The HLG color display standard and/or the PQ color display standard; or, the target color display standard is the HLG color display standard, and the non-target color display standard includes the SDR color display standard and/or the PQ color display standard.

It should be noted that the target color display standard and the non-target color display standard are not limited to the above situations. For example, the target color display standard may also be a color display standard with a wider color space and a higher brightness range.

The present disclosure does not limit the implementation manner in which the electronic device acquires multiple multimedia materials. Exemplarily, the electronic device can start the application 1, and obtain a plurality of multimedia materials from the local storage space of the electronic device through related entries provided by the application 1. In addition, when the electronic device acquires the multimedia material, it simultaneously acquires the relevant attribute information of each multimedia material, and determines the target color display standard corresponding to the multimedia material based on the relevant attribute information of the multimedia material.

For example, the relevant attribute information of the multimedia material may include color space information, color description information (color transfer information), etc., and the electronic device may determine the color display standard corresponding to the multimedia material based on the color description information corresponding to the multimedia material.

Exemplarily, it is assumed that the format of the color description information corresponding to the multimedia material is as follows:

"color_transfer": "smpte2840"; or,

"color_transfer": "arib-std-b67".

Then according to "smpte2840" or "arib-std-b67", it can be determined that the color display standard corresponding to the multimedia material is HDR.

Based on this, application 1 can determine the color display standard corresponding to each multimedia material in the multiple multimedia materials by traversing the relevant information of multiple multimedia materials, and compare the color display standard corresponding to each multimedia material with the target color display standard , so as to determine the non-target color display standard multimedia materials included in the multiple multimedia materials. Among them, the non-target color display standard multimedia material can be one or more.

For example, there are 5 multimedia materials, 3 of which are PQ color display standard multimedia materials, 1 is HLG color display standard multimedia material, and the other is SDR color display standard multimedia material; suppose the target color display standard is PQ color display standard, the non-target color display standard multimedia material includes HLG color display standard multimedia material and SDR color display standard multimedia material.

In some cases, when the corresponding image is inserted at the specified position through the freeze function provided by application 1, if the color display standard of the original multimedia material of the acquired image is different from the color display standard of the saved image, the inserted image will Chromatic aberration occurs. For example, the original multimedia material corresponding to the specified location is an HDR color display standard, but the electronic device only supports saving the image at the specified location as an image in jpeg format, and the image in jpeg format is an image in SDR color display standard, which will result in an inserted There is a color difference between the image and the original multimedia material.

Therefore, when saving an image extracted from the original multimedia material through the freeze function, a description file can be generated for the image, and relevant attribute information of the image can be stored in the description file, for example, the color display corresponding to the image source (that is, the original multimedia material) standard information. When inserting the image, if it is determined based on the description file that the image comes from a multimedia material with a target color display standard, although the image is saved as a non-target color display standard, it is not necessary to convert the image; if the source of the image is determined based on the description file If the image is a non-target color display standard multimedia material, it is determined that the image is a non-target color display standard multimedia material that needs to be converted.

That is to say, if a multimedia material among the multiple multimedia materials is generated based on another multimedia material among the multiple multimedia materials, it is determined whether the multimedia material needs to be converted based on the color display standard of the source multimedia material.

S102. Convert the non-target color display standard multimedia material into the target color display standard multimedia material.

In some embodiments, implementing step S102 may include the following steps:

Step a: Decode the multimedia material of the non-target color display standard, and obtain the color value of each pixel of the multimedia material under the non-target color display standard, that is, obtain the first set of color values.

Step b. Based on the electro-optical transfer function (EOTF) corresponding to the non-target color display standard, each color value in the first color value set is converted into an optical signal under the non-target color display standard to obtain the first optical signal set.

Among them, the electro-optical conversion function is a function used to describe the relationship between the nonlinear color value input to the display (digitally encoded pixel value, electrical signal) and the linear color value (optical signal) displayed by the display.

Since the equipment stores and transmits electrical signals (i.e. color values), and the electro-optical conversion functions corresponding to different color display standards are different, therefore, the color value of each pixel of the multimedia material in the non-target color display standard is converted into the target color When displaying standard color values, it is first necessary to convert the color values of each pixel of the multimedia material in the non-target color display standard into light signals, so as to realize the conversion of color values in different color color spaces in linear space.

Step c. Based on an optical-to-optical transfer function (OOTF), each optical signal in the first optical signal set is converted to obtain a second optical signal set under the target color display standard.

There is a specific conversion function (also known as a conversion matrix) between the optical signals under different color display standards, that is, the optical-to-optical conversion function; through the optical-to-optical conversion function, the optical signal under the non-target color display standard can be converted into The target color shows the light signal under the standard.

Step d. Based on the optoelectronic transfer function (OETF) corresponding to the target color display standard, each optical signal in the second optical signal set is converted into a color value to obtain a second color value set of the multimedia material under the target color display standard.

By converting the optical signal under the target color display standard into the electrical signal (color value) under the target color display standard, the color value of each pixel of the multimedia material under the target color display standard is obtained.

During the above conversion process, the multimedia material may be converted in units of frames.

Exemplarily, the target color display standard is the PQ color display standard, and the non-target color display standard is the HLG color display standard as an example, with reference to FIG. 2A , for illustration.

Wherein, the multimedia material of the HLG color display standard is referred to as the HLG material for short, and the multimedia material of the PQ color display standard is referred to as the PQ material for short.

First, the HLG material will be decoded to obtain the RGB value of each pixel of the HLG material under the HLG color display standard; according to the EOTF of the HLG color display standard, each RGB value under the HLG color display standard will be converted to the HLG color display standard under the optical signal; based on the optical-to-optical conversion function between the HLG color display standard and the PQ color display standard, each optical signal under the HLG color display standard is converted into an optical signal under the PQ color display standard; and then according to the PQ color display standard The photoelectric conversion function converts each optical signal under the PQ color display standard into the RGB value under the PQ color display standard, so as to realize the conversion of HLG material into SDR material.

It should be noted that when the target color display standard and the non-target color display standard are other situations, the implementation methods are similar, the difference lies in the differences in the electro-optical conversion function, optical-optical conversion function, and photoelectric conversion function used, and no examples are given here. illustrate.

In some possible implementations, after converting the light signals of non-target color display standards into light signals of target color display standards through the above step c, gamut mapping can also be performed to align different color display standards. The color value in the standard. The present disclosure does not limit the specific implementation manner of the color gamut mapping process.

For example, when the target color display standard is the SDR color display standard and the non-target color display standard is the PQ color display standard, the same RGB value represents different colors in the PQ color display standard and in the SDR color display standard. Therefore, the PQ color can be After the optical signal of the display standard is converted into the optical signal of the SDR color display standard, the color gamut mapping process is performed on each optical signal, so as to align the color values in different color display standards.

Exemplarily, with reference to the embodiment shown in Figure 2B, when the multimedia material of the SDR color display standard is converted into the multimedia material of the PQ color display standard, each optical signal under the SDR color display standard is converted into a PQ by an optical-to-optical conversion function. After the optical signals under the color display standard, color gamut mapping processing can be performed on each optical signal, and then the optical signals obtained after the color gamut mapping processing under the PQ color display standard are converted into electrical signals under the PQ color display standard.

Similarly, the color gamut mapping process can also be performed when converting the multimedia material of the PQ color display standard to the SDR color display standard multimedia material, and converting the HLG color display standard multimedia material to the SDR color display standard multimedia material. Its implementation is similar to that shown in FIG. 2B , and for the sake of brevity, details are not repeated here.

In a practical application, the application 1 can implement step S102 by calling an application programming interface (Application Programming Interface, API) provided by the function module that realizes the conversion between the above-mentioned different color display standards.

For example, the functional module that realizes the conversion of HLG color display standard multimedia materials into PQ color display standard multimedia materials provides an API. The API name is "TECoreHLG2PQCore". Convert the multimedia material of HLG color display standard to the multimedia material of PQ color display standard.

The function module used to realize the conversion of multimedia material of SDR color display standard into multimedia material of PQ color display standard provides an API. The API name is "TECoreSDR2PQCore". Multimedia material with color display standard is converted into multimedia material with PQ color display standard.

When the target color display standard and non-target color display standard are in other situations, the conversion method through the API is similar to this example, and will not be repeated here for the sake of brevity.

S103. Perform image rendering on the multiple multimedia materials according to the target color display standard, so as to display the multiple multimedia materials on the display screen of the electronic device according to the image rendering result, or, according to the image rendering result, render the The multiple multimedia materials are exported as video files in a specific format.

The image rendering of the multimedia material can be realized by using the currently existing rendering technology, and the present disclosure does not limit the specific implementation manner of image rendering.

In the scene where the multimedia material is displayed on the display screen of the electronic device, the electronic device can convert the RGB color value of each pixel of the multimedia image material into an optical signal according to the electro-optic conversion function corresponding to the target color display standard and the image rendering result, so as to perform exhibit. In addition, based on different scenarios, part or all of the multiple multimedia materials can be displayed.

For example, when a user locates at a specific video frame position in the process of editing multiple multimedia materials through the application 1, the multimedia material corresponding to the specific position may be displayed according to the target color display standard.

For another example, when the user triggers the playing of multiple multimedia materials during the process of editing multiple multimedia materials through the application 1, the multiple multimedia materials are played sequentially according to the target color display standard.

When exporting the multiple multimedia materials after the color display standard conversion to a video file in a specific format, the image rendering result can be based on the RGB color value information corresponding to the multimedia material of the target color display standard to encode and save For video files of a specific format, for example, mp4 format.

The method provided in this embodiment, by traversing a plurality of multimedia materials, according to the target color display standard, determine the non-target color display standard multimedia materials included in the multiple multimedia materials; convert the non-target color display standard multimedia materials into target color display Standard multimedia materials; perform image rendering on multiple multimedia materials according to the target color display standard, so as to display multiple multimedia materials on the display screen of the electronic device according to the image rendering results, or, according to the image rendering results, render multiple multimedia materials The footage is exported as a video file in a specific format. Therefore, it can be realized that the display effect of the multimedia material meets expectations.

On the basis of the embodiment shown in Figure 1, before performing image rendering on the multiple multimedia materials according to the target color display standard, the multimedia materials can also be clipped, for example, according to one or more The editing operation mode corresponding to the editing type is used to edit the multimedia material.

Wherein, the editing type includes, for example: filters, special effects, stickers and the like. The editing operation mode is a specific editing mode provided by the above editing types, for example, the filter may include: a landscape filter, a food filter, and the like.

Fig. 3 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure. Referring to Figure 3, the method provided in this embodiment includes:

S301. Determine a target color display standard.

In some embodiments, the target color display standard may be determined based on an adaptive strategy, that is, the adaptive strategy is a strategy representing the change of the target color display standard along with the target color display standard corresponding to multiple multimedia materials. Exemplarily, the target color display standard may be the color display standard with the widest color space among the color display standards respectively corresponding to the plurality of multimedia materials.

For example, when 3 multimedia materials are imported through application 1, the color display standards corresponding to the 3 multimedia materials are the same, assuming that the 3 multimedia materials are all multimedia materials with SDR color display standards; in the process of video editing, the newly imported A multimedia material, the newly imported multimedia material is a multimedia material of the PQ color display standard, based on the adaptive strategy, it can be determined that the target color display standard is the PQ color display standard, that is, it is necessary to convert the previously imported 3 multimedia materials into PQ Color display standard.

In another possible implementation manner, the target color display standard may be a pre-designated color display standard. For example, the target color display standard may be the color display standard specified by application 1 .

Among them, when determining the target color display standard, it is also necessary to consider the color display standard supported by the electronic device to avoid errors. The color display standard displayed on the screen is tested.

Exemplarily, by checking relevant parameters in a specific API, it can be determined that the electronic device supports the target color display standard for on-screen display. For example, when determining whether an electronic device using the android system supports the HDR color display standard on the screen, the display parameters supported by the screen of the electronic device and the graphics API can be verified. When both verifications pass, the electronic device is determined to be Support HDR color display standard on the screen. This is only an example, and the specific implementation is not limited to the implementation of checking the display parameters supported by the screen of the electronic device and checking the graphics API.

S302. Determine, according to the target color display standard, non-target color display standard multimedia materials included in the plurality of multimedia materials.

S303. Convert the non-target color display standard multimedia material into the target color display standard multimedia material.

S304. Perform image rendering on the multiple multimedia materials according to the target color display standard, so as to display the multiple multimedia materials on the display screen of the electronic device according to the image rendering result, or, according to the image rendering result , exporting the plurality of multimedia materials as video files in a specific format.

Steps S302 to S304 in this embodiment are similar to S101 to S103 in the embodiment shown in FIG. 1 , and reference may be made to the detailed description of the embodiment shown in FIG. 1 . For the sake of brevity, details are not repeated here.

The method provided in this embodiment, by traversing a plurality of multimedia materials, according to the target color display standard, determine the non-target color display standard multimedia materials included in the multiple multimedia materials; convert the non-target color display standard multimedia materials into target color display Standard multimedia materials; according to the target color display standard, image rendering is performed on multiple multimedia materials, and based on the image rendering results, it can be displayed on the screen or exported as a video file in a specific video format. Therefore, the problem of abnormal display effect of the synthesized video obtained by editing multimedia materials with different color display standards is solved. In addition, in the application process, the flexibility of image processing can be improved by determining the target color display standard through an adaptive strategy or a specified method.

Exemplarily, the present disclosure also provides an image processing device.

FIG. 4 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. Referring to FIG. 4, the image processing device 400 provided in this embodiment includes:

The identification module 401 is configured to determine, according to the target color display standard, the multimedia materials included in the plurality of multimedia materials that are not of the target color display standard.

The conversion module 402 is configured to convert the multimedia material of the non-target color display standard into the multimedia material of the target color display standard.

The rendering module 403 is configured to perform image rendering on the plurality of multimedia materials according to the target color display standard, and obtain an image rendering result.

The image processing apparatus 400 may further include: a control module 404, configured to control the display screen of the electronic device to display a plurality of multimedia materials according to the image rendering result, or, according to the image rendering result, control to export the plurality of multimedia materials as A video file of a specific format.

In some embodiments, the identification module 401 is further configured to determine the target color display standard before determining the non-target color display standard multimedia materials included in the plurality of multimedia materials according to the target color display standard.

In some embodiments, the identification module 401 is specifically configured to determine the target color display standard according to the color display standards respectively corresponding to the plurality of multimedia materials.

In some embodiments, the identification module 401 is specifically configured to determine the target color display standard according to the color display standard with the widest color space among the color display standards respectively corresponding to the plurality of multimedia materials.

In some embodiments, the identifying module 401 is specifically configured to determine the target color display standard according to a pre-specified color display standard.

In some embodiments, the conversion module 402 is specifically configured to if the multimedia material of the non-target color display standard is a video at a freeze frame position according to another multimedia material of the non-target color display standard among the multiple multimedia materials If the frame image is generated, the multimedia material of the non-target color display standard is converted into the multimedia material of the target color display standard; or, if the multimedia material of the non-target color display standard is based on the multiple multimedia materials If the target color display standard multimedia material is generated from the video frame image at the freeze position, the non-target color display standard multimedia material will not be converted.

In some embodiments, the target color display standard is the PQ color display standard, and the non-target color display standard includes the SDR color display standard and/or the HLG color display standard; or, the target color display standard is the SDR color display standard Standard, the non-target color display standard includes HLG color display standard and/or PQ color display standard; or, the target color display standard is HLG color display standard, and the non-target color display standard includes SDR color display standard and/or or PQ color display standard.

In some embodiments, the image processing apparatus 400 may further include: a clipping processing module, configured to process the multiple multimedia materials according to clipping operation modes respectively corresponding to one or more clipping types.

In the process of clipping the multimedia material, the rendering module 403 can render the image in real time according to the processing result of the clipping operation mode, and preview the clipping effect on the display screen of the electronic device.

The image processing device provided in this embodiment can be used to execute any of the above method embodiments, and its implementation principles and technical effects are similar, and reference can be made to the detailed description of the foregoing method embodiments. For the sake of brevity, details are not repeated here.

Exemplarily, the present disclosure also provides an electronic device.

Fig. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Referring to FIG. 5 , an electronic device 500 provided in this embodiment includes: a memory 501 and a processor 502 .

Wherein, the memory 501 may be an independent physical unit, and may be connected with the processor 502 through the bus 503 . The memory 501 and the processor 502 may also be integrated together, implemented by hardware, and the like.

The memory 501 is used to store program instructions, and the processor 502 invokes the program instructions to execute the technical solution of any one of the above method embodiments.

Optionally, when part or all of the methods in the foregoing embodiments are implemented by software, the foregoing electronic device 500 may also include only the processor 502 . The memory 501 for storing programs is located outside the electronic device 500, and the processor 502 is connected to the memory through circuits/wires, and is used to read and execute the programs stored in the memory.

The processor 502 may be a central processing unit (central processing unit, CPU), a network processor (network processor, NP) or a combination of CPU and NP.

The processor 502 may further include a hardware chip. The aforementioned hardware chip may be an application-specific integrated circuit (application-specific integrated circuit, ASIC), a programmable logic device (programmable logic device, PLD) or a combination thereof. The aforementioned PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a general array logic (generic array logic, GAL) or any combination thereof.

The memory 501 may include a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); the memory may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory) ), a hard disk (hard disk drive, HDD) or a solid-state drive (solid-state drive, SSD); the memory can also include a combination of the above-mentioned types of memory.

The present disclosure also provides a readable storage medium, including: computer program instructions;

When the computer program instructions are executed by at least one processor of the electronic device, the image processing method shown in any one of the above method embodiments is implemented.

The present disclosure also provides a computer program product. When the computer program product is executed by a computer, the computer implements the image processing method shown in any one of the above method embodiments.

The present disclosure also provides a computer program, including instructions, which, when executed by a processor, enable the processor to implement the image processing method shown in any one of the above method embodiments.

It should be noted that in this article, relative terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these No such actual relationship or order exists between entities or operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above descriptions are only specific implementation manners of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

An image processing method, comprising:

According to the target color display standard, determine the non-target color display standard multimedia materials included in the multiple multimedia materials;

converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard; and

Perform image rendering on the multiple multimedia materials according to the target color display standard, so as to display the multiple multimedia materials on the display screen of the electronic device according to the image rendering result, or, according to the image rendering result, render the The multiple multimedia materials are exported as video files in a specific format.
The method according to claim 1, wherein, before determining the non-target color display standard multimedia materials included in the plurality of multimedia materials according to the target color display standard, the method further comprises:

The target color display standard is determined.
The method according to claim 2, wherein said determining said target color display standard comprises:

The target color display standard is determined according to the color display standard with the widest color space among the color display standards respectively corresponding to the plurality of multimedia materials.
The method according to claim 2, wherein said determining said target color display standard comprises:

The target color display standard is determined according to a pre-designated color display standard.
The method according to any one of claims 1 to 4, wherein said converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard comprises:

If the multimedia material of the non-target color display standard is generated according to the video frame image of another multimedia material of the non-target color display standard in the plurality of multimedia materials at the freeze frame position, then the non-target color display converting the standard multimedia material to display the standard multimedia material in said target color; or

If the multimedia material of the non-target color display standard is generated according to the video frame image of the multimedia material of the target color display standard in the plurality of multimedia materials at the fixed position, then the non-target color display standard is not displayed. multimedia material for conversion.
The method according to any one of claims 1 to 5, wherein said converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard comprises:

Decoding the multimedia material of the non-target color display standard to obtain a first color value set of each pixel of the multimedia material under the non-target color display standard;

Based on the electro-optical conversion function corresponding to the non-target color display standard, converting each color value in the first color value set into an optical signal under the non-target color display standard to obtain a first optical signal set;

Converting each optical signal in the first optical signal set based on an optical-to-optical conversion function to obtain a second optical signal set under the target color display standard; and

Convert each optical signal in the second optical signal set into a color value based on the photoelectric conversion function corresponding to the target color display standard, and obtain a second color value set of the multimedia material under the target color display standard .
The method according to claim 6, wherein, based on the optical-to-optical conversion function, each optical signal in the first optical signal set is converted, and after obtaining the second optical signal set under the target color display standard , also includes: performing color gamut mapping processing to align color values in different color display standards.
The method according to any one of claims 1 to 7, wherein the target color display standard is a perceptually quantized PQ color display standard, and the non-target color display standard includes a standard dynamic range SDR color display standard and/or a mixed pair digital gamma curve HLG color display standard; or,

The target color display standard is the SDR color display standard, and the non-target color display standard includes the HLG color display standard and/or the PQ color display standard; or,

The target color display standard is the HLG color display standard, and the non-target color display standard includes the SDR color display standard and/or the PQ color display standard.
The method according to any one of claims 1 to 8, wherein, before performing image rendering on the plurality of multimedia materials according to the target color display standard, the method further comprises:

The plurality of multimedia materials are processed according to clipping operation modes respectively corresponding to one or more clip types.
An image processing device, comprising:

The identification module is used to determine the non-target color display standard multimedia materials included in the multiple multimedia materials according to the target color display standard;

A conversion module, configured to convert the multimedia material of the non-target color display standard into the multimedia material of the target color display standard;

a rendering module, configured to perform image rendering on the plurality of multimedia materials; and

A control module, configured to control displaying the multiple multimedia materials on the display screen of the electronic device according to the image rendering result, or control exporting the multiple multimedia materials into a video file in a specific format according to the image rendering result .
An electronic device comprising: a memory and a processor;

the memory is configured to store computer program instructions;

The processor is configured to execute the computer program instructions, so that the electronic device implements the image processing method according to any one of claims 1 to 9.
A readable storage medium, comprising: computer program instructions;

When the computer program instructions are executed by at least one processor of the electronic device, the electronic device realizes the image processing method according to any one of claims 1-9.
A computer program product, when the computer program product is executed by a computer, it enables the computer to implement the image processing method according to any one of claims 1 to 9.
A computer program comprising instructions, which, when executed by a processor, cause the processor to implement the image processing method according to any one of claims 1 to 9.