CN115690650A

CN115690650A - Video processing method, video processing apparatus, electronic device, and storage medium

Info

Publication number: CN115690650A
Application number: CN202211268641.6A
Authority: CN
Inventors: 谢俊
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2022-10-17
Filing date: 2022-10-17
Publication date: 2023-02-03

Abstract

The application discloses a video processing method, a video processing device, an electronic device and a computer readable storage medium. The video processing method is used for processing a first video, and comprises the following steps: identifying a scene change location of a first video; dividing the first video into a plurality of first segmented videos according to the scene change position; identifying a scene classification of each first segmented video; acquiring a brightness mapping relation corresponding to scene classification; and converting the first segmented video into a second segmented video according to the brightness mapping relation. In the video processing method, the video processing device, the electronic device and the computer-readable storage medium, the first video is divided into the plurality of first segment videos according to the scene conversion position, the scene classification of each first segment video is identified, and therefore the corresponding brightness mapping relation is adopted for each first segment video to be converted to obtain the second segment video, so that the first video has a better display effect.

Description

Video processing method, video processing apparatus, electronic device, and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a video processing method, a video processing apparatus, an electronic device, and a computer-readable storage medium.

Background

In the related art, in order to display a Standard Dynamic Range (SDR) video on a High-Dynamic Range (HDR) display device, a corresponding luminance variation curve is generated for each frame of an image of the SDR video, and the display effect is poor.

Disclosure of Invention

Embodiments of the present application provide a video processing method, a video processing apparatus, an electronic device, and a computer-readable storage medium.

The video processing method of the embodiment of the application is used for processing a first video, and comprises the following steps: identifying a scene transition location of the first video; dividing the first video into a plurality of first segmented videos according to the scene change positions; identifying a scene classification for each of the first segmented videos; acquiring a brightness mapping relation corresponding to the scene classification; and converting the first segmented video into a second segmented video according to the brightness mapping relation. .

The video processing device comprises a first identification module, a first processing module, a second identification module, an acquisition module and a second processing module. The first identification module is used for identifying a scene change position of the first video; the first processing module is used for dividing the first video into a plurality of first segmented videos according to the scene conversion position; the second identification module is used for identifying the scene classification of each first segmented video; the obtaining module is used for obtaining a brightness mapping relation corresponding to the scene classification; the second processing module is used for converting the first segmented video into a second segmented video according to the brightness mapping relation.

The electronic device of the embodiment of the application comprises a processor. The processor is configured to identify a scene transition location of the first video; dividing the first video into a plurality of first segmented videos according to the scene change positions; identifying a scene classification for each of the first segmented videos; acquiring a brightness mapping relation corresponding to the scene classification; and converting the first segmented video into a second segmented video according to the brightness mapping relation.

The computer-readable storage medium of the present embodiment has stored thereon a computer program which, when executed by a processor, implements the steps of the video processing method as described above.

According to the video processing method, the video processing device, the electronic equipment and the computer readable storage medium, the first video is divided into the plurality of first segmented videos according to the scene conversion position, the scene classification of each first segmented video is identified, and therefore each first segmented video is converted by adopting the corresponding brightness mapping relation to obtain the second segmented video, and therefore the first video has a better display effect.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic flow diagram of a video processing method according to some embodiments of the present application;

FIG. 2 is a schematic diagram of a video processing apparatus according to some embodiments of the present application;

FIG. 3 is a schematic view of an electronic device of some embodiments of the present application;

fig. 4-7 are schematic flow charts of video processing methods according to some embodiments of the present disclosure;

FIG. 8 is a graph illustrating variation of a frame number with respect to a first Euler distance in accordance with certain embodiments of the present application;

fig. 9-12 are schematic flow charts of video processing methods according to some embodiments of the present application;

FIG. 13 is a schematic illustration of a luminance mapping curve according to some embodiments of the present application;

fig. 14 is a schematic connection diagram of an electronic device and a computer-readable storage medium according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.

Referring to fig. 1, a video processing method according to the present application is used for processing a first video, and the video processing method includes:

01: identifying a scene change location of a first video;

02: dividing the first video into a plurality of first segmented videos according to the scene change position;

03: identifying a scene classification of each first segmented video;

04: acquiring a brightness mapping relation corresponding to scene classification;

05: and converting the first segmented video into a second segmented video according to the brightness mapping relation.

Referring to fig. 2, the present application discloses a video processing apparatus 10. The video processing apparatus 10 includes a first identifying module 11, a first processing module 12, a second identifying module 13, an obtaining module 14, and a second processing module 15. The video processing method according to the embodiment of the present application can be implemented by the video processing apparatus 10 according to the embodiment of the present application, wherein the step 01 can be implemented by the first identification module 11. Step 02 may be implemented by the first processing module 12. Step 03 may be implemented by the second identification module 13. Step 04 may be implemented by the obtaining module 14. Step 05 may be implemented by the second processing module 15. That is, the first identifying module 11 is configured to identify a scene change location of the first video. The first processing module 12 is configured to divide the first video into a plurality of first segment videos according to the scene transition position. The second identifying module 13 is configured to identify a scene classification of each first segmented video. The obtaining module 14 is configured to obtain a luminance mapping relationship corresponding to the scene classification. The second processing module 15 is configured to convert the first segmented video into the second segmented video according to the luminance mapping relationship.

Referring to fig. 3, the present application discloses an electronic device 100. The electronic device 100 includes a processor 20, the processor 20 configured to identify a scene transition location of a first video; dividing the first video into a plurality of first segmented videos according to the scene change position; identifying a scene classification of each first segmented video; acquiring a brightness mapping relation corresponding to scene classification; and converting the first segmented video into a second segmented video according to the brightness mapping relation.

According to the video processing method, the video processing device 10 and the electronic equipment 100 of the embodiment of the application, the first video is divided into the plurality of first segmented videos according to the scene conversion position, the scene classification of each first segmented video is identified, and therefore the corresponding brightness mapping relation is adopted for each first segmented video to carry out conversion so as to obtain the second segmented video, and therefore the first video has a better display effect.

In some embodiments, the electronic device 100 may be a terminal device having a function of capturing video. For example, the electronic device 100 may include a smart phone, a tablet computer, and a digital camera or other terminal devices having a function of capturing video. The electronic device 100 according to the embodiment of the present application is illustrated as a smart phone, and is not to be construed as a limitation to the present application.

In one embodiment, the first video comprises Standard Dynamic Range imaged video (i.e., SDR video), and the second video comprises High Dynamic Range imaged video (i.e., HDR video). SDR video utilizes standard dynamic range imaging techniques. HDR video utilizes high dynamic range imaging. In computer graphics and cinematography, is a group of techniques used to achieve a greater dynamic range of exposure (i.e., greater shading) than common digital image techniques. Thus, the video processing method of the embodiment of the application can be used for processing the SDR video.

Referring to fig. 4, an SDR video is first input, an SDR video is identified to obtain a scene transition position of the SDR video, and the SDR video is segmented into a plurality of segments of the SDR video (the plurality of segments of the SDR video are a plurality of first segmented videos) according to the scene transition position, so that scenes with larger differences in the SDR video can be distinguished. And identifying the segments of the SDR videos to realize scene classification, wherein the segments of each SDR video correspondingly look for a brightness mapping relation according to the scene classification, and then the segments of each SDR video are converted into the segments of each HDR video according to the brightness mapping relation. Therefore, the SDR video can have better display effect on the HDR display equipment.

It should be noted that the luminance mapping relationship corresponding to the scene classification may be preset. Therefore, the brightness mapping relation can be determined according to the scene classification more quickly, and the SDR video is converted into the HDR video according to the brightness mapping relation. The method can effectively improve the processing speed of the video, reduce the calculated amount and have better display effect.

Referring to fig. 5, in some embodiments, step 01 includes:

011: counting a histogram of each color channel of each frame of the first image;

012: determining a feature vector of each frame of first image according to the histogram of each color channel of each frame of first image;

013: calculating a first Euler distance of a current first image relative to a first target image, a second Euler distance relative to a second target image and a third Euler distance relative to a third target image according to the feature vectors, wherein the first target image is a first image of a previous frame of the current first image, the second target image is a first image of the current first image before a first set time, the third target image is a first image of the current first image before a second set time, and the second set time is longer than the first set time;

014: and determining a division frame number as a scene change position according to the first Euler distance, the second Euler distance and the third Euler distance.

In some embodiments, step 011, step 012, step 013, and step 014 can all be implemented by video processing apparatus 10. That is, the video processing apparatus 10 is configured to: counting a histogram of each color channel of each frame of the first image; determining a feature vector of each frame of first image according to the histogram of each color channel of each frame of first image; calculating a first Euler distance of a current first image relative to a first target image, a second Euler distance relative to a second target image and a third Euler distance relative to a third target image according to the feature vectors, wherein the first target image is a first image of a previous frame of the current first image, the second target image is a first image of the current first image before a first set time, the third target image is a first image of the current first image before a second set time, and the second set time is longer than the first set time; and determining a division frame number as a scene change position according to the first Euler distance, the second Euler distance and the third Euler distance.

In some embodiments, step 011, step 012, step 013, and step 014 may all be implemented by processor 20 of electronic device 100. That is, processor 20 is configured to: counting a histogram of each color channel of each frame of the first image; determining a feature vector of each frame of first image according to the histogram of each color channel of each frame of first image; calculating a first Euler distance of a current first image relative to a first target image, a second Euler distance relative to a second target image and a third Euler distance relative to a third target image according to the feature vectors, wherein the first target image is a first image of a previous frame of the current first image, the second target image is a first image of the current first image before a first set time, the third target image is a first image of the current first image before a second set time, and the second set time is longer than the first set time; and determining a division frame number as a scene change position according to the first Euler distance, the second Euler distance and the third Euler distance.

In particular, the first video may be an SDR video and the first segmented video may be an SDR video segment. The first segmented video includes a first picture, which may be an SDR picture. The histogram distribution of each frame SDR image is calculated. The SDR image may include R, G, B three color channels. Wherein, R is a red channel, G is a green channel, and B is a blue channel. The SDR image is a color image, each color channel R, G, B is calculated respectively, a histogram of each color channel determines a feature vector of each frame of the SDR image, and finally the feature vectors are spliced together. The final histogram is obtained by directly stitching together the results of the three channels. In one embodiment, the value of each color channel is usually represented by 8 bits, i.e. the value ranges from 0 to 255, and the histogram length of each color channel is 256 (i.e. counting the number of occurrences of each value for each pixel in the SDR image for each color channel), and the results of the three channels are directly merged, and the SDR image will generate a feature vector with a length of 256 × 3=768.

After determining the current first image, determining a first target image, a second target image and a third target image according to the current first image. The first target image is a first image of a previous frame of the current first image, the second target image is a first image of the current first image before a first set time, the third target image is a first image of the current first image before a second set time, and the second set time is longer than the first set time. And calculating a first Euler distance of the current first image relative to the first target image, a second Euler distance relative to the second target image and a third Euler distance relative to the third target image by using the feature vector, and finally determining a segmentation frame number according to the first Euler distance, the second Euler distance and the third Euler distance to be used as a scene conversion position.

In one embodiment, the current first image may be a current SDR image, the first target image may be a first SDR image, and the first SDR image is a previous frame image of the current SDR image. The second target image may be a second SDR image, the second SDR image being an image before the first set time of the current SDR image, and the first set time may be 0.5 seconds. The third target image may be a third SDR image, the third SDR image being an image before a second set time of the current SDR image, and the second set time may be 1 second. Calculating a first Euler distance of the current SDR image relative to the first SDR image according to the characteristic vector, and recording the first Euler distance as A [ i [ ]](ii) a Calculating a second Euler distance of the current SDR image relative to a second SDR image according to the characteristic vector, and marking the second Euler distance as B [ i](ii) a Calculating a third Euler distance of the current SDR image relative to a third SDR image according to the feature vector, and recording the third Euler distance as C [ i]The length of the eigenvector may be 256 × 3=768.i represents the current frame number. Euler distance calculation may use the euler distance calculation formula:

the squares of all corresponding element differences are summed and then root-coded. After the first Euler distance, the second Euler distance and the third Euler distance are calculated, segmentation is determined according to the first Euler distance, the second Euler distance and the third Euler distanceThe frame number is used as a scene change position. In some embodiments, the frame number with the largest euler distance in the histogram may be selected, i.e. the segmentation frame number, so that the segmentation frame number may be determined as the scene transition location.

It is worth mentioning that, assuming that the current SDR picture is at the beginning of the video, it may not have the corresponding first SDR picture of the previous frame or the second SDR picture 0.5 seconds ago, the third SDR picture 1 seconds ago, and then set the corresponding euler distance to 0.

Referring to fig. 6, in some embodiments, step 01 further includes:

015: adjusting the resolution of each frame of first image to a preset resolution;

step 011 includes:

0111: and counting the histogram of each color channel of each frame of the adjusted first image.

In some embodiments, step 015 and step 0111 may be implemented by the video processing apparatus 10. That is, the video processing apparatus 10 is configured to: adjusting the resolution of each frame of first image to a preset resolution; and counting the histogram of each color channel of each frame of the adjusted first image.

In some embodiments,

steps

015 and 0111 may be implemented by the processor 20 of the electronic device 100. That is, processor 20 is configured to: adjusting the resolution of each frame of first image to a preset resolution; and counting the histogram of each color channel of each frame of the adjusted first image.

Specifically, a preset resolution may be set, the resolution of the first image of all frames may be adjusted to the preset resolution, and the first image of all frames may be adjusted to a standard size. Therefore, the video with different resolutions can be conveniently and universally determined by subsequently determining the segmentation frame number, the corresponding brightness mapping relation and the like. After the resolution of each frame of first image is adjusted to a preset resolution, the histogram of each color channel of each frame of first image after adjustment is counted. In one embodiment, the preset resolution may be 1920 × 1080, so that the resolution of the first image per frame is adjusted to 1920 × 1080, and the histogram of each color channel is counted after the adjustment is completed.

Referring to fig. 7, in some embodiments, the split frame numbers include a first split frame number, a second split frame number, and a third split frame number, and step 014 includes:

0140: determining a first image frame number corresponding to a first pre-selected Euler distance greater than a first threshold in the first Euler distances;

0141: dividing the first image frame number into a plurality of groups according to the continuity of the image frame numbers;

0142: acquiring a frame number with the maximum first Euler distance corresponding to each group of first image frame numbers as a first segmentation frame number;

0143: determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances;

0144: dividing the second image frame number into a plurality of groups according to the continuity of the image frame numbers;

0145: acquiring a frame number with the maximum second Euler distance corresponding to each group of second image frame numbers as a second segmentation frame number;

0146: determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold in the third Euler distances;

0147: dividing the third image frame number into a plurality of groups according to the continuity of the image frame numbers;

0148: and acquiring a frame number with the maximum third Euler distance corresponding to each group of third image frame numbers as a third segmentation frame number.

In some embodiments,

steps

0140, 0141, 0142, 0143, 0144, 0145, 0146, 0147, and 0148 may all be implemented by video processing device 10. That is, the video processing apparatus 10 is configured to: determining a first image frame number corresponding to a first pre-selected Euler distance greater than a first threshold in the first Euler distances; dividing the first image frame number into a plurality of groups according to the continuity of the image frame numbers; acquiring a frame number with the maximum first Euler distance corresponding to each group of first image frame numbers as a first segmentation frame number; determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances; dividing the second image frame number into a plurality of groups according to the continuity of the image frame numbers; acquiring a frame number with the maximum second Euler distance corresponding to each group of second image frame numbers as a second segmentation frame number; determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold in the third Euler distances; dividing the third image frame number into a plurality of groups according to the continuity of the image frame numbers; and acquiring a frame number with the maximum third Euler distance corresponding to each group of third image frame numbers as a third segmentation frame number.

In certain embodiments,

steps

0140, 0141, 0142, 0143, 0144, 0145, 0146, 0147, and 0148 may all be implemented by the processor 20 of the electronic device 100. That is, processor 20 is configured to: determining a first image frame number corresponding to a first pre-selected Euler distance greater than a first threshold in the first Euler distances; dividing the first image frame number into a plurality of groups according to the continuity of the image frame numbers; acquiring a frame number with the maximum first Euler distance corresponding to each group of first image frame numbers as a first segmentation frame number; determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances; dividing the second image frame number into a plurality of groups according to the continuity of the image frame numbers; acquiring a frame number with the maximum second Euler distance corresponding to each group of second image frame numbers as a second segmentation frame number; determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold in the third Euler distances; dividing the third image frame number into a plurality of groups according to the continuity of the image frame numbers; and acquiring a frame number with the maximum third Euler distance corresponding to each group of third image frame numbers as a third segmentation frame number.

In some embodiments, after the first euler distance, the second euler distance, and the third euler distance are calculated, the split frame number may be determined as the scene change position by combining the preset first threshold, the preset second threshold, and the preset third threshold. And (3) preselecting frames which are larger than a threshold value and are frames with larger image change, wherein the frame number with the maximum Euler distance of the histogram is selected as a segmentation frame number, so that the segmentation frame number can be determined as a scene conversion position.

Specifically, the first euler distances of all the first images with respect to the last frame image of the first images may be calculated using the feature vectors. And judging all first preselected Euler distances of which the first Euler distances are greater than the first threshold value by combining a preset first threshold value, and determining a first image frame number corresponding to each first preselected Euler distance. All first Euler distances are denoted as A [ i ], and the first threshold may be denoted as THA. The first image frame numbers having continuity are divided into a plurality of groups. For example: the frame number 6, the frame number 7 and the frame number 8 have continuity, and images with the frame number 6, the frame number 7 and the frame number 8 can be divided into a group; the frame number 18, the frame number 19, the frame number 20, and the frame number 21 have continuity, and images with the frame number 18, the frame number 19, the frame number 20, and the frame number 21 may be divided into one group. Referring to fig. 8, the frame number with the maximum first euler distance corresponding to the first image frame number is used as the first split frame number, so that the first split frame number can be obtained.

A second euler distance of each first image with respect to the first image before the first set time may be calculated using the feature vector. The first set time may be 0.5 seconds. And judging all second preselected Euler distances of which the second Euler distances are larger than a second threshold value by combining a preset second threshold value, and determining a second image frame number corresponding to each second preselected Euler distance. All second Euler distances are denoted as B [ i ], and the second threshold may be denoted as thB. The second image frame numbers having continuity are divided into a plurality of groups. The second image frame numbers having continuity are divided into a plurality of groups. And then taking the frame number with the maximum second Euler distance corresponding to the frame number of the second image as a second segmentation frame number, thus obtaining the second segmentation frame number.

A third euler distance of each first image with respect to the first image before the second set time may be calculated using the feature vector. The second set time may be 1 second. And judging all third preselected Euler distances of which the third Euler distances are greater than the third threshold value by combining a preset third threshold value, and determining a third image frame number corresponding to each third preselected Euler distance. All third Euler distances are denoted as C [ i ], and the third threshold may be denoted as thC. The third image frame numbers having continuity are divided into a plurality of groups. The third image frame numbers having continuity are divided into a plurality of groups. And then taking the frame number with the maximum third Euler distance corresponding to the third image frame number as a third segmentation frame number, thus obtaining the third segmentation frame number.

Thus, the first, second, and third split frame numbers can be extracted. The multi-threshold Euler distance calculation method has the advantages that the multi-threshold Euler distances of the histograms of the current frame and the frames of a plurality of previous time points are calculated, the situation that transition between videos cannot be identified can be avoided, and the identification effect can be effectively improved.

Referring to fig. 9, in some embodiments, step 014 further includes:

01491: determining a first identified image according to the first segmentation frame number;

01492: determining a second identified image according to the second segmentation frame number;

step 0143 includes:

01441: after the first identified image is removed, determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances;

step 0146 includes:

01461: and after the first recognized image and the second recognized image are removed, determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold value in the third Euler distances.

In certain embodiments,

steps

01491, 01492, 01441 and 01461 may all be implemented by the video processing device 10. That is, the video processing apparatus 10 is configured to: determining a first identified image according to the first segmentation frame number; determining a second identified image according to the second segmentation frame number; after the first identified image is removed, determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances; and after the first recognized image and the second recognized image are removed, determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold value in the third Euler distances.

In certain embodiments,

steps

01491, 01492, 01441 and 01461 may each be implemented by the processor 20 of the electronic device 100. That is, the processor 20 is configured to: determining a first identified image according to the first segmentation frame number; determining a second identified image according to the second segmentation frame number; after the first identified image is removed, determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances; and after the first recognized image and the second recognized image are removed, determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold value in the third Euler distances.

Specifically, a first identified image may be determined from the first segmented frame number, and the first identified image may include a plurality of images, for example: the first identified image includes all images within one second before and after the first segmented frame number. A second identified image may be determined from the second split frame number, and the second identified image may include multiple images, such as: the second identified image includes all images within one second before and after the second split frame number. And removing the first identified image, and determining a second image frame number corresponding to a second pre-selected Euler distance which is greater than a second threshold value in the second Euler distance. Therefore, the first segmentation frame number identified by the previous process can be eliminated, and repeated identification is prevented.

And after the first recognized image and the second recognized image are removed, determining a third image frame number corresponding to a third preselected Euler distance which is greater than a third threshold value in the third Euler distances. Therefore, the first division frame number and the second division frame number identified by the previous process can be eliminated, and repeated identification is prevented.

It will be appreciated that the more segmented frame number identification of the previous flow, the more severe the identified segmented frame number video transitions. For example, the first split frame number identified by thA may be recorded by using the first euler distance and the first threshold, and the video transition is the most severe (the euler distance between histograms is larger because of severe changes in image content between the previous and next 2 frames), so that frames within one second before and after the identified first split frame number do not participate in the subsequent identification process any more, and repeated identification can be effectively avoided.

Referring to fig. 10, in some embodiments, step 03 includes:

031: inputting each frame of first image of each first segmented video into a picture classifier for classification so as to determine the scene type of each frame of first image;

032: and counting the number of images corresponding to each scene type in each first segmented video and determining the scene type with the largest number of images as a scene classification.

In some embodiments, step 031 and step 032 can both be implemented by the video processing apparatus 10. That is, the video processing apparatus 10 is configured to: inputting each frame of first image of each first segmented video into a picture classifier for classification so as to determine the scene type of each frame of first image; and counting the number of images corresponding to each scene type in each first segmented video and determining the scene type with the largest number of images as the scene classification.

In some embodiments, step 031 and step 032 may both be implemented by processor 20 of electronic device 100. That is, processor 20 is configured to: inputting each frame of first image of each first segmented video into a picture classifier for classification so as to determine the scene type of each frame of first image; and counting the number of images corresponding to each scene type in each first segmented video and determining the scene type with the largest number of images as a scene classification.

In one embodiment, the first video may be an SDR video and the first segmented video may be a segment of the SDR video. The method comprises the steps of determining a scene conversion position of an SDR video, segmenting the SDR video into a plurality of video segments (namely segmenting into a plurality of first segmented videos) according to the scene conversion position, inputting each frame of first image of each video segment into a picture classifier, determining the scene type of each frame of first image by the picture classifier, counting the number of images corresponding to each scene type in each video segment, and determining the scene type with the largest number of images as the scene type. Therefore, each video clip can be classified, different parameters can be conveniently used for optimizing each video clip in the later period, and the display effect is improved.

In some embodiments, each frame of the first image of each first segment video may be input to a picture classifier for classification using a deep learning method. Specifically, a picture classifier is trained in advance, and the picture classification can classify the first image of each frame into different classes. For example: the scene type of the first image is classified as seaside, forest, indoor, city, etc. And sending each frame of first image of each first segmented video into a classifier for classification, and finally setting the most scene type obtained by the plurality of frames of first images of the first segmented video as the scene type of the first segmented video. In one example, the current first segmented video includes 100 frames of the first image. And sending each frame of first image of the current first subsection video into a classifier for classification, wherein the scene type of 85 frames of first images is seaside, the scene type of 15 frames of first images is city, and thus the scene type of seaside is determined as the scene type of the current first subsection video.

Referring to fig. 11, in some embodiments, before step 05, the video processing method further includes:

061: generating a buffer area, wherein the buffer area is used for the output display of the second display equipment;

after step 05, the video processing method further comprises:

062: the second segmented video is output to a buffer.

In some embodiments,

steps

061 and 062 may be implemented by the video processing apparatus 10. That is, the video processing apparatus 10 is configured to: generating a buffer area, wherein the buffer area is used for the second display equipment to output and display; the second segmented video is output to a buffer.

In some embodiments,

steps

061 and 062 may each be implemented by processor 20 of electronic device 100. That is, processor 20 is configured to: generating a buffer area, wherein the buffer area is used for the second display equipment to output and display; the second segmented video is output to a buffer.

Specifically, before the first segmented video is converted into the second segmented video according to the luminance mapping relationship, a buffer may be generated first, and the buffer is used for the second display device to perform output display. The second display device may be an HDR display device. The first segmented video may be a video segment of an SDR video and the second segmented video may be a video segment of an HDR video. And after the buffer area is generated, converting the video clip of the SDR video into the corresponding video clip of the HDR video according to the brightness mapping relation, and outputting the second video clip to the buffer area. Therefore, the buffer area is utilized, the video format can not be concerned, the final display effect of the video can be concerned, the complexity of video processing can be reduced, and the second display equipment can be ensured to normally display.

Referring to fig. 12, in some embodiments, the luminance mapping relationship includes a correspondence relationship between input luminance and output luminance, the first segmented video includes a first image, the second segmented video includes a second image, and step 05 includes:

051: determining a luminance value of a current pixel of a first image as an input luminance;

052: determining output brightness according to the input brightness and the brightness mapping relation;

053: determining a gain value according to the output brightness and the input brightness;

054: gaining the color channel of the current pixel according to the gain value;

055: all pixels of the first image are gain processed to convert the first image into the second image.

In certain embodiments, step 051, step 052, step 053, step 054 and step 055 may all be implemented by video processing device 10. That is, the video processing apparatus 10 is configured to: determining a luminance value of a current pixel of a first image as an input luminance; determining output brightness according to the input brightness and the brightness mapping relation; determining a gain value according to the output brightness and the input brightness; gaining the color channel of the current pixel according to the gain value; all pixels of the first image are gain processed to convert the first image into the second image.

In certain embodiments, step 051, step 052, step 053, step 054 and step 055 may all be implemented by processor 20 of electronic device 100. That is, processor 20 is configured to: determining a luminance value of a current pixel of a first image as an input luminance; determining output brightness according to the input brightness and the brightness mapping relation; determining a gain value according to the output brightness and the input brightness; gaining the color channel of the current pixel according to the gain value; all pixels of the first image are gain processed to convert the first image into the second image.

In particular, the first video may be an SDR video and the first segmented video may be an SDR video segment. The first segmented video includes a first picture, which may be an SDR picture. The second segmented video may be HDR video segments, each HDR video segment corresponding to each SDR video segment, and the second image may be an HDR image. And determining the brightness value of the current pixel of the SDR image to be used as input brightness, and determining output brightness according to the mapping relation of the input brightness and the brightness. And determining output brightness, determining a gain value by combining with input brightness, performing gain on a color channel of a current pixel of the SDR image by using the gain value, and finally performing gain processing on all pixels of the SDR image to convert the SDR image into the HDR image. Therefore, the first segmented video can be converted into the second segmented video according to the brightness mapping relation, and the display effect can be optimized.

Specifically, the luminance mapping relationship includes a luminance transformation curve. As shown in fig. 13, the abscissa of the luminance mapping curve is the input pixel luminance (i.e., the input luminance of the current pixel), and the ordinate of the luminance mapping curve is the output pixel luminance (i.e., the output luminance of the current pixel). In this manner, the output luminance may be determined from the input luminance and the luminance mapping curve.

It is worth mentioning that the three color channels of the SDR video R, G, B are all represented by 8 bits, and each color channel of the HDR image may be 10 bits or higher, depending on the support of the display device (e.g., projector, display), which requires different luminance mapping curves for different target color bit numbers for compatibility.

Thus, each scene classification requires a luminance mapping curve to be adapted for the luminance value of each pixel in the first image. In one embodiment, determining the luminance value of the current pixel of the first image may use a calculation formula for the luminance of the pixel, such as: l = R0.299G 0.587+ B0.114. Wherein R, G, B is the value of the pixel with red, green and blue 3 channels, R is the red channel, G is the green channel, and B is the blue channel. A weighted average formula may also be used, such as: l = R0.333 + G0.333 + B0.333. Wherein R, G, B is the value of the pixel with red, green and blue 3 channels, R is the red channel, G is the green channel, and B is the blue channel. The luminance value of the current pixel of the first image is calculated as the input luminance, and the input luminance is denoted as Lin. And obtaining output brightness according to the applied brightness mapping curve, and recording the output brightness as Lout. The Gain value is determined using the output luminance and the input luminance, and is noted as Gain, where Gain = Lout/Lin. The Gain value is used for gaining the R, G, B channel of the current pixel, wherein Rout = R × Gain, gout = G × Gain and Bout = B × Gain. In this way, all pixels of the first image can be gain-processed using the luminance mapping curve to convert the first image into the second image, and the display effect can also be optimized.

Referring to fig. 1 and 14 together, the present application discloses a computer readable storage medium 300 having a computer program stored thereon, wherein when the computer executable instructions are executed by one or more processors 20, the processors 20 execute the steps of the video processing method according to any one of the above embodiments of the present application. For example, the following video processing methods are completed:

01: identifying a scene change location of a first video;

03: identifying a scene classification of each first segmented video;

As shown in fig. 14, the video processing method according to the embodiment of the present application can be implemented by the electronic device 100 according to the embodiment of the present application. Note that the computer-readable storage medium 300 may be a storage medium built in the electronic device 100, or may be a storage medium that can be plugged into the electronic device 100.

In the description of the embodiments of the present application, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

In the description herein, reference to the description of the terms "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example" or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.

The logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processing module-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

It should be understood that portions of the embodiments of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.

Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations of the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

Claims

1. A video processing method for processing a first video, the video processing method comprising:

identifying a scene transition location of the first video;

dividing the first video into a plurality of first segmented videos according to the scene change positions;

identifying a scene classification for each of the first segmented videos;

acquiring a brightness mapping relation corresponding to the scene classification;

and converting the first segmented video into a second segmented video according to the brightness mapping relation.

2. The method of claim 1, wherein the first video comprises a plurality of frames of a first image, and wherein identifying the scene transition location of the first video comprises:

counting a histogram of each color channel of each frame of the first image;

determining a feature vector of each frame of the first image according to a histogram of each color channel of each frame of the first image;

calculating a first Euler distance of a current first image relative to a first target image, a second Euler distance relative to a second target image and a third Euler distance relative to a third target image according to the feature vectors, wherein the first target image is a first image of a previous frame of the current first image, the second target image is a first image of the current first image before a first set time, the third target image is a first image of the current first image before a second set time, and the second set time is longer than the first set time;

and determining a segmentation frame number according to the first Euler distance, the second Euler distance and the third Euler distance to serve as the scene conversion position.

3. The video processing method of claim 2, wherein the identifying the scene transition location of the first video further comprises:

adjusting the resolution of each frame of the first image to a preset resolution;

the statistics of the histogram of each color channel of each frame of the first image comprises:

and counting the histogram of each color channel of each frame of the adjusted first image.

4. The video processing method of claim 2, wherein the split frame number comprises a first split frame number, a second split frame number, and a third split frame number; the determining a division frame number as the scene transition position according to the first euler distance, the second euler distance, and the third euler distance includes:

determining a first image frame number corresponding to a first pre-selected Euler distance greater than a first threshold in the first Euler distances;

dividing the first image frame number into a plurality of groups according to the continuity of the image frame numbers;

acquiring a frame number with the maximum first Euler distance corresponding to each group of first image frame numbers as the first segmentation frame number;

determining a second image frame number corresponding to a second preselected Euler distance which is greater than a second threshold value in the second Euler distances;

dividing the second image frame number into a plurality of groups according to the continuity of the image frame numbers;

acquiring a frame number with the maximum second Euler distance corresponding to each group of second image frame numbers as the second segmentation frame number;

determining a third image frame number corresponding to a third preselected Euler distance greater than a third threshold in the third Euler distances;

dividing the third image frame number into a plurality of groups according to the continuity of the image frame numbers;

and acquiring a frame number with the maximum third Euler distance corresponding to each group of the third image frame numbers as the third segmentation frame number.

5. The video processing method according to claim 4, wherein said determining a split frame number as the scene change position based on the first Euler distance, the second Euler distance, and the third Euler distance further comprises:

determining a first identified image according to the first segmentation frame number;

determining a second identified image according to the second segmentation frame number;

the determining a second image frame number corresponding to a second preselected euler distance greater than a second threshold value in the second euler distances comprises:

after the first identified image is removed, determining a second image frame number corresponding to a second preselected Euler distance which is greater than the second threshold value in the second Euler distances;

the determining a third image frame number corresponding to a third preselected euler distance greater than a third threshold among the third euler distances comprises:

and after removing the first recognized image and the second recognized image, determining a third image frame number corresponding to a third preselected Euler distance greater than the third threshold value in the third Euler distances.

6. The method of claim 1, wherein the identifying the scene classification of each of the first segmented videos comprises:

inputting each frame of first image of each first segmented video into a picture classifier for classification so as to determine the scene type of each frame of the first image;

and counting the number of images corresponding to each scene type in each first segmented video and determining the scene type with the largest number of images as the scene classification.

7. The video processing method according to claim 1, wherein before converting the first segmented video into the second segmented video according to the luminance mapping relationship, the video processing method further comprises:

generating a buffer area, wherein the buffer area is used for output display of second display equipment;

after converting the first segmented video into a second segmented video according to the luminance mapping relationship, the video processing method further includes:

outputting the second segmented video to the buffer.

8. The method according to claim 1, wherein the luminance mapping relationship comprises a correspondence relationship between input luminance and output luminance, wherein the first segmented video comprises a first image, wherein the second segmented video comprises a second image, and wherein converting the first segmented video into the second segmented video according to the luminance mapping relationship comprises:

determining a luminance value of a current pixel of the first image as the input luminance;

determining the output brightness according to the input brightness and the brightness mapping relation;

determining a gain value according to the output brightness and the input brightness;

gaining the color channel of the current pixel according to the gain value;

gain processing is performed on all pixels of the first image to convert the first image into the second image.

9. A video processing apparatus for processing a first video, the video processing apparatus comprising:

a first identification module to identify a scene transition location of the first video;

a first processing module to segment the first video into a plurality of first segmented videos according to the scene transition location;

a second identification module for identifying a scene classification of each of the first segmented videos;

the acquisition module is used for acquiring a brightness mapping relation corresponding to the scene classification;

and the second processing module is used for converting the first segmented video into a second segmented video according to the brightness mapping relation.

10. An electronic device, comprising a processor configured to identify a scene transition location of the first video; dividing the first video into a plurality of first segmented videos according to the scene change positions; identifying a scene classification for each of the first segmented videos; acquiring a brightness mapping relation corresponding to the scene classification; and converting the first segmented video into a second segmented video according to the brightness mapping relation.

11. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, carries out the steps of the video processing method according to claims 1 to 8.