CN110555799A - Method and apparatus for processing video - Google Patents
Method and apparatus for processing video Download PDFInfo
- Publication number
- CN110555799A CN110555799A CN201910919210.3A CN201910919210A CN110555799A CN 110555799 A CN110555799 A CN 110555799A CN 201910919210 A CN201910919210 A CN 201910919210A CN 110555799 A CN110555799 A CN 110555799A
- Authority
- CN
- China
- Prior art keywords
- image
- channel
- video
- fusion
- video frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000006243 chemical reaction Methods 0.000 claims abstract description 10
- 230000004927 fusion Effects 0.000 claims description 69
- 238000010586 diagram Methods 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 6
- 238000010421 pencil drawing Methods 0.000 abstract description 18
- 230000000694 effects Effects 0.000 description 7
- 239000003086 colorant Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the disclosure discloses a method and a device for processing video. One embodiment of the method comprises: for the video frames in the acquired original video, the following processing operations are performed: obtaining an edge image based on the gray level image corresponding to the video frame; fusing the edge image and the gray level image to obtain a first fused image; fusing the first fused image and a preset sketch stroke image to obtain a second fused image; converting the video frame into an LAB mode to obtain an A channel and a B channel of the video frame; obtaining a third fused image for the video frame based on the second fused image, the channel A and the channel B; and generating a video based on the obtained third fused image, and outputting the generated video. The implementation realizes the conversion of the original video to the colored pencil drawing style video.
Description
Technical Field
The disclosed embodiments relate to the field of computer technologies, and in particular, to a method and an apparatus for processing a video.
Background
In daily life, after people shoot a video, in order to enable the video to have a certain artistic style, the video is subjected to post-processing to generate a video with an artistic effect, for example, a video with a color pencil drawing style is generated. Currently, the means for generating video with a colored pencil drawing style is mostly based on neural networks. For example, a neural network is trained by using a large amount of sample data, and then the trained neural network is used for processing a video, so that a color pencil drawing style video is obtained. Generally, neural networks are computationally expensive and require higher-configured hardware devices for support. In addition, the video frames in the video can be processed by using a more traditional image processing mode, so that the video with the color pencil drawing style is obtained. The traditional image processing mode is poor in processing effect on color, brightness and the like, so that the obtained video effect of the color pencil painting style is poor.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for processing video.
In a first aspect, an embodiment of the present disclosure provides a method for processing video, where the method includes: for the video frames in the acquired original video, the following processing operations are performed: obtaining an edge image based on the gray level image corresponding to the video frame; fusing the edge image and the gray level image to obtain a first fused image; fusing the first fused image and a preset sketch stroke image to obtain a second fused image; converting the video frame into an LAB mode to obtain an A channel and a B channel of the video frame; obtaining a third fused image for the video frame based on the second fused image, the channel A and the channel B; and generating a video based on the obtained third fused image, and outputting the generated video.
In some embodiments, the obtaining a third fused image for the video frame based on the second fused image, the a channel, and the B channel includes: processing the original values of the channel A and the channel B respectively to obtain a first image for the channel A and a second image for the channel B; obtaining an LAB image based on the second fused image, the first image, and the second image; and converting the LAB image into an RGB image, and taking the RGB image as a third fusion image for the video frame.
In some embodiments, the processing the original values of the a channel and the B channel respectively to obtain a first image for the a channel and a second image for the B channel includes: processing the original value of the channel A based on a preset first coefficient to obtain a first image; and processing the original value of the B channel based on a preset second coefficient to obtain a second image.
In some embodiments, the obtaining an edge image based on the gray-scale image corresponding to the video frame includes: carrying out reverse color processing on the gray level image corresponding to the video frame to obtain a reverse color image; and performing edge extraction on the color reflection image to obtain an edge image.
In some embodiments, the fusing the first fused image and the preset sketch stroke map to obtain a second fused image includes: and for the pixel points in the first fusion image, in response to the fact that the pixel values of the pixel points are smaller than a preset threshold value, fusing the pixel values of the pixel points with the pixel values of the pixel points at the corresponding positions in the sketch stroke diagram, and taking the fusion result as the pixel values of the pixel points to obtain a second fusion image.
in a second aspect, an embodiment of the present disclosure provides an apparatus for processing video, the apparatus including: an execution unit configured to execute a preset processing operation on a video frame in the acquired original video, wherein the execution unit includes: an extraction unit configured to obtain an edge image based on a gray image corresponding to the video frame; a first fusion unit configured to fuse the edge image and the grayscale image to obtain a first fused image; a second fusion unit configured to fuse the first fusion image and a preset sketch stroke image to obtain a second fusion image; a conversion unit configured to convert the video frame into an LAB mode, resulting in an a-channel and a B-channel of the video frame; a third fusion unit configured to obtain a third fusion image for the video frame based on the second fusion image, the a channel, and the B channel; and an output unit configured to generate a video based on the obtained third fused image, and output the generated video.
In some embodiments, the third fusing unit includes: a processing unit configured to process original values of the a channel and the B channel, respectively, to obtain a first image for the a channel and a second image for the B channel; an acquisition unit configured to obtain an LAB image based on the second fusion image, the first image, and the second image; and an image conversion unit configured to convert the LAB image into an RGB image, the RGB image serving as a third fusion image for the video frame.
In some embodiments, the processing unit is further configured to: processing the original value of the channel A based on a preset first coefficient to obtain a first image; and processing the original value of the B channel based on a preset second coefficient to obtain a second image.
In some embodiments, the above extraction unit is further configured to: carrying out reverse color processing on the gray level image corresponding to the video frame to obtain a reverse color image; and performing edge extraction on the color reflection image to obtain an edge image.
In some embodiments, the second fusion unit is further configured to: and for the pixel points in the first fusion image, in response to the fact that the pixel values of the pixel points are smaller than a preset threshold value, fusing the pixel values of the pixel points with the pixel values of the pixel points at the corresponding positions in the sketch stroke diagram, and taking the fusion result as the pixel values of the pixel points to obtain a second fusion image.
In a third aspect, an embodiment of the present disclosure provides an apparatus, including: one or more processors; a storage device, on which one or more programs are stored, which, when executed by the one or more processors, cause the one or more processors to implement the method as described in any implementation manner of the first aspect.
In a fourth aspect, the disclosed embodiments provide a computer-readable medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method as described in any implementation manner of the first aspect.
According to the method and the device for processing the video, the following processing operations can be executed for each video frame in the acquired original video: 1) obtaining an edge image based on the gray level image corresponding to the video frame; 2) fusing the edge image and the gray level image to obtain a first fused image; 3) fusing the first fused image and the sketch stroke image to obtain a second fused image; 4) converting the video frame into an LAB mode to obtain an A channel and a B channel of the video frame; 5) and obtaining a third fused image aiming at the video frame based on the second fused image, the A channel and the B channel. Then, a video is generated based on the obtained third fused image, and the generated video is output. Therefore, the conversion from the original video to the color pencil drawing style video is realized.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present disclosure may be applied;
FIG. 2 is a flow diagram for one embodiment of a method for processing video, according to the present disclosure;
FIG. 3 is a schematic diagram of a sketch pen-touch diagram of the present disclosure;
FIG. 4 is a schematic diagram of one application scenario of a method for processing video according to the present disclosure;
FIG. 5 is a flow diagram of yet another embodiment of a method for processing video according to the present disclosure;
FIG. 6 is a schematic block diagram illustrating one embodiment of an apparatus for processing video according to the present disclosure;
FIG. 7 is a schematic block diagram of a computer system suitable for use in implementing an electronic device of an embodiment of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 of a method for processing video or an apparatus for processing video to which embodiments of the present disclosure may be applied.
as shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
the user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as video processing software, image processing software, a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting video processing, including but not limited to smart cameras, smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server providing various services, such as a background server providing support for video playing on the terminal devices 101, 102, 103. The backend server may analyze and otherwise process the received data such as video, and feed back the processing result (e.g., video in the style of colored pencil drawing) to the terminal devices 101, 102, and 103.
the server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
It should be noted that the method for processing video provided by the embodiment of the present disclosure may be executed by the terminal devices 101, 102, and 103, or may be executed by the server 105. Accordingly, the apparatus for processing video may be provided in the terminal devices 101, 102, 103, or may be provided in the server 105. This is not limited in this application.
With continued reference to fig. 2, a flow 200 of one embodiment of a method for processing video in accordance with the present disclosure is shown. The method for processing video comprises the following steps:
in step 201, the following processing operation steps 2011-2015 are executed for the video frames in the acquired original video.
In the present embodiment, an executing subject of the method for processing video (e.g., the terminal apparatus 101, 102, 103 or the server 105 shown in fig. 1) may execute the following preset processing operation steps 2011-2015 for each video frame in the acquired original video.
here, when the execution subject is a terminal device, the original video may refer to a video that is locally stored in advance in the terminal device, or may refer to a video that is captured in real time by a video capture device (e.g., a camera) installed in the terminal device. When the execution subject is a server, the original video may refer to a video that is locally stored in advance by the server, or may refer to a video that is received in real time from the terminal device.
In step 2011, an edge image is obtained based on the gray image corresponding to the video frame.
in this embodiment, the execution subject may obtain the edge image based on the grayscale image corresponding to the video frame. Here, the grayscale image corresponding to the video frame may be an image obtained by graying the video frame. The gray scale image is also called a gray scale image. The relationship between white and black is logarithmically divided into several levels, called gray scale. The general gray scale is divided into 256 steps of 0 to 255. The white color was 255, and the black color was 0. In practice, the video frames of the original video may be color images, e.g., RGB images. The execution body may convert the color image into a grayscale image in various ways. The execution subject may then extract edges from the grayscale image in various ways, resulting in an edge image.
in some optional implementations of this embodiment, the step 2011 may be specifically performed as follows:
Firstly, the gray level image corresponding to the video frame is subjected to inverse color processing to obtain an inverse color image.
In this implementation, the execution subject may perform inverse color processing on the grayscale image corresponding to the video frame, so as to obtain an inverse color image. The inverse color processing means that the gray value of the pixel point in the gray image is inverted, that is, the gray value of the pixel point is subtracted from 255 to obtain the gray value of the pixel point after the inverse color processing.
Then, edge extraction is carried out on the reverse color image to obtain an edge image.
In this implementation, the execution subject may perform edge extraction on the inverse color map by using various edge extraction methods, so as to obtain an edge image. As an example, the execution subject may first perform minimum value filtering on the inverse color map to obtain a filtered image. And then, carrying out mean value filtering on the filtered image to obtain a fuzzy image. And finally, fusing the blurred image and the gray image corresponding to the video frame to obtain an edge image.
Step 2012, the edge image and the gray level image are fused to obtain a first fused image.
In this embodiment, the executing entity may fuse the edge image obtained in step 2011 and the grayscale image corresponding to the video frame by using various fusion methods, so as to obtain a first fused image. By fusing the edge image and the gray level image, a first fused image comprising edges and gray level distribution can be obtained, and image shadows and edges are enhanced, so that the generated image is more in line with the pencil drawing style.
And 2013, fusing the first fused image and a preset sketch pen-touch image to obtain a second fused image.
In this embodiment, a sketch stroke map may be stored in the execution body in advance. Here, the brush stroke may refer to a trace left when the brush touches the drawing during drawing. As an example, the sketch pen drawing may be a sketch image drawn manually using a pencil. Lines with different colors are distributed in the sketch pen-touch diagram, and the effect can be shown as figure 3. It should be noted that the lines of the sketch pen-touch diagram in fig. 3 are only schematic, and are not limited to the density, the color shade, the trend, and the like of the lines of the sketch pen-touch diagram. In practice, the density, the color depth, the trend and the like of the lines of the sketch pen-touch diagram can be set according to actual needs.
In this way, the execution subject may adopt various image fusion methods to fuse the first fused image obtained in step 2012 and the sketch stroke map, so as to obtain a second fused image.
In some optional implementations of this embodiment, the step 2013 may be specifically performed as follows:
And for the pixel point in the first fusion image, in response to the fact that the pixel value of the pixel point is smaller than the preset threshold value, fusing the pixel value of the pixel point with the pixel value of the pixel point at the corresponding position in the sketch stroke diagram, and taking the fusion result as the pixel value of the pixel point to obtain a second fusion image.
In this implementation manner, for each pixel point in the first fusion image obtained in step 2012, the executing entity may determine whether a pixel value of the pixel point is smaller than a preset threshold. If the pixel value of the pixel point is smaller than the preset threshold value, the execution main body can fuse the pixel value of the pixel point with the pixel value of the pixel point at the corresponding position in the sketch pen-touch diagram. For example, the pixel values are multiplied by preset weighting coefficients respectively and then added to obtain a fusion result, and the fusion result is used as the pixel value of the pixel point.
Here, the range of the pixel value of the pixel point in the first fused image may be 0 to 255, where 255 denotes white and 0 denotes black. Smaller pixel values indicate darker color for the pixel. Therefore, if the pixel value of a certain pixel point in the first fusion image is smaller than the preset threshold, it indicates that the pixel point may be a pixel point of a shadow part in the first fusion image. In the actual drawing process of the pencil drawing, the shaded part is often represented by dense lines, and the effect is similar to that displayed in a sketch pen-touch diagram. Therefore, the pixel point with the pixel value smaller than the preset threshold value in the first fusion image is fused with the pixel point at the corresponding position in the sketch pen-touch image, so that the shadow part in the obtained second fusion image is closer to the real pencil sketch effect. In practice, the preset threshold may be set according to actual needs.
Step 2014, converting the video frame into an LAB mode to obtain an a channel and a B channel of the video frame.
in this embodiment, the executing entity may convert the video frame into LAB mode. Thereby resulting in the L, a, and B channels of the video frame. Here, the LAB pattern includes an L channel, an a channel, and a B channel. Where L channel represents luminance, a channel represents a range from magenta to green, and B channel represents a range from yellow to blue.
Step 2015, obtaining a third fused image for the video frame based on the second fused image, the a channel and the B channel.
In this embodiment, the executing entity may obtain a third fused image for the video frame based on the second fused image obtained in step 2013 and the a-channel and the B-channel obtained in step 2014. As an example, the executing entity may first take the second fused image obtained in step 2013 as the new L channel. Next, an LAB image is determined based on the new L channel, and the a and B channels of the video frame. The determined LAB image may then be converted into an RGB image, and the converted RGB image may be used as a third fused image.
And 202, generating a video based on the obtained third fusion image, and outputting the generated video.
In this embodiment, the execution subject may obtain a third fused image for each video frame in the original video. And generating a video according to the obtained third fusion images. Here, the generated video is a color pencil drawing style video. Thereafter, the execution body may output the generated video. As an example, when the execution subject is a terminal device, the execution subject may directly output and play a video. When the execution main body is a server, the execution main body can output the video to the terminal equipment so as to play the video by the terminal equipment.
With continued reference to fig. 4, fig. 4 is a schematic diagram of an application scenario of the method for processing video according to the present embodiment. In the application scenario of fig. 4, the terminal device 401 first performs the following processing operations for each video frame in the original video captured in real time: 1) obtaining an edge image based on the gray level image corresponding to the video frame; 2) fusing the edge image and the gray level image to obtain a first fused image; 3) fusing the first fused image and a preset sketch stroke image to obtain a second fused image; 4) converting the video frame into an LAB mode to obtain an A channel and a B channel of the video frame; 5) and obtaining a third fused image aiming at the video frame based on the second fused image, the A channel and the B channel. After that, the terminal apparatus 401 generates a video based on the obtained third fusion image, and outputs the generated video.
The method provided by the above embodiment of the present disclosure first obtains a pencil drawing style image of a video frame based on a gray level image and a sketch pen-touch image of the video frame in an original video. And then, fusing the image in the pencil drawing style with the channel A and the channel B of the video frame to obtain an image in the color pencil drawing style, thereby realizing the conversion from the original video to the video in the color pencil drawing style.
with further reference to fig. 5, a flow 500 of yet another embodiment of a method for processing video is shown. The flow 500 of the method for processing video comprises the following steps:
In step 501, the following processing operations 5011 to 5017 are performed on the video frames in the acquired original video.
In the present embodiment, an executing subject of the method for processing video (e.g., the terminal apparatus 101, 102, 103 or the server 105 shown in FIG. 1) can execute the following preset processing operation steps 5011 to 5017 for each video frame in the acquired original video.
In step 5011, an edge image is obtained based on the grayscale image corresponding to the video frame.
In this embodiment, step 5011 is similar to step 2011 of the embodiment shown in fig. 2, and is not described herein again.
Step 5012, the edge image and the gray level image are fused to obtain a first fused image.
in this embodiment, step 5012 is similar to step 2012 of the embodiment shown in fig. 2, and is not described herein again.
And step 5013, fusing the first fused image and a preset sketch pen-touch image to obtain a second fused image.
In this embodiment, step 5013 is similar to step 2013 of the embodiment shown in fig. 2, and is not described here again.
in step 5014, the video frame is converted into an LAB mode to obtain the channel a and the channel B of the video frame.
In this embodiment, step 5014 is similar to step 2014 of the embodiment shown in fig. 2, and is not described herein again.
in step 5015, the original values of the a channel and the B channel are processed respectively to obtain a first image for the a channel and a second image for the B channel.
In this embodiment, the execution body may process the original values of the a channel and the B channel obtained in step 5014, respectively, to obtain a first image for the a channel and a second image for the B channel. In practice, the colors of the video frames obtained by direct shooting are not the same as the colors of the colored pencil drawings, and therefore, the original values of the a channel and the B channel need to be processed. As an example, the execution subject may perform various processes (e.g., subtracting a preset value) on the original value of the a channel, thereby obtaining a first image. The execution subject may also perform various processing (e.g., subtracting a preset value) on the raw value of the B channel, thereby obtaining a second image.
In some optional implementations of this embodiment, the step 5015 may specifically proceed as follows:
Firstly, processing an original value of an A channel based on a preset first coefficient to obtain a first image.
In this implementation, for each original value in the a-channel, the execution entity may process the original value using a preset first coefficient. For example, the original value is multiplied by a first coefficient, thereby obtaining a first image. Here, the first coefficient may be set by a skilled person according to actual needs. For example, the first coefficient may be 0.98.
and secondly, processing the original value of the B channel based on a preset second coefficient to obtain a second image.
In this implementation, for each original value in the B channel, the executing entity may process the original value using a preset second coefficient, for example, multiply the original value by the second coefficient, thereby obtaining a second image. Here, the second coefficient may be set by a skilled person according to actual needs. For example, the second coefficient may be 0.92. The original values of the channel A and the channel B used for representing colors in the video frame can be processed through the implementation mode, so that the obtained third fused image is closer to the style of a colored pencil drawing.
in step 5016, an LAB image is obtained based on the second fused image, the first image, and the second image.
In this embodiment, the execution subject may obtain an LAB image based on the second fusion image obtained in step 5013, and the first image and the second image obtained in step 5015. As an example, the executing subject may first determine the LAB image with the second fused image as the L-channel, the first image as the a-channel, and the second image as the B-channel.
In step 5017, the LAB image is converted into an RGB image, and the RGB image is used as a third fusion image for the video frame.
in this embodiment, the executing body may convert the LAB image obtained in step 5016 into an RGB image, and use the converted RGB image as the third fusion image for the video frame. As one example, the executing entity may directly use the converted RGB image as the third fused image for the video frame. As another example, the execution subject may further take each pixel value of the converted RGB image as a third fused image after multiplying the resulting image by a preset coefficient. The coefficient may be set according to actual needs, and may be 1.09, for example.
And 502, generating a video based on the obtained third fusion image, and outputting the generated video.
In this embodiment, step 502 is similar to step 202 of the embodiment shown in fig. 2, and is not described herein again.
as can be seen from fig. 5, compared with the embodiment corresponding to fig. 2, the flow 500 of the method for processing a video in this embodiment highlights a process of processing the a-channel and the B-channel of the video frame to obtain the first image and the second image, and obtaining the LAB image based on the second fusion image, the first image and the second image. Therefore, the method described in this embodiment can make the colors in the obtained third fused image more conform to the colors of real color pencil drawings, so that the video of the color pencil drawing style obtained through conversion is more real.
With further reference to fig. 6, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for processing video, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 6, the apparatus 600 for processing video of the present embodiment includes: an execution unit 601 and an output unit 602. Wherein, the execution unit 601 is configured to execute a preset processing operation on a video frame in the acquired original video, wherein the execution unit 601 comprises: an extracting unit 6011 configured to obtain an edge image based on the grayscale image corresponding to the video frame; a first fusion unit 6012 configured to fuse the edge image and the grayscale image to obtain a first fusion image; a second fusion unit 6013 configured to fuse the first fusion image and a preset sketch stroke image to obtain a second fusion image; a conversion unit 6014 configured to convert the video frame into an LAB mode, resulting in an a channel and a B channel of the video frame; a third fusion unit 6015 configured to obtain a third fusion image for the video frame based on the second fusion image, the a channel, and the B channel; the output unit 602 is configured to generate a video based on the obtained third fused image, and output the generated video.
in this embodiment, specific processing of the execution unit 601 and the output unit 602 of the apparatus 600 for processing a video and technical effects brought by the specific processing can refer to related descriptions of step 201 and step 202 in the corresponding embodiment of fig. 2, which are not described herein again.
In some optional implementations of the present embodiment, the third fusing unit 6015 includes: a processing unit (not shown in the figure) configured to process the original values of the a channel and the B channel respectively to obtain a first image for the a channel and a second image for the B channel; an acquisition unit (not shown in the figure) configured to obtain an LAB image based on the second fusion image, the first image, and the second image; an image conversion unit (not shown in the figure) configured to convert the LAB image into an RGB image as a third fusion image for the video frame.
In some optional implementations of this embodiment, the processing unit is further configured to: processing the original value of the channel A based on a preset first coefficient to obtain a first image; and processing the original value of the B channel based on a preset second coefficient to obtain a second image.
In some optional implementations of this embodiment, the extraction unit 6011 is further configured to: carrying out reverse color processing on the gray level image corresponding to the video frame to obtain a reverse color image; and performing edge extraction on the color reflection image to obtain an edge image.
in some optional implementations of the present embodiment, the second fusion unit 6013 is further configured to: and for the pixel points in the first fusion image, in response to the fact that the pixel values of the pixel points are smaller than a preset threshold value, fusing the pixel values of the pixel points with the pixel values of the pixel points at the corresponding positions in the sketch stroke diagram, and taking the fusion result as the pixel values of the pixel points to obtain a second fusion image.
Referring now to fig. 7, a schematic diagram of an electronic device (e.g., the server or terminal device of fig. 1) 700 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 7, electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from storage 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Generally, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 7 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 708, or may be installed from the ROM 702. The computer program, when executed by the processing device 701, performs the above-described functions defined in the methods of embodiments of the present disclosure.
It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: for the video frames in the acquired original video, the following processing operations are performed: obtaining an edge image based on the gray level image corresponding to the video frame; fusing the edge image and the gray level image to obtain a first fused image; fusing the first fused image and a preset sketch stroke image to obtain a second fused image; converting the video frame into an LAB mode to obtain an A channel and a B channel of the video frame; obtaining a third fused image for the video frame based on the second fused image, the channel A and the channel B; and generating a video based on the obtained third fused image, and outputting the generated video.
computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an execution unit and an output unit. Here, the names of these units do not constitute a limitation to the unit itself in some cases, and for example, the output unit may also be described as "a unit that generates a video based on the obtained third fusion image and outputs the generated video".
the foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (12)
1. A method for processing video, comprising:
For the video frames in the acquired original video, the following processing operations are performed: obtaining an edge image based on the gray level image corresponding to the video frame; fusing the edge image and the gray level image to obtain a first fused image; fusing the first fused image and a preset sketch stroke image to obtain a second fused image; converting the video frame into an LAB mode to obtain an A channel and a B channel of the video frame; obtaining a third fused image aiming at the video frame based on the second fused image, the channel A and the channel B;
And generating a video based on the obtained third fused image, and outputting the generated video.
2. The method of claim 1, wherein said deriving a third fused image for the video frame based on the second fused image, the a channel, and the B channel comprises:
Respectively processing the original values of the channel A and the channel B to obtain a first image for the channel A and a second image for the channel B;
Obtaining an LAB image based on the second fusion image, the first image and the second image;
And converting the LAB image into an RGB image, and taking the RGB image as a third fusion image for the video frame.
3. The method of claim 2, wherein the processing raw values of the a-channel and the B-channel, respectively, resulting in a first image for the a-channel and a second image for the B-channel, comprises:
Processing the original value of the channel A based on a preset first coefficient to obtain a first image;
And processing the original value of the B channel based on a preset second coefficient to obtain a second image.
4. the method of claim 1, wherein the obtaining an edge image based on the gray-scale image corresponding to the video frame comprises:
Carrying out reverse color processing on the gray level image corresponding to the video frame to obtain a reverse color image;
And performing edge extraction on the color reflection image to obtain an edge image.
5. The method according to claim 1, wherein the fusing the first fused image and the preset sketch stroke map to obtain a second fused image comprises:
and for the pixel points in the first fusion image, in response to the fact that the pixel values of the pixel points are smaller than a preset threshold value, fusing the pixel values of the pixel points with the pixel values of the pixel points at the corresponding positions in the sketch stroke diagram, and taking the fusion result as the pixel values of the pixel points to obtain a second fusion image.
6. An apparatus for processing video, comprising:
An execution unit configured to execute a preset processing operation on a video frame in the acquired original video, wherein the execution unit includes: an extraction unit configured to obtain an edge image based on a gray image corresponding to the video frame; a first fusion unit configured to fuse the edge image and the grayscale image to obtain a first fusion image; the second fusion unit is configured to fuse the first fusion image and a preset sketch stroke image to obtain a second fusion image; a conversion unit configured to convert the video frame into an LAB mode, resulting in an a-channel and a B-channel of the video frame; a third fusion unit configured to obtain a third fusion image for the video frame based on the second fusion image, the a channel, and the B channel;
and an output unit configured to generate a video based on the obtained third fused image, and output the generated video.
7. The apparatus of claim 6, wherein the third fusion unit comprises:
A processing unit configured to process original values of the a channel and the B channel, respectively, resulting in a first image for the a channel and a second image for the B channel;
An acquisition unit configured to obtain an LAB image based on the second fusion image, the first image, and the second image;
An image conversion unit configured to convert the LAB image into an RGB image, the RGB image being a third fused image for the video frame.
8. The apparatus of claim 7, wherein the processing unit is further configured to:
Processing the original value of the channel A based on a preset first coefficient to obtain a first image;
And processing the original value of the B channel based on a preset second coefficient to obtain a second image.
9. the apparatus of claim 6, wherein the extraction unit is further configured to:
Carrying out reverse color processing on the gray level image corresponding to the video frame to obtain a reverse color image;
and performing edge extraction on the color reflection image to obtain an edge image.
10. the apparatus of claim 6, wherein the second fusion unit is further configured to:
And for the pixel points in the first fusion image, in response to the fact that the pixel values of the pixel points are smaller than a preset threshold value, fusing the pixel values of the pixel points with the pixel values of the pixel points at the corresponding positions in the sketch stroke diagram, and taking the fusion result as the pixel values of the pixel points to obtain a second fusion image.
11. an apparatus, comprising:
One or more processors;
A storage device having one or more programs stored thereon,
When executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910919210.3A CN110555799A (en) | 2019-09-26 | 2019-09-26 | Method and apparatus for processing video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910919210.3A CN110555799A (en) | 2019-09-26 | 2019-09-26 | Method and apparatus for processing video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110555799A true CN110555799A (en) | 2019-12-10 |
Family
ID=68741690
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910919210.3A Pending CN110555799A (en) | 2019-09-26 | 2019-09-26 | Method and apparatus for processing video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110555799A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111524166A (en) * | 2020-04-22 | 2020-08-11 | 北京百度网讯科技有限公司 | Video frame processing method and device |
| CN112241941A (en) * | 2020-10-20 | 2021-01-19 | 北京字跳网络技术有限公司 | Method, device, equipment and computer readable medium for acquiring image |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101340510A (en) * | 2008-08-07 | 2009-01-07 | 中兴通讯股份有限公司 | Method for video enhancement and apparatus thereof |
| US20090219298A1 (en) * | 2008-02-28 | 2009-09-03 | Befun Bilgi Teknolojileri A.S. | Method and system for generating online cartoon outputs |
| CN102496180A (en) * | 2011-12-15 | 2012-06-13 | 李大锦 | Method for automatically generating wash landscape painting image |
| CN102547063A (en) * | 2012-02-08 | 2012-07-04 | 南京航空航天大学 | Natural sense color fusion method based on color contrast enhancement |
| CN103021002A (en) * | 2011-09-27 | 2013-04-03 | 康佳集团股份有限公司 | Colorful sketch image generating method |
| CN103685858A (en) * | 2012-08-31 | 2014-03-26 | 北京三星通信技术研究有限公司 | Real-time video processing method and equipment |
| CN104915975A (en) * | 2015-06-03 | 2015-09-16 | 厦门美图之家科技有限公司 | Image processing method and system for simulating crayon colored drawing |
| CN105096358A (en) * | 2015-08-05 | 2015-11-25 | 云南大学 | Line enhanced simulation method for pyrography artistic effect |
| CN105374007A (en) * | 2015-12-02 | 2016-03-02 | 华侨大学 | Generation method and generation device of pencil drawing fusing skeleton strokes and textural features |
| TW201640888A (en) * | 2015-02-04 | 2016-11-16 | 騰訊科技（深圳）有限公司 | Method, device, and terminal for generating video |
| CN107438178A (en) * | 2016-05-25 | 2017-12-05 | 掌赢信息科技（上海）有限公司 | A kind of image color antidote and electronic equipment |
| CN109903250A (en) * | 2019-02-25 | 2019-06-18 | 大连海事大学 | Underwater picture clearness processing method based on multi-scale gradient domain contrast stretching |
| CN110070495A (en) * | 2019-02-20 | 2019-07-30 | 北京字节跳动网络技术有限公司 | Processing method, device and the electronic equipment of image |
-
2019
- 2019-09-26 CN CN201910919210.3A patent/CN110555799A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090219298A1 (en) * | 2008-02-28 | 2009-09-03 | Befun Bilgi Teknolojileri A.S. | Method and system for generating online cartoon outputs |
| CN101340510A (en) * | 2008-08-07 | 2009-01-07 | 中兴通讯股份有限公司 | Method for video enhancement and apparatus thereof |
| CN103021002A (en) * | 2011-09-27 | 2013-04-03 | 康佳集团股份有限公司 | Colorful sketch image generating method |
| CN102496180A (en) * | 2011-12-15 | 2012-06-13 | 李大锦 | Method for automatically generating wash landscape painting image |
| CN102547063A (en) * | 2012-02-08 | 2012-07-04 | 南京航空航天大学 | Natural sense color fusion method based on color contrast enhancement |
| CN103685858A (en) * | 2012-08-31 | 2014-03-26 | 北京三星通信技术研究有限公司 | Real-time video processing method and equipment |
| TW201640888A (en) * | 2015-02-04 | 2016-11-16 | 騰訊科技（深圳）有限公司 | Method, device, and terminal for generating video |
| CN104915975A (en) * | 2015-06-03 | 2015-09-16 | 厦门美图之家科技有限公司 | Image processing method and system for simulating crayon colored drawing |
| CN105096358A (en) * | 2015-08-05 | 2015-11-25 | 云南大学 | Line enhanced simulation method for pyrography artistic effect |
| CN105374007A (en) * | 2015-12-02 | 2016-03-02 | 华侨大学 | Generation method and generation device of pencil drawing fusing skeleton strokes and textural features |
| CN107438178A (en) * | 2016-05-25 | 2017-12-05 | 掌赢信息科技（上海）有限公司 | A kind of image color antidote and electronic equipment |
| CN110070495A (en) * | 2019-02-20 | 2019-07-30 | 北京字节跳动网络技术有限公司 | Processing method, device and the electronic equipment of image |
| CN109903250A (en) * | 2019-02-25 | 2019-06-18 | 大连海事大学 | Underwater picture clearness processing method based on multi-scale gradient domain contrast stretching |
Non-Patent Citations (4)
| Title |
|---|
| WENHUA QIAN ET AL.: "Aesthetic art simulation for embroidery style", 《MULTIMED TOOLS APPL》 * |
| XIANLIN ZHANG ET AL.: "Photo-to-Sketch Transformation in a Complex Background", 《IEEE ACCESS》 * |
| 冯捷等: "基于灰度合成的图像素描效果生成算法", 《浙江工业大学学报》 * |
| 张江红等: "基于FPGA 的铅笔画绘制算法结构设计及优化", 《工程图学学报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111524166A (en) * | 2020-04-22 | 2020-08-11 | 北京百度网讯科技有限公司 | Video frame processing method and device |
| CN111524166B (en) * | 2020-04-22 | 2023-06-30 | 北京百度网讯科技有限公司 | Video frame processing method and device |
| US11748895B2 (en) | 2020-04-22 | 2023-09-05 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for processing video frame |
| CN112241941A (en) * | 2020-10-20 | 2021-01-19 | 北京字跳网络技术有限公司 | Method, device, equipment and computer readable medium for acquiring image |
| CN112241941B (en) * | 2020-10-20 | 2024-03-22 | 北京字跳网络技术有限公司 | Method, apparatus, device and computer readable medium for acquiring image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109344752B (en) | Method and apparatus for processing mouth image | |
| US11216924B2 (en) | Method and apparatus for processing image | |
| CN110516678B (en) | Image processing method and device | |
| CN107622504B (en) | Method and device for processing pictures | |
| CN111476871A (en) | Method and apparatus for generating video | |
| CN110059623B (en) | Method and apparatus for generating information | |
| CN110211030B (en) | Image generation method and device | |
| CN111080595A (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN110349107B (en) | Image enhancement method, device, electronic equipment and storage medium | |
| CN109102484B (en) | Method and apparatus for processing image | |
| JP7261732B2 (en) | Method and apparatus for determining character color | |
| CN113962859A (en) | Panorama generation method, device, equipment and medium | |
| CN112419179A (en) | Method, device, equipment and computer readable medium for repairing image | |
| CN110555799A (en) | Method and apparatus for processing video | |
| CN112819691B (en) | Image processing method, device, equipment and readable storage medium | |
| CN110636331B (en) | Method and apparatus for processing video | |
| CN114331823A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113610720A (en) | Video denoising method and device, computer readable medium and electronic device | |
| CN111757100B (en) | Method and device for determining camera motion variation, electronic equipment and medium | |
| CN110399802B (en) | Method, apparatus, medium, and electronic device for processing eye brightness of face image | |
| CN110097520B (en) | Image processing method and device | |
| CN112348910A (en) | Method, device, equipment and computer readable medium for acquiring image | |
| CN111784726A (en) | Image matting method and device | |
| CN110599437A (en) | Method and apparatus for processing video | |
| CN113256785B (en) | Image processing method, apparatus, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |