CN110149554B - Video image processing method and device, electronic equipment and storage medium - Google Patents
Video image processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN110149554B CN110149554B CN201910472833.0A CN201910472833A CN110149554B CN 110149554 B CN110149554 B CN 110149554B CN 201910472833 A CN201910472833 A CN 201910472833A CN 110149554 B CN110149554 B CN 110149554B
- Authority
- CN
- China
- Prior art keywords
- video image
- processed
- pass filter
- target
- low
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application discloses a method and a device for processing video images, electronic equipment and a storage medium. The method comprises the steps of obtaining a video image to be processed, which is acquired by an image acquisition element; performing fuzzification processing on the video image to be processed by adopting a target low-pass filter, wherein the fuzzification processing is used for reducing high-frequency components in the video image to be processed; coding and compressing the video image to be processed after the fuzzification processing to obtain a target video image; and transmitting the target video image. According to the method, the video image data is fuzzified before being input to the encoder, so that high-frequency components in the video image are reduced, the problem of uneven data distribution of the processed video image when the processed video image is encoded by the encoder can be solved, mosaic and block effects are reduced in the playing process after the video is integrally encoded, and the playing fluency is improved.
Description
Technical Field
The present application relates to the field of video image processing technologies, and in particular, to a method and an apparatus for video image processing, an electronic device, and a storage medium.
Background
With the development of internet technology and electronic device technology, more and more electronic devices are equipped with cameras with higher pixels, and the camera pixels of the electronic devices are higher and higher, and the definition of recorded videos is also higher and higher. Although the update frequency of the camera of the electronic device is high, the bandwidth update speed of the channel for video transmission is slow, so that the transmission bandwidth is limited, thereby limiting the code rate of the video transmitted by the electronic device, and under the condition that the code rate is limited, the conditions of mosaic and block effect are easy to occur under the current video coding frame, and the playing effect of the video image is greatly influenced.
Disclosure of Invention
In view of the above problems, the present application provides a method, an apparatus, an electronic device, and a storage medium for video image processing to improve the above problems.
In a first aspect, an embodiment of the present application provides a method for processing a video image, which is applied to an electronic device, and the method includes: acquiring a to-be-processed video image acquired by an image acquisition element; performing fuzzification processing on the video image to be processed by adopting a target low-pass filter, wherein the fuzzification processing is used for reducing high-frequency components in the video image to be processed; coding and compressing the video image to be processed after the fuzzification processing to obtain a target video image; and transmitting the target video image.
In a second aspect, an embodiment of the present application provides an apparatus for video image processing, which is executed in an electronic device, and includes: the video image acquisition module is used for acquiring a to-be-processed video image acquired by the image acquisition element; the video image processing module is used for performing fuzzification processing on the video image to be processed by adopting a target low-pass filter, wherein the fuzzification processing is used for reducing high-frequency components of the video image to be processed; the video image coding module is used for coding and compressing the video image to be processed after the fuzzification processing is carried out, so as to obtain a target video image; and the video image output module is used for transmitting the target video image.
In a third aspect, an embodiment of the present application provides an electronic device, including one or more processors and a memory; one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of the first aspect described above.
In a fourth aspect, the present application provides a computer-readable storage medium, in which a program code is stored, where the program code executes the method of the first aspect.
According to the video image processing method, the video image processing device, the electronic equipment and the storage medium, the video image to be processed acquired by the image acquisition element is acquired, then the video image to be processed is subjected to fuzzification processing by adopting the target low-pass filter before the video image to be processed is subjected to coding compression, so that the high-frequency component of the video image to be processed is reduced, then the video image to be processed after the fuzzification processing is subjected to coding compression, the target video image is obtained, and then the target video image is transmitted. Therefore, before the video image is input to the encoder, the high-frequency filtering is carried out on the video image through the low-pass filter to realize the fuzzification processing, so that the high-frequency component of the video image obtained by the encoder and subjected to the coding is reduced through the filtering processing, the problem of uneven data distribution of the video image when the video image is coded by the encoder is solved, the mosaic and block effects are reduced in the playing process after the video is integrally coded, the playing effect is consistent, and the playing fluency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 shows a flowchart of a method for processing a video image according to an embodiment of the present application.
Fig. 2 shows a method flowchart of a method for video image processing according to another embodiment of the present application.
Fig. 3 shows a flowchart of the method of step S230 in fig. 2.
Fig. 4 shows a flowchart of the method of step S233 in fig. 3.
Fig. 5 is a block diagram illustrating an exemplary processing flow of a method for processing a video image according to an embodiment of the present application.
Fig. 6 shows a block diagram of a video image processing apparatus according to an embodiment of the present application.
Fig. 7 shows a block diagram of an electronic device for executing a method of video image processing according to an embodiment of the present application.
Fig. 8 is a storage unit for storing or carrying program codes for implementing a method of video image processing according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
With the development of science and technology, the pixels of cameras of electronic devices such as mobile phones and tablet computers are higher and higher, and the definition of recorded video images is higher and higher. As a mode, generally, an electronic device is provided with two cameras, and an electronic device with three cameras and even four cameras starts to emerge endlessly, and moreover, the updating frequency of the cameras of the electronic device is very high, so that a shot video image is clearer, and the use requirements of users are greatly met.
However, the inventor finds that although the update frequency of the camera of the electronic device is high, the bandwidth update speed of the transmission channel is slow, so that the transmission bandwidth is limited within a certain time, and the code rate of the video image data transmitted by the electronic device is limited, thereby causing conditions such as mosaic, block effect or blocking during playing of the video image, and affecting the viewing experience of a user.
Therefore, the inventor proposes a method, an apparatus, an electronic device, and a storage medium for video image processing to improve the occurrence of mosaic and stutter of video images in the present application.
Embodiments of the present application will be described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a method for processing a video image according to an embodiment of the present application is applied to an electronic device, and the method includes:
step S110: and acquiring a to-be-processed video image acquired by the image acquisition element.
The image capturing element refers to a camera configured in the electronic device for capturing video image data. In the embodiment of the application, the video image to be processed is an image which needs to be subjected to video coding after being acquired by the image acquisition element. Moreover, the processed video image may be a video image in which a part is clear and another part is fuzzy due to nonuniform data size distribution possibly existing after the intra-coding compression of the same frame of video image, or may also be a video image in which a part is clear and another part is fuzzy due to nonuniform data size distribution possibly occurring after the coding compression of a plurality of adjacent frames of video images, and other images have a large mosaic, optionally, the video image to be processed may cause the problems of poor playing effect, unsmooth playing and the like of the video image.
As a mode, the camera can acquire video images in real time after being started, and the electronic equipment can continuously cache the video images acquired by the camera in real time after detecting that the camera is started. It should be noted that, in the embodiment of the present application, a video image acquired by an electronic device in real time by a camera may have situations of blocking, mosaic, and the like due to limitation of a transmission code rate, so that smoothness of playing of the video image is affected, it can be understood that the video image acquired by the camera in real time can be used as a video image to be processed, and the video image is displayed to be provided for a user to browse in real time after the video image to be processed is processed, so that the electronic device can acquire the video image to be processed acquired by an image acquisition element by continuously caching the video image acquired by the camera in real time.
Step S120: and performing fuzzification processing on the video image to be processed by adopting a target low-pass filter, wherein the fuzzification processing is used for reducing high-frequency components in the video image to be processed.
It will be appreciated that images typically include high frequency components as well as low frequency components. The high frequency component (high frequency signal) corresponds to a portion of the image where the change is severe, that is, an edge (contour) and a detailed portion of the image. While the low frequency components (low frequency signals) represent areas of the image where the brightness or grey value changes slowly, i.e. large flat areas of the image, describe the main part of the image.
As one way, for the video image to be processed, which is shot by the camera according to different image configuration parameters, different target low-pass filters may be used to perform blurring processing on the video image. In this case, after detecting that the camera is started, the electronic device may acquire the current image configuration parameter, so as to perform blurring processing on the video image to be processed according to the target low-pass filter corresponding to the current image configuration parameter, thereby reducing the high-frequency component in the video image to be processed.
Step S130: and coding and compressing the video image to be processed after the fuzzification processing to obtain a target video image.
The high-frequency components in the video image are reduced by the video image to be processed after the fuzzification processing, and the video image to be processed after the fuzzification processing is coded and compressed, so that the problem of uneven data size distribution after the video image is coded and compressed can be reduced, and a target video image with more uniform data size distribution can be obtained.
Step S140: and transmitting the target video image.
According to the video image processing method, the video image processing device, the electronic equipment and the storage medium, the video image to be processed acquired by the image acquisition element is acquired, then the video image to be processed is subjected to fuzzification processing by adopting the target low-pass filter before the video image to be processed is subjected to coding compression, so that the high-frequency component of the video image to be processed is reduced, then the video image to be processed after the fuzzification processing is subjected to coding compression, the target video image is obtained, and then the target video image is transmitted. Therefore, before the video image is input to the encoder, the high-frequency filtering convolution is carried out on the video image through the low-pass filter so as to realize the fuzzification processing, so that the high-frequency component of the video image obtained by the encoder and subjected to the encoding is reduced through the filtering processing, the problem of uneven data distribution of the video image when the video image is encoded by the encoder is solved, the mosaic and block effects are reduced in the playing process after the video is integrally encoded, the playing effect is consistent, and the playing fluency is improved.
Referring to fig. 2, a method for processing a video image according to an embodiment of the present application is applied to an electronic device, and the method includes:
step S210: and acquiring a to-be-processed video image acquired by the image acquisition element.
Step S220: and acquiring image configuration parameters of the video image to be processed.
The image configuration parameters may include a resolution and a transmission rate of the video image to be processed (more or other parameters may also be included, which are not limited herein, and for example, a frame rate may also be included). As a way, in order to facilitate adjustment of a playing effect and fluency in a transmission process of a to-be-processed video image, the electronic device in the embodiment of the application may obtain an image configuration parameter of the to-be-processed video image when receiving a video recording function called by a related application program.
As one mode, the electronic device may pre-configure a plurality of video image resolutions for a user to select, and different video image resolutions may be adapted to the corresponding transmission code rate, for example, the electronic device may automatically adapt to the transmission code rate according to a file size of a captured video image. In this case, when the electronic device detects that the camera is started, the electronic device may obtain the resolution of the video image selected by the current user, and further obtain the transmission code rate of the video image according to the resolution of the video image, thereby obtaining the resolution and the code rate of the video image to be processed.
Alternatively, the resolution and the code rate of the video image may be changed in real time according to the shooting distance of the camera, the difference of the shot objects, or the difference of the shooting angles, for example, the user may adjust the resolution of the shot video image in real time at the camera shooting interface. In this case, the electronic device may monitor the moving distance and the moving trend of the camera in real time after detecting that the camera is started, and simultaneously monitor whether there is an image configuration parameter adjustment instruction acting on the electronic device, so as to obtain the image configuration parameters of the video image to be processed in real time. The electronic equipment can detect the moving distance and the moving trend of the camera by detecting the focal length.
For example, in a specific application scenario, when a sunset is shot by the electronic device, if a scene in which the sunset gradually disappears is to be shot, a specific disappearance process of the sunset may be shot by adjusting a resolution of the camera, optionally, the shot image may be enlarged before the sunset disappears, and the shot image may be reduced before the sunset disappears to highlight the scene in which the sunset gradually disappears.
Step S230: acquiring the target low-pass filter based on the image configuration parameters.
It can be understood that, with the difference of the object shot by the camera and the difference of the shooting distance or the shooting angle, the image quality and the image fluency of the video image to be processed are different, so that the data volume distribution of the video image to be processed is more uniform, and further the fluency of the video image playing is improved, the target low-pass filter can be obtained based on the image configuration parameters, so as to facilitate the targeted adjustment. The target low-pass filter is used for performing low-pass filtering on the video image to be processed, reducing high-frequency components of the video image to be processed, and keeping low-frequency information, and the target low-pass filters corresponding to different image configuration parameters may be different or the same (if the target low-pass filters corresponding to different image configuration parameters are within a set threshold range).
Referring to fig. 3, as an alternative, step S230 may include:
step S231: and calculating the initial code rate of the video image to be processed based on the resolution and the transmission code rate.
The initial code rate represents the number of data bits transmitted in unit time of a real-time video image to be processed shot by the camera. As one way, when the resolution and the transmission code rate of the video image to be processed are obtained, the initial code rate of the video image to be processed may be calculated based on the resolution and the transmission code rate according to the real-time shooting data of the video image shot by the camera.
In one implementation, the video image to be processed is calculated from the product of the initial bitrate, the number of bits in the input format of the video image data captured by the camera (i.e. the transmission bitrate can be understood as the transmission bitrate), the resolution and the frame rate. For example, for a 720P video image, the frame rate is 30 frames/second, the data given by the camera is in YUV420 format, and one pixel occupies 12 bits, then the initial code rate can be expressed as: 12 x 720 x 1280 x 30 bits.
Step S232: and acquiring the target code rate of the video image to be processed.
As one mode, the electronic device in this embodiment of the application may control the camera to perform image acquisition based on a required resolution of the video image (for example, a preset resolution selected by a user for reading, or a resolution adjusted in the shooting process acquired in real time). The target bitrate refers to that the electronic device is adapted according to the required resolution of the video image of the told camera, optionally, the target bitrate may be specified according to the actual requirement, for example, the target bitrate of the video image with the resolution of 720P, which is listed in step S231, may be specified to be 2Mbps, which is not limited herein.
Step S233: and acquiring the target low-pass filter according to the ratio of the initial code rate to the target code rate.
The electronic device provided by the application can provide the required resolution of the video image of the camera and the required coding format and code rate requirement of the encoder to a preset filter bank, and the filter bank automatically calculates the corresponding filter template (namely the target low-pass filter). For example, the ratio K of the initial bitrate to the target bitrate of the video image as illustrated in the foregoing steps S231 and S232 is: dividing the original code rate by the target code rate, wherein the original code rate is equal to the number of bits of the input format multiplied by the resolution multiplied by the frame rate, and calculating K to 2.
As one way, the filter bank may obtain the target low-pass filter according to a ratio of the initial code rate to the target code rate, which is specifically described as follows:
as one way, referring to fig. 4, step S233 may include:
step S2331: and matching the ratio of the initial code rate to the target code rate with a plurality of preset threshold ranges.
As one mode, multiple threshold ranges of the ratio of the initial code rate to the target code rate may be preset according to historical calculation data, and target low-pass filters corresponding to different threshold ranges are different, so as to perform corresponding low-pass filtering processing on images with different qualities in the video image. For example, the ratio of the initial code rate to the target code rate may be set to be in threshold range intervals such as (0, 2), (2, 5) and (5, + infinity), and then k may correspond to the low-pass filter a when k belongs to (0, 2), the low-pass filter B when k belongs to (2, 5) and the low-pass filter C when k belongs to (5, + infinity), wherein the low-pass filters A, B, C are respectively different target low-pass filters, for example, as the boundary value of a larger range of the threshold range increases, the effect of the corresponding low-pass filter on filtering the high-frequency component decreases, in this case, the effect of the low-pass filter C, the low-pass filter B and the low-pass filter a on filtering the high-frequency component increases sequentially, it should be noted that the numerical interval here is not absolute, by way of example only, and not by way of limitation.
It is understood that the larger the k value is, the less blurred (i.e. the clearer) the corresponding low-pass filter processed video image is, and that when k is sufficiently large, it indicates that the video image is less prone to mosaic and blocking after encoding, and the low-pass filter may not be needed.
Step S2332: and taking the low-pass filter corresponding to the threshold range matched with the ratio as a target low-pass filter.
As one way, the video image to be processed in the embodiment of the present application may include multiple frames of video images, in which case, it may be determined whether there is an abnormal variation between a current frame video image to be processed and a previous frame video image in the video image to be processed, where the abnormal variation refers to that the content of the current frame video image to be processed and the previous frame video image changes more than a target condition, and optionally, the content includes, but is not limited to, a data amount distribution of the video images (e.g., whether there is a severe sharpening phenomenon between the current frame video image and the previous frame video image, etc.), image display content, and the like.
In one implementation, if there is no abnormal variation, the first low-pass filter corresponding to the threshold range matching the ratio may be used as the target low-pass filter. In another implementation manner, if there is an abnormal variation, the second low-pass filter may be obtained as the target low-pass filter, where it should be noted that the second low-pass filter in the embodiment of the present application has a higher capability of filtering high-frequency components than the first low-pass filter.
For example, in a specific application scenario, a threshold range matched with a ratio between an initial bitrate and a target bitrate corresponding to multiple frames of video images in a filter bank may be set in a step-like manner, optionally, a step region may be set into a plurality of levels, and a level capable of filtering more high-frequency components may give a target low-pass filter with a higher number of taps (optionally, the low-pass filter here may be a two-dimensional low-pass filter).
Specifically, if it is detected that there is no abnormal change between the current frame video image to be processed and the previous frame video image, the first low-pass filter corresponding to the threshold range matched with the ratio may be directly used as the target low-pass filter, for example, the threshold range of the ratio preset according to the level from high to low (which may be understood as the capability of filtering out high-frequency components in the video image) is: if abnormal variation between the current frame video image and the previous frame video image to be processed is detected, the low-pass filter with a level lower than (2, 5) may be selected as the second low-pass filter, for example, the low-pass filter with the threshold range of (0, 2] may be selected as the second low-pass filter, and in this case, the second low-pass filter may be selected as the target low-pass filter.
As another mode, when the ratio of the initial bitrate to the target bitrate meets a certain condition, or it can be understood that the ratio is large enough, the video image to be processed is relatively clear, and can be directly input into the encoder to be encoded and compressed without low-pass filtering, and then output, so as to save the time for processing the video image. In this case, a specific threshold of the ratio of the initial bitrate to the target bitrate may be set, so that when the ratio in the video image to be processed captured by the detection camera reaches the specific threshold, the video image to be processed may be directly encoded, and it should be noted that in this case, the image quality of the video image to be processed may not be affected by direct encoding without performing low-pass filtering. The specific value of the designated threshold is not limited, and may be set according to the actual situation.
In an implementation manner, the electronic device may detect whether the ratio of the initial code rate to the target code rate matches a specified threshold in real time, optionally, if it is detected that the ratio of the initial code rate to the target code rate matches the specified threshold, the obtaining of the target low-pass filter may be cancelled, so as to cancel the blurring process before the encoding compression, which may shorten the processing flow of the video image and improve the user experience.
Step S240: and performing convolution operation on the video image to be processed according to the target low-pass filter.
The convolution operation in the embodiment of the application is to perform blurring processing on the video image to be processed.
It should be noted that, in order to reduce the high frequency component in the video image to be processed and retain the low frequency information, thereby improving the image quality of the video image and the overall playing effect (e.g., smoothness of playing, etc.), the obtained target low-pass filter may be used to perform convolution operation on the video image to be processed, so that the video image obtained by the encoder is a video image subjected to filtering processing, and thus the data size distribution of the encoded video image is more uniform.
As an implementation manner, assuming that an original video image to be processed is S, a video image after being subjected to convolution processing is S ', and a target low-pass filter LF is a two-dimensional low-pass filter with n taps, where n is determined by a filter bank and represents the number of taps of the low-pass filter (which can also be understood as a coefficient of the low-pass filter), the original video image to be processed may be subjected to convolution operation by S × LF — S', where × is convolution.
For example, in a specific application scenario, the target low-pass filter (LF) is assumed to be a three-tap low-pass filter:
then the pixel value at the (m, n) position in the image S' after the blurring process can be obtained:
where (m, n) is the coordinate of a certain point on the image S' after the blurring process.
Step S250: and coding and compressing the video image to be processed after the fuzzification processing to obtain a target video image.
As a mode, the video image to be processed after the fuzzification processing may be encoded and compressed based on the transmission code rate, so as to obtain the target video image. It can be understood that, because the template of the low-pass filter is calculated according to the resolution ratio transmitted to the camera and the code rate transmitted to the encoder, the problem of uneven distribution of data quantity can be reduced when the video image after fuzzification processing is input into the encoder for encoding, the problem of uneven distribution of image data among video frames can be avoided, and further after the video image is integrally encoded and compressed, the playing is smoother, the phenomena of mosaic reduction, block effect reduction and the like are realized, and the user experience is improved.
Step S260: and transmitting the target video image.
The following describes the present embodiment with reference to fig. 5.
Fig. 5 is a block diagram illustrating an exemplary processing flow of a method for processing a video image according to an embodiment of the present application. When the electronic device receives the configuration parameters called by an upper layer (for example, an application program that needs to call a video recording function), the electronic device sends the resolution of the video image that needs to be provided by the camera (for example, the resolution of the video image in fig. 5) to the camera, sends the required bitrate (for example, the required bitrate in fig. 5) and the compression format of the video image to the encoder, and simultaneously provides the resolution of the video image and the required transmission bitrate to the filter bank, so that the filter bank provides the filtering template (i.e., the low-pass filter) according to the data.
As a mode, after a camera acquires a video image, in order to reduce high frequency components in the video image, before the video image is sent to an encoder for encoding and compression, a low pass filter provided by a filter bank may be used to perform blurring processing on the video image to be processed, that is, high frequency components in the video image are uniformly reduced or even removed, low frequency information is retained, the video image to be processed after blurring processing is sent to the encoder, the encoder performs encoding and compression on the video image to be processed after blurring processing according to a received required bit rate and a received compression format to obtain a target video image, and then the target image is output to a transmittable file through a bit stream. The specific process of performing the blurring processing on the video image to be processed may refer to the description in the foregoing embodiments, and is not described herein again.
By carrying out fuzzy filtering processing on the image to be processed, the probability of data distribution strategy errors in the video coding process can be effectively reduced, and the video playing effect after coding is smoother.
It should be noted that, as a mode, in order to ensure that the playing effect of the video after encoding is smoother and at the same time still enable the video to have higher definition during playing, the relevant parameters of the high-frequency components filtered out in the blurring processing process may also be stored, for example, the areas where the high-frequency components of each frame of image exist. Therefore, when the coded video image is transmitted, the filtered related parameters of the high-frequency component are transmitted at the same time, so that the display effect of the video can be enhanced based on the related parameters of the high-frequency component when the video is played, for example, the high-frequency component of each frame in the video image is improved, and the definition is further improved. In addition, because the video enhancement is performed on each frame of image based on the related parameters of the high-frequency components filtered out from each frame of video image, the definition of the video image can be improved while the video image is played in the flow, and the video can better restore the actual display effect during shooting.
According to the video image processing method, before video image data are input to the encoder, the video image is subjected to blurring processing through convolution of the low-pass filter and the video image, so that the input video image obtained by the encoder is subjected to filtering processing, the filter bank can dynamically adapt to the provided target low-tube filter according to the content obtained by the camera, flexibility is improved, then encoding is performed, the problem that the data size distribution of the processed video image is uneven when the processed video image is encoded by the encoder can be solved, further, after the video is integrally encoded, the effect difference is not large when the processed video image is played, mosaic and block effects are reduced, the playing effect is consistent, and the playing smoothness is improved.
Referring to fig. 6, an apparatus 400 for video image processing according to an embodiment of the present application is executed on an electronic device, where the apparatus 400 includes:
and a video image obtaining module 410, configured to obtain a to-be-processed video image collected by the image collecting element.
The video image processing module 420 is configured to perform blurring processing on the video image to be processed by using a target low-pass filter, where the blurring processing is used to reduce high-frequency components of the video image to be processed.
As one mode, the video image processing module 420 may be configured to obtain an image configuration parameter of the video image to be processed, obtain the target low-pass filter based on the image configuration parameter, and perform convolution operation on the video image to be processed according to the target low-pass filter.
Optionally, the image configuration parameters in this embodiment may include a resolution and a transmission rate of the video image to be processed, and in this manner, the video image processing module 420 may be specifically configured to calculate an initial rate of the video image to be processed based on the resolution and the transmission rate, obtain a target rate of the video image to be processed, and then obtain the target low-pass filter according to a ratio of the initial rate to the target rate.
As a mode, a ratio of the initial code rate to the target code rate may be matched with a plurality of preset threshold ranges, and then a low-pass filter corresponding to the threshold range matched with the ratio may be used as the target low-pass filter.
Optionally, the video image to be processed in this embodiment of the application may include multiple video images, and in an implementation manner, it may be detected whether there is an abnormal change between a current frame video image to be processed and a previous frame video image in the video image to be processed, where the abnormal change is that a content change of the current frame video image to be processed and the previous frame video image is greater than a target condition. As one way, if not, the first low-pass filter corresponding to the threshold range matching the ratio may be used as the target low-pass filter; alternatively, if the low-pass filter exists, a second low-pass filter may be obtained as the target low-pass filter, wherein the second low-pass filter has a higher capability of filtering out high-frequency components than the first low-pass filter.
In another implementation, if the ratio of the initial code rate to the target code rate matches a specified threshold, the target low-pass filter may be cancelled to cancel the blurring process before the encoding compression.
And the video image coding module 430 is configured to code and compress the video image to be processed after the blurring processing is performed, so as to obtain a target video image.
As one way, the video image encoding module 430 may be configured to perform encoding and compression on the to-be-processed video image after the blurring processing based on the transmission code rate to obtain a target video image.
And a video image output module 440, configured to transmit the target video image.
It should be noted that the device embodiment and the method embodiment in the present application correspond to each other, and specific principles in the device embodiment may refer to the contents in the method embodiment, which is not described herein again.
An electronic device provided by the present application will be described with reference to fig. 7.
Referring to fig. 7, based on the above-mentioned method and apparatus for processing a video image, another electronic device 100 capable of performing the above-mentioned method for processing a video image is provided in the embodiments of the present application. The electronic device 100 includes one or more processors 102 (only one shown), a memory 104, a network module 106, and an image acquisition element 108 coupled to each other. The memory 104 stores therein a program that can execute the contents of the foregoing embodiments, and the processor 102 can execute the program stored in the memory 104, where the memory 104 includes the apparatus 400 described in the foregoing embodiments.
The Memory 104 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 104 may be used to store instructions, programs, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, a video image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The data storage area may also store data created by the electronic device 100 during use (e.g., phone book, audio-video data, chat log data), and the like.
The network module 106 is configured to receive and transmit electromagnetic waves, and implement interconversion between the electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, for example, an audio playing device. The network module 106 may include various existing circuit elements for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a Subscriber Identity Module (SIM) card, memory, and so forth. The network module 106 may communicate with various networks, such as the internet, an intranet, a wireless network, or with other devices via a wireless network. The wireless network may comprise a cellular telephone network, a wireless local area network, or a metropolitan area network. For example, the network module 106 may interact with a base station.
The image capturing element 108 is used for capturing video image data, including photos, videos, and the like, captured by a camera of the electronic device 100. Image capture element 108 sends the captured video image data to processor 102 for processing by processor 102.
Referring to fig. 8, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable medium 500 has stored therein a program code that can be called by a processor to execute the method described in the above-described method embodiments.
The computer-readable storage medium 500 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 500 includes a non-volatile computer-readable storage medium. The computer readable storage medium 500 has storage space for program code 510 for performing any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 510 may be compressed, for example, in a suitable form.
According to the video image processing method, the video image processing device, the electronic equipment and the storage medium, the video image to be processed acquired by the image acquisition element is acquired, then the video image to be processed is subjected to fuzzification processing by adopting the target low-pass filter before the video image to be processed is subjected to coding compression, so that the high-frequency component of the video image to be processed is reduced, then the video image to be processed after the fuzzification processing is subjected to coding compression, the target video image is obtained, and then the target video image is transmitted. Therefore, before the video image is input to the encoder, the high-frequency filtering convolution is carried out on the video image through the low-pass filter so as to realize the fuzzification processing, so that the high-frequency component of the video image obtained by the encoder and subjected to the encoding is reduced through the filtering processing, the problem of uneven data distribution of the video image when the video image is encoded by the encoder is solved, the mosaic and block effects are reduced in the playing process after the video is integrally encoded, the playing effect is consistent, and the playing fluency is improved.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (7)
1. A method of video image processing, applied to an electronic device, the method comprising:
acquiring a to-be-processed video image acquired by an image acquisition element;
acquiring image configuration parameters of the video image to be processed, wherein the image configuration parameters comprise the resolution and the transmission code rate of the video image to be processed;
calculating an initial code rate of the video image to be processed based on the resolution and the transmission code rate;
acquiring a target code rate of the video image to be processed;
matching the ratio of the initial code rate to the target code rate with a plurality of preset threshold ranges;
taking a low-pass filter corresponding to the threshold range matched with the ratio as a target low-pass filter;
performing convolution operation on the video image to be processed according to the target low-pass filter, wherein the convolution operation is used for reducing high-frequency components in the video image to be processed;
coding and compressing the video image to be processed after the convolution operation to obtain a target video image;
and transmitting the target video image.
2. The method according to claim 1, wherein the video images to be processed comprise a plurality of frames of video images, and the step of using a low-pass filter corresponding to a threshold range matching the ratio as a target low-pass filter comprises:
detecting whether abnormal change exists between a current frame video image to be processed and a previous frame video image in the video image to be processed, wherein the abnormal change is that the content change of the current frame video image to be processed and the previous frame video image is larger than a target condition;
if the target low-pass filter does not exist, taking the first low-pass filter corresponding to the threshold range matched with the ratio as the target low-pass filter;
and if the high-frequency component exists, acquiring a second low-pass filter as a target low-pass filter, wherein the second low-pass filter has higher capability of filtering the high-frequency component than the first low-pass filter.
3. The method of claim 1, wherein the step of using a low-pass filter corresponding to a threshold range matching the ratio as a target low-pass filter comprises:
and if the ratio of the initial code rate to the target code rate is matched with a specified threshold, canceling to obtain the target low-pass filter so as to cancel fuzzification processing before encoding compression.
4. The method according to any one of claims 1-3, wherein said step of performing coding compression on said video image to be processed after performing convolution operation comprises:
and coding and compressing the video image to be processed after the convolution operation based on the transmission code rate.
5. An apparatus for video image processing, operable in an electronic device, the apparatus comprising:
the video image acquisition module is used for acquiring a to-be-processed video image acquired by the image acquisition element;
the video image processing module is used for acquiring image configuration parameters of the video image to be processed, and the image configuration parameters comprise the resolution and the transmission code rate of the video image to be processed; calculating an initial code rate of the video image to be processed based on the resolution and the transmission code rate; acquiring a target code rate of the video image to be processed; matching the ratio of the initial code rate to the target code rate with a plurality of preset threshold ranges; taking a low-pass filter corresponding to the threshold range matched with the ratio as a target low-pass filter; performing convolution operation on the video image to be processed according to the target low-pass filter, wherein the convolution operation is used for reducing high-frequency components of the video image to be processed;
the video image coding module is used for coding and compressing the video image to be processed after the convolution operation is carried out to obtain a target video image;
and the video image output module is used for transmitting the target video image.
6. An electronic device comprising an image capture element, one or more processors, and a memory;
one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-4.
7. A computer-readable storage medium, having a program code stored therein, wherein the program code when executed by a processor performs the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910472833.0A CN110149554B (en) | 2019-05-31 | 2019-05-31 | Video image processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910472833.0A CN110149554B (en) | 2019-05-31 | 2019-05-31 | Video image processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110149554A CN110149554A (en) | 2019-08-20 |
| CN110149554B true CN110149554B (en) | 2021-06-15 |
Family
ID=67590192
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910472833.0A Active CN110149554B (en) | 2019-05-31 | 2019-05-31 | Video image processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110149554B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110996131B (en) * | 2020-03-02 | 2020-11-10 | 腾讯科技（深圳）有限公司 | Video encoding method, video encoding device, computer equipment and storage medium |
| CN111402165B (en) * | 2020-03-18 | 2022-12-09 | 展讯通信（上海）有限公司 | Image processing method, device, equipment and storage medium |
| CN112087636B (en) * | 2020-08-07 | 2022-01-11 | 北京博雅慧视智能技术研究院有限公司 | Image coding processing method and device, storage medium and terminal |
| CN114363693B (en) * | 2020-10-13 | 2023-05-12 | 华为技术有限公司 | Image quality adjusting method and device |
| CN112565660A (en) * | 2020-12-08 | 2021-03-26 | 维沃移动通信有限公司 | Image processing method and device |
| CN112631524B (en) * | 2020-12-29 | 2023-04-07 | 杭州海康汽车软件有限公司 | Storage medium distribution method and device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06233266A (en) * | 1993-02-02 | 1994-08-19 | Matsushita Electric Ind Co Ltd | Video signal encoder and video signal decoder |
| WO2000025312A1 (en) * | 1998-10-27 | 2000-05-04 | Hitachi Maxell, Ltd. | Information recording method and system, image compression/decompression system, system control method, and monitoring system including part or all of them |
| CN1607809A (en) * | 2003-10-15 | 2005-04-20 | 株式会社Ntt都科摩 | Image signal processing method and apparatus and program |
| CN1799492A (en) * | 2005-12-02 | 2006-07-12 | 清华大学 | Quasi-lossless image compression and decompression method of wireless endoscope system |
| CN101668196A (en) * | 2009-09-25 | 2010-03-10 | 西安电子科技大学 | Low code rate image compression method based on down sampling and interpolation |
| CN104661023A (en) * | 2015-02-04 | 2015-05-27 | 天津大学 | Image or video coding method based on predistortion and training filter |
| JP2015226260A (en) * | 2014-05-29 | 2015-12-14 | パナソニックＩｐマネジメント株式会社 | Image processor and image processing method |
| CN107592535A (en) * | 2017-08-18 | 2018-01-16 | 西安邮电大学 | H.265/HEVC image layer bit rate control method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7302104B2 (en) * | 2001-12-28 | 2007-11-27 | Ricoh Co., Ltd. | Smoothing tile boundaries of images encoded and decoded by JPEG 2000 |
-
2019
- 2019-05-31 CN CN201910472833.0A patent/CN110149554B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06233266A (en) * | 1993-02-02 | 1994-08-19 | Matsushita Electric Ind Co Ltd | Video signal encoder and video signal decoder |
| WO2000025312A1 (en) * | 1998-10-27 | 2000-05-04 | Hitachi Maxell, Ltd. | Information recording method and system, image compression/decompression system, system control method, and monitoring system including part or all of them |
| CN1607809A (en) * | 2003-10-15 | 2005-04-20 | 株式会社Ntt都科摩 | Image signal processing method and apparatus and program |
| CN1799492A (en) * | 2005-12-02 | 2006-07-12 | 清华大学 | Quasi-lossless image compression and decompression method of wireless endoscope system |
| CN101668196A (en) * | 2009-09-25 | 2010-03-10 | 西安电子科技大学 | Low code rate image compression method based on down sampling and interpolation |
| JP2015226260A (en) * | 2014-05-29 | 2015-12-14 | パナソニックＩｐマネジメント株式会社 | Image processor and image processing method |
| CN104661023A (en) * | 2015-02-04 | 2015-05-27 | 天津大学 | Image or video coding method based on predistortion and training filter |
| CN107592535A (en) * | 2017-08-18 | 2018-01-16 | 西安邮电大学 | H.265/HEVC image layer bit rate control method |
Non-Patent Citations (2)
| Title |
|---|
| 《基于全相位双正交变换和全相位内插的图像压缩算法研究》;谢松昭;《中国优秀硕士学位论文全文数据库》;20150615;全文 * |
| 《基于拉普拉斯下采样和方向波插值的低码率图像压缩》;李栋;《中国优秀硕士学位论文全文数据库》;20100615;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110149554A (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110149554B (en) | Video image processing method and device, electronic equipment and storage medium | |
| US11006046B2 (en) | Image processing method and mobile terminal | |
| CN105472205B (en) | Real-time video noise reduction method and device in encoding process | |
| CN101371589B (en) | Adaptive filtering to enhance video encoder performance | |
| EP3014880B1 (en) | Encoding video captured in low light | |
| CN109640169B (en) | Video enhancement control method and device and electronic equipment | |
| CN112805990A (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| CN109618228B (en) | Video enhancement control method and device and electronic equipment | |
| CN109688465B (en) | Video enhancement control method and device and electronic equipment | |
| CN110445951B (en) | Video filtering method and device, storage medium and electronic device | |
| CN109640179B (en) | Video decoding control method and device and electronic equipment | |
| JP2014523713A (en) | Frame coding selection based on similarity, visual quality, and interest | |
| JPH04219089A (en) | Image encoding device | |
| CN108668166B (en) | Coding method, device and terminal equipment | |
| CN116847087A (en) | Video processing method and device, storage medium and electronic equipment | |
| CN109120979B (en) | Video enhancement control method and device and electronic equipment | |
| CN110728180A (en) | Image processing method, device and storage medium | |
| US11538169B2 (en) | Method, computer program and system for detecting changes and moving objects in a video view | |
| CN111416937B (en) | Image processing method, image processing device, storage medium and mobile equipment | |
| CN112183227B (en) | Intelligent face region coding method and device | |
| CN109819187B (en) | Video recording control method and device, mobile terminal and storage medium | |
| CN107547895B (en) | Image processing method and device | |
| KR100778924B1 (en) | Image processing device | |
| CN117615146A (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| CN109712100B (en) | Video enhancement control method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |