CN118233675B - Live image optimization method and system based on artificial intelligence - Google Patents
Live image optimization method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN118233675B CN118233675B CN202410643084.4A CN202410643084A CN118233675B CN 118233675 B CN118233675 B CN 118233675B CN 202410643084 A CN202410643084 A CN 202410643084A CN 118233675 B CN118233675 B CN 118233675B
- Authority
- CN
- China
- Prior art keywords
- image
- live broadcast
- optimized
- optimization
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005457 optimization Methods 0.000 title claims abstract description 361
- 238000000034 method Methods 0.000 title claims abstract description 109
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 252
- 238000004422 calculation algorithm Methods 0.000 claims description 326
- 238000005070 sampling Methods 0.000 claims description 117
- 230000009467 reduction Effects 0.000 claims description 62
- 238000013508 migration Methods 0.000 claims description 61
- 230000005012 migration Effects 0.000 claims description 61
- 230000002452 interceptive effect Effects 0.000 claims description 32
- 230000001629 suppression Effects 0.000 claims description 15
- 238000004140 cleaning Methods 0.000 claims description 11
- 230000000694 effects Effects 0.000 description 123
- 230000000875 corresponding effect Effects 0.000 description 99
- 230000008569 process Effects 0.000 description 60
- 230000000007 visual effect Effects 0.000 description 52
- 238000013507 mapping Methods 0.000 description 23
- 238000011156 evaluation Methods 0.000 description 21
- 239000013598 vector Substances 0.000 description 15
- 230000006872 improvement Effects 0.000 description 9
- 230000001360 synchronised effect Effects 0.000 description 8
- 238000003860 storage Methods 0.000 description 7
- 238000012937 correction Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000012216 screening Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 4
- 230000001965 increasing effect Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000005065 mining Methods 0.000 description 3
- 238000013139 quantization Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012854 evaluation process Methods 0.000 description 2
- 230000003631 expected effect Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000036548 skin texture Effects 0.000 description 2
- 230000026676 system process Effects 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 230000008485 antagonism Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234345—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Image Processing (AREA)
Abstract
The application relates to the technical field of image processing, in particular to a live image optimization method and a live image optimization system based on artificial intelligence.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a live image optimization method and system based on artificial intelligence.
Background
In the current live technical field, quality optimization of live images is an important technical challenge. The traditional live image processing method can only apply a single optimization algorithm, and cannot be flexibly adjusted according to the actual requirements and scenes of the images, so that further improvement of the image optimization effect is limited. Meanwhile, how to efficiently and accurately screen out the key images to be optimized is a great difficulty faced by the current technology in the face of a large amount of live image data.
In addition, with the continuous improvement of the requirements of audience on live broadcast image quality, the traditional image processing method has difficulty in meeting the requirements of high definition and fluency on live broadcast. Particularly, under the condition of limited network bandwidth or low original video source resolution, how to effectively improve the resolution and definition of live images becomes a problem to be solved urgently.
Disclosure of Invention
In order to improve the problems, the application provides a live image optimization method and system based on artificial intelligence.
The embodiment of the application provides a live image optimization method based on artificial intelligence, which is applied to a live image optimization system, and comprises the following steps:
Acquiring a plurality of live broadcast optimized images obtained by operating the target live broadcast images in a quality optimized state according to target optimization indication features; the live broadcast optimized images are generated through at least two image optimization processing algorithms;
Based on an image optimization processing algorithm corresponding to each live broadcast optimized image, comprehensively evaluating and sampling the live broadcast optimized images to obtain at least one target live broadcast optimized image to be processed;
if a target image optimization request matched with the target optimization indicating characteristic is obtained under the condition that the target live image is in a quality optimization state, determining a current image optimization processing algorithm;
Determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimized processing algorithm and the image optimized processing algorithm respectively corresponding to each target live broadcast optimized image;
And performing over-processing on the core live broadcast optimized image to adapt to the image quality optimization requirement corresponding to the target image optimization request.
In an optional technical solution, the performing comprehensive evaluation sampling on the plurality of live broadcast optimized images based on the image optimization processing algorithm corresponding to each live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed includes:
Searching redundant image contents from a plurality of live broadcast optimized images through an image optimization processing algorithm respectively corresponding to each live broadcast optimized image;
performing cleaning treatment on redundant image content in the live broadcast optimized images to obtain at least one style of intermediate live broadcast optimized image;
and sampling each intermediate live broadcast optimized image by utilizing at least one of an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or the superdivision state of each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed.
In an optional technical solution, the retrieving, by an image optimization processing algorithm corresponding to each live broadcast optimized image, redundant image contents from a plurality of live broadcast optimized images includes:
The method comprises the steps of enabling an image optimization processing algorithm corresponding to a current optimized image frame in a plurality of live optimized images to realize redundancy discrimination with an image optimization processing algorithm corresponding to each live optimized image except the current optimized image frame in the plurality of live optimized images;
and taking the current optimized image frame as redundant image content on the basis of the live optimized image which is successfully distinguished from the current optimized image frame in redundancy.
In an optional technical solution, the image optimization processing algorithm includes a face beautifying algorithm variable, a style migration algorithm variable and a color noise reduction algorithm variable, and the implementing redundancy discrimination between the image optimization processing algorithm corresponding to a current optimized image frame in a plurality of live broadcast optimized images and the image optimization processing algorithm corresponding to each live broadcast optimized image except the current optimized image frame in the plurality of live broadcast optimized images includes:
for each live broadcast optimized image except the current optimized image frame in a plurality of live broadcast optimized images, determining whether a face beautifying algorithm variable corresponding to the current optimized image frame is consistent with a face beautifying algorithm variable corresponding to the corresponding live broadcast optimized image;
Determining whether a style migration algorithm variable corresponding to the current optimized image frame is consistent with a style migration algorithm variable corresponding to the corresponding live broadcast optimized image;
determining whether a color noise reduction algorithm variable corresponding to the current optimized image frame is consistent with a color noise reduction algorithm variable corresponding to the corresponding live broadcast optimized image;
and taking the live broadcast optimized image corresponding to and consistent with the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable of the current optimized image frame as a live broadcast optimized image successfully distinguished with the redundancy of the current optimized image frame.
In an optional technical solution, the sampling operation is performed on each intermediate live broadcast optimized image by using at least one of an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or a superdivision state of each intermediate live broadcast optimized image, to obtain at least one target live broadcast optimized image to be processed, including:
performing at least one type of operation in time sequence feature sampling operation or style updating sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed;
the time sequence feature sampling operation is realized based on the image optimization processing algorithm corresponding to each intermediate live broadcast optimized image, and the style updating sampling operation is realized based on the superminute state of each intermediate live broadcast optimized image.
In an optional technical solution, the image optimization processing algorithm includes a face beautifying algorithm variable, a style migration algorithm variable and a color noise reduction algorithm variable, and the step of the time sequence feature sampling operation includes:
Determining interactive live broadcast images from the intermediate live broadcast optimized images;
for each intermediate live broadcast optimized image except the interactive live broadcast image, determining whether a face beautifying algorithm variable corresponding to the interactive live broadcast image is consistent with a face beautifying algorithm variable corresponding to the corresponding intermediate live broadcast optimized image;
determining whether the style migration algorithm variables corresponding to the interactive live broadcast image and the corresponding intermediate live broadcast optimized image are consistent;
determining whether the color noise reduction algorithm variable corresponding to the interactive live broadcast image contains the color noise reduction algorithm variable corresponding to the corresponding intermediate live broadcast optimized image;
And determining the intermediate live broadcast optimized image which corresponds to and is consistent with the face beautifying algorithm variable and the style migration algorithm variable of the interactive live broadcast image and is contained by the color noise reduction algorithm variable of the interactive live broadcast image as a time sequence associated live broadcast image, and cleaning the time sequence associated live broadcast image to obtain at least one target live broadcast optimized image to be processed.
In an alternative solution, the step of style update sampling operation includes:
determining superprocessing iteration values of the intermediate live broadcast optimized images by using superdivision states of the intermediate live broadcast optimized images;
And taking the intermediate live broadcast optimized image with the superdivision processing iteration value meeting the set iteration requirement as a target live broadcast optimized image.
In an optional technical solution, the performing at least one type of operation of time sequence feature sampling operation or style updating sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed includes:
Performing time sequence feature sampling operation on the intermediate live broadcast optimized images by using an image optimization processing algorithm respectively corresponding to each intermediate live broadcast optimized image;
Performing style updating sampling operation on the middle live broadcast optimized images by using the superdivision state of each middle live broadcast optimized image;
And determining at least one target live broadcast optimized image to be processed based on the live broadcast optimized image after the time sequence feature sampling operation and the live broadcast optimized image after the style updating sampling operation.
In an optional technical solution, the performing at least one type of operation of time sequence feature sampling operation or style updating sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed includes:
Performing time sequence feature sampling operation on the intermediate live broadcast optimized images by using an image optimization processing algorithm respectively corresponding to each intermediate live broadcast optimized image;
And carrying out style updating sampling operation on the live broadcast optimized images after the time sequence feature sampling operation according to the superdivision state of each live broadcast optimized image after the time sequence feature sampling operation to obtain at least one target live broadcast optimized image to be processed.
In an optional technical scheme, the image optimization processing algorithm comprises a face beautifying algorithm variable, a style migration algorithm variable and a color noise reduction algorithm variable;
The determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimizing processing algorithm and the image optimizing processing algorithm corresponding to each target live broadcast optimized image respectively includes:
Determining a first linkage live broadcast optimized image with a face beautifying algorithm variable consistent with a current face beautifying algorithm variable of an image optimizing client and a style migration algorithm variable consistent with a current style migration algorithm variable of the image optimizing client from at least one target live broadcast optimized image;
For each first linkage live broadcast optimized image, determining the confidence weight of each noise suppression task in the color noise reduction algorithm variable corresponding to the current first linkage live broadcast optimized image based on the current optimizing processing label of the image optimizing client;
and determining a core live broadcast optimized image from the first linkage live broadcast optimized image according to the confidence weight of each noise suppression task.
In an optional technical solution, the image optimization processing algorithm includes: a face beautification algorithm variable, a style migration algorithm variable and a color noise reduction algorithm variable;
The determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimizing processing algorithm and the image optimizing processing algorithm corresponding to each target live broadcast optimized image respectively includes:
Determining that a face beautifying algorithm variable is consistent with a current face beautifying algorithm variable of an image optimizing client from at least one target live broadcast optimized image, a style migration algorithm variable is consistent with a current style migration algorithm variable of the image optimizing client, and a second linkage live broadcast optimized image of a noise suppression task incompatible with a current optimizing processing label of the image optimizing client does not exist in a color noise reduction algorithm variable;
And selecting the second linkage live broadcast optimized image with the largest superprocessing iteration value from the second linkage live broadcast optimized images as a core live broadcast optimized image.
The embodiment of the application provides a live image optimization system, which comprises at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored by the memory, causing the at least one processor to perform the method described above.
Embodiments of the present application provide a computer readable storage medium having stored thereon a computer program which, when run, implements the method described above.
According to the embodiment of the application, through a series of steps of applying various image optimization processing algorithms, comprehensively evaluating and sampling, determining the core live broadcast optimized image, performing super-division processing and the like, not only are the optimization effect and processing efficiency of the live broadcast image improved, but also the image quality optimization requirements of viewers on high-quality live broadcast content are successfully adapted.
Drawings
Fig. 1 is a flowchart of a live image optimization method based on artificial intelligence according to an embodiment of the present application.
Fig. 2 is a schematic structural diagram of a live image optimization system 200 according to an embodiment of the present application.
Detailed Description
In order to better understand the above technical solutions, the following detailed description of the technical solutions of the present application is made by using the accompanying drawings and specific embodiments, and it should be understood that the specific features of the embodiments and the embodiments of the present application are detailed descriptions of the technical solutions of the present application, and not limiting the technical solutions of the present application, and the technical features of the embodiments and the embodiments of the present application may be combined with each other without conflict.
Fig. 1 shows an artificial intelligence based live image optimization method applied to a live image optimization system, the method comprising the following steps 110-150.
Step 110, a live broadcast image optimization system acquires a plurality of live broadcast optimized images obtained by operating a target live broadcast image in a quality optimization state according to target optimization indication characteristics; the live broadcast optimized images are generated through at least two image optimization processing algorithms.
In the embodiment of the application, the original image which needs to be optimally processed in the live broadcast process can be a real-time picture captured by a live broadcast camera or a frame of image in a video stream transmitted by other equipment (such as a mobile phone, a tablet computer and the like). The method is the basis for processing and optimizing the live image optimizing system.
The quality optimization state refers to a process in which the live image optimization system is performing a series of optimization processes on the target live image. In this process, the system applies different image optimization processing algorithms to improve the visual effects of the image, such as sharpness, color saturation, contrast, etc. The image entering the quality optimization state is analyzed and processed by the system so as to achieve the best live effect.
The target optimization indicating features are a set of specific parameters or criteria for guiding the live image optimization system how to optimize the target live image, and these features may include visual properties such as brightness, contrast, saturation, etc. of the image, and may also include more complex features such as edge sharpness, color distribution, etc. For example, one example of a numerical feature vector may be [0.8,0.4,0.6], where each number represents an optimal target value for brightness, contrast, and saturation, respectively. The system adjusts the corresponding properties of the image based on these characteristics to achieve the desired optimization.
The live broadcast optimized image is an image processed by the live broadcast image optimizing system, and the quality, visual effect and the like of the image are improved compared with the original target live broadcast image. The live optimized image is generated by applying one or more image optimization processing algorithms, which aims to improve the viewing experience of the audience.
Image optimization processing algorithms refer to a series of mathematical and computer vision techniques for improving image quality, which may include, but are not limited to, noise reduction, sharpening, color correction, contrast enhancement, and the like. Each algorithm is modified for a particular aspect of the image to enhance the overall visual effect. For example, a noise reduction algorithm may reduce noise and mottle in an image, while a sharpening algorithm may enhance edges and details of the image.
It will be appreciated that step 110 is one of the key steps of the live image optimization system, which involves the process of acquiring and processing live images. Specifically, when the target live image is in a quality optimization state, the system operates according to preset target optimization indication characteristics to generate a plurality of live optimization images, and the operations are completed based on at least two image optimization processing algorithms.
First, the system receives image data from a real-time video stream or other device transmission from a camera, which constitutes the original target live image. Once these images enter the quality optimization state, the system may begin analyzing and processing them.
During processing, the system refers to the target optimization indicating feature, which is a set of preset parameters or criteria for guiding the optimization direction of the image. For example, if higher brightness and contrast values are set in the target optimization designation feature, the system may attempt to achieve these target values by adjusting the brightness and contrast of the image.
To achieve these adjustments, the system applies a variety of image optimization processing algorithms that may include noise reduction algorithms to reduce noise and mottle in the image, sharpening algorithms to enhance edges and details of the image, color correction algorithms to adjust the color balance of the image, and so forth. Each algorithm is improved for some aspect or aspects of the image, thereby collectively improving the overall quality of the image.
After processing with these algorithms, the system generates several live optimized images, which are all improved based on the original target live image, but each image can be applied with different algorithm combinations or parameter settings, so that they have differences in visual effect, which are the attempts and exploration by the system to find the best optimized effect.
Thus, step 110 is a key step in the live image optimization system to initially process and optimize the target live image. Through the step, the system can generate a plurality of live broadcast optimized images with different optimized effects, and rich materials and selection space are provided for subsequent comprehensive evaluation, sampling and further processing.
And 120, the live broadcast image optimization system comprehensively evaluates and samples the live broadcast optimized images based on image optimization processing algorithms respectively corresponding to the live broadcast optimized images to obtain at least one target live broadcast optimized image to be processed.
In the embodiment of the application, the comprehensive evaluation sampling is a key link in the live broadcast image optimization system, and refers to a process that the system comprehensively evaluates and selects all generated live broadcast optimization images. In the process, the system considers the effect of an image optimization processing algorithm corresponding to each optimized image, the overall quality of the image and the matching degree with a preset optimization target. The comprehensive evaluation may involve aspects of sharpness, color appearance, noise control, edge preservation, etc. of the image. The system samples the best performing image as a candidate for subsequent processing through a series of quantization indices and subjective visual effect evaluations.
The target live broadcast optimized image to be processed refers to an image which is selected by a live broadcast image optimizing system and is ready for further processing after comprehensive evaluation and sampling, and the images are excellent in the previous optimizing process and accord with or approach to the optimizing target preset by the system, so that the images are regarded as excellent candidates for subsequent super-resolution processing or other advanced image processing, the images to be processed can have higher visual quality, and the detail performance, color restoration and other aspects are obviously improved.
Further, step 120 is one of the key steps in the live image optimization process involving comprehensive evaluation and sampling selection of multiple live optimized images.
First, the system reviews several live optimized images generated in step 110 by different image optimization processing algorithms according to the target optimization designation feature, which images may each have different optimization effects and characteristics, so it is necessary to determine which images best meet the preset optimization target by comprehensive evaluation.
In the comprehensive evaluation process, the system considers a number of aspects including, but not limited to, sharpness of the image, color saturation, contrast, noise control, and preservation of edge details. For more scientific evaluation, the system may employ a series of quantization indices, such as peak signal-to-noise ratio (PSNR), structural Similarity Index (SSIM), etc., to objectively measure the quality of each optimized image.
In addition to the quantization index, the system may evaluate in combination with subjective visual effects, since the visual perception of the viewer is one of the important criteria for evaluating image quality. The system may further screen the image by simulating the response of the human visual system or inviting professionals to make subjective evaluations.
After the comprehensive evaluation is completed, the system performs sampling selection, and the purpose of this step is to select at least one image with the best performance from a plurality of optimized images as a target of subsequent processing. The sampling selection may be based on a composite score of the evaluation results or targeted based on specific requirements such as sharpness optimization in low light conditions.
Finally, the target live broadcast optimized image to be processed obtained by comprehensive evaluation sampling is used as the basis of the next super processing or other advanced image processing, and the images not only have remarkable improvement on visual effect, but also are more in line with the viewing demands and expectations of audiences.
And 130, if the live image optimization system obtains a target image optimization request matched with the target optimization indicating characteristic under the condition that the target live image is in a quality optimization state, determining a current image optimization processing algorithm.
In the embodiment of the application, the target image optimization request refers to an instruction or demand automatically initiated by a user or a system in the live broadcast process, and requires the live broadcast image optimization system to perform specific optimization processing on the current live broadcast image, wherein the request can contain specific optimization targets and parameters, such as improving the definition of the image, adjusting the color balance, reducing noise and the like. The request may be sent by the viewer through a user interface or the system may be triggered automatically based on the current live environment and image quality. The target image optimization request is a direct cause of triggering the live image optimization system to perform the corresponding operation.
The current image optimization processing algorithm refers to the most suitable optimization algorithm selected by the system according to the target image optimization request and the state of the target live image in the live image optimization process, and the algorithm can be selected from a plurality of available image optimization algorithms so as to achieve the best optimization effect. For example, if the target image optimization request is to increase the sharpness of the image, the current image optimization processing algorithm may be a sharpening algorithm or a super-resolution reconstruction algorithm. The system dynamically selects the most appropriate algorithm for processing according to the specific content of the request and the current state of the image.
It will be appreciated that step 130 plays a key role in the process flow of the live image optimization system. In this step, the system determines the current image optimization processing algorithm to be used next based on the current quality optimization state of the target live image and the received target image optimization request.
First, the system continuously monitors the quality optimization state of the target live image, which reflects the current visual effect of the image and the optimization direction that may be required. At the same time, the system also receives target image optimization requests from the user or the system itself, which explicitly indicate that the user or the system desires to perform an optimization operation on the image, such as enhancing sharpness, enhancing color, etc.
Step 130 may be triggered when the system detects that the target live image is in a state that requires optimization and a target image optimization request is received that matches the target optimization designation feature. The system analyzes specific optimization objectives and parameters in the request, such as the required level of sharpness, color adjustment direction, etc.
Next, the system comprehensively considers the current state of the target live image, the specific requirements in the received optimization request, and the available image optimization processing algorithm library to determine the optimization algorithm most suitable for the current situation, and the process can involve evaluation and comparison of multiple algorithms to ensure that the selected algorithm can most effectively achieve the optimization target in the request.
Finally, the system determines one or more current image optimization processing algorithms based on the above analysis, which will be applied to the target live image in a subsequent step to achieve the user or system desired optimization effect, the decision of this step being critical to the success of the overall optimization procedure, as it directly determines the direction and effect of the subsequent processing.
And 140, the live broadcast image optimization system determines a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimization processing algorithm and the image optimization processing algorithm respectively corresponding to each target live broadcast optimized image.
In the embodiment of the application, the core live broadcast optimized image refers to an image which is identified as the most important or most preferable image to be optimized through the processing of a live broadcast image optimizing system in the live broadcast process, and the image can be an image which is judged by the system to be the most critical to the current live broadcast quality improvement according to a specific optimizing algorithm and processing logic in a plurality of target live broadcast images. The determination of the core live broadcast optimized image not only depends on the content of the image, but also depends on the optimization processing algorithm of the current image and the optimization requirements corresponding to each target live broadcast image. Once the core live optimized image is determined, the system will preferentially apply a corresponding optimization process to it to ensure that the overall visual effect of the live is optimal.
Further, step 140 involves a process of picking out a core live optimized image from a plurality of target live optimized images.
In this step, the live image optimization system first reviews and evaluates the current image optimization processing algorithm based on the algorithm that has been determined and selected in the previous step (e.g., step 130). The system knows the optimization emphasis and expected effect of these algorithms, which is crucial for the subsequent selection of core live optimized images.
Next, the system analyzes each of the target live optimization images, which may contain different scenes, people, or objects, and their respective image optimization processing algorithms, each of which may require a different optimization strategy. For example, some images may require sharpness enhancement, while others may require color correction or noise reduction.
In determining the core live optimized image, the live image optimization system integrates a number of factors including, but not limited to, importance of the image content, audience attention, urgency of image optimization, and potential impact of the optimization process on overall live quality improvement, which decision process may be dynamic, adjusting as the live content and audience feedback changes.
Finally, the system selects one or a few of the plurality of target live optimized images as core live optimized images, which will be preferentially processed to ensure that they are presented to the viewer in the best state in the live. By the method, the live image optimization system can improve the visual quality of live in real time and pertinently, so that the viewing experience of audiences is enhanced.
And 150, performing over-processing on the core live broadcast optimized image by a live broadcast image optimization system so as to adapt to the image quality optimization requirements corresponding to the target image optimization request.
In the embodiment of the present application, that is, super-resolution processing, is an image processing technology, which aims to reconstruct a high-resolution image from a low-resolution image. In the process, the algorithm analyzes the details and the structure of the image, and the resolution and the definition of the image are improved by increasing the pixel density, sharpening the edges, filling the details and the like, so that the image is displayed more finely and truly. In live scenes, the super-processing is particularly important for improving image quality and improving the visual experience of audiences, especially in the case of limited network bandwidth or low resolution of the original video source.
Image quality optimization requirements may refer to specific requirements placed on live video quality by a viewer or a live platform, and these requirements may include, but are not limited to, improving sharpness, color saturation, contrast of an image, reducing noise and artifacts in an image, enhancing smoothness of a dynamic picture, and the like. The image quality optimization requirements are an important reference when the live image optimization system processes images, and the system adjusts and optimizes the image processing algorithm according to the requirements so as to achieve the best viewing effect.
In detail, step 150 is a key step in the live broadcast image optimization process, and involves performing superprocessing on the core live broadcast optimized image to meet the specific image quality optimization requirement.
In this step, the live image optimization system first acquires the core live optimized images determined in the previous step, and these images may not reach the desires of the audience or the platform for the image quality due to various reasons, such as network transmission limitation, low resolution of the original acquisition device, and the like. Therefore, it is necessary to improve their resolution and overall image quality by the super-processing.
Next, the system analyzes specific image quality optimization requirements in the target image optimization request, which may include aspects of improving sharpness, enhancing color performance, reducing noise, and the like. The system selects the appropriate superdivision processing algorithm and parameters according to these requirements.
And then, the live broadcast image optimization system processes the core live broadcast optimized image by applying a selected superdivision processing algorithm, in the process, the algorithm deeply analyzes each pixel and detail of the image, and the detail information of the image is increased and the resolution and definition of the image are improved through complex mathematical calculation and image processing technology. Meanwhile, the system can adjust the attributes such as color, contrast, sharpness and the like of the image according to the image quality optimization requirement, so that the system is more in line with the vision habit of the audience.
Finally, the core live broadcast optimized image subjected to the super-processing can be output with higher image quality and resolution, so that the optimization requirement of the audience and the platform on the live broadcast image quality is met, and the completion of the step marks that the live broadcast image optimization system has important progress in improving the viewing experience of the audience.
The solution described in the above steps 110-150 is next presented by way of a complete example.
Firstly, a live broadcast image optimization system acquires a series of live broadcast optimized images, wherein the images are obtained by operating according to target optimization indication characteristics when the target live broadcast images are in a quality optimization state. Importantly, these images are generated by at least two image optimization processing algorithms. For example, some images can be processed by a sharpening algorithm, so that the edges of the images are clearer; some may use noise reduction algorithms to reduce noise and mottle in the image. Next, the live image optimization system performs comprehensive evaluation sampling on the images according to an image optimization processing algorithm corresponding to each live optimized image, where the evaluation process may include aspects of image sharpness, color saturation, contrast, and the like. Through this step, the system screens out at least one target live optimized image to be processed, which is relatively excellent in all aspects. After determining the target live broadcast optimized image to be processed, the system checks whether a target image optimization request matching the target optimization indication characteristic is received under the condition that the target live broadcast image is in a quality optimization state, wherein the request can be manually set by a user or automatically adjusted according to the current live broadcast environment and audience feedback. Upon receipt of such a request, the system determines the currently most appropriate image optimization processing algorithm. With the current image optimization processing algorithm, the live broadcast image optimization system combines the algorithm corresponding to the target live broadcast optimization image screened before to determine a core live broadcast optimization image, wherein the core live broadcast optimization image is the image which can reach the expected effect most under the current optimization requirement and the available algorithm. Finally, in order to further improve the image quality and meet the image quality optimization requirements of users or systems, the live broadcast image optimization system performs super-division processing on the core live broadcast optimization image. The super processing is a technology for improving the resolution and detail expression of an image through an algorithm, and can make the image look clearer and finer. Through the steps, the live image optimization system can effectively improve the visual effect of the live image, and brings clearer and vivid viewing experience for audiences.
According to the technical scheme provided by the embodiment of the application, a plurality of live broadcast optimized images are generated by applying a plurality of image optimization processing algorithms to the target live broadcast images, so that the diversity and flexibility of image optimization are enhanced, and rich samples are provided for subsequent selection. And then, screening out target live broadcast optimized images to be processed from a plurality of optimized images by a comprehensive evaluation sampling method, wherein the method not only ensures the processing efficiency, but also improves the accuracy of the optimized result.
When the current image optimization processing algorithm is determined, the quality optimization state of the target live image and the target image optimization request are considered, so that the selected algorithm is more in line with actual requirements and scenes, and the optimization effect is further improved. It is particularly worth mentioning that the core live broadcast optimized image is determined from the screened target live broadcast optimized images, and the step ensures that resources can be concentrated on the most critical image, so that the pertinence and the effectiveness of the optimizing work are improved.
Finally, the core live broadcast optimized image is subjected to super-division processing, so that the definition and detail performance of the image are remarkably improved, the image quality optimization requirement corresponding to the target image optimization request is successfully adapted, the visual experience of the live broadcast image is greatly enhanced, and the expectation of a user on high-quality live broadcast content is met.
In summary, the technical scheme of the embodiment of the application improves the optimization effect of the live image and the processing efficiency through a series of carefully designed steps, so that the live content meets the visual requirement of the audience better, and makes positive contribution to the technical progress of the live industry.
In some preferred embodiments, the performing comprehensive evaluation sampling on the plurality of live broadcast optimized images based on the image optimization processing algorithm corresponding to each live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed includes: searching redundant image contents from a plurality of live broadcast optimized images through an image optimization processing algorithm respectively corresponding to each live broadcast optimized image; performing cleaning treatment on redundant image content in the live broadcast optimized images to obtain at least one style of intermediate live broadcast optimized image; and sampling each intermediate live broadcast optimized image by utilizing at least one of an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or the superdivision state of each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed.
Based on the embodiment, in the process of executing comprehensive evaluation sampling, the live broadcast image optimization system adopts a refined method to screen live broadcast optimized images, and the process firstly involves retrieving redundant image contents from the images based on image optimization processing algorithms respectively corresponding to the live broadcast optimized images. Redundant image content may include duplicate pictures, nonsensical blank areas, or portions of images that are too highly similar due to processing by various optimization algorithms.
The live image optimization system uses advanced image recognition and analysis technology to comprehensively scan and compare the live optimized images to recognize and mark redundant image contents, which is critical because the live image optimization system can help the system reduce the data volume of subsequent processing, improve the processing efficiency and avoid unnecessary repeated optimization work.
And then, the system cleans redundant image contents in the live optimized images. The cleaning process is a data cleansing process that aims to remove or replace those image portions marked as redundant, thereby resulting in a set of differently stylized intermediate live optimized images that are more refined and unique in content and visual effect, providing a higher quality basis for subsequent optimization work.
Next, the live broadcast image optimization system performs further sampling operation by using an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or considering the superdivision state of the images, and the purpose of the step is to select the target live broadcast optimized image with the most representation and optimization potential from the cleaned intermediate images. The sampling operation may be performed based on multiple dimensions of image complexity, information content, visual effects, etc., to ensure that the selected image reflects the diversity and detail of the original live content to the greatest extent.
Finally, through the series of refined processing steps, the live broadcast image optimization system can accurately screen at least one target live broadcast optimized image to be processed from a plurality of live broadcast optimized images, redundant contents of the images are removed, key information and visual effects of original live broadcast are reserved, and a solid foundation is laid for subsequent superdivision processing and image quality optimization.
Therefore, the efficiency and the accuracy of the live image optimization system in processing a large number of live images are remarkably improved by comprehensively utilizing the technical means such as image recognition, cleaning processing and fine sampling. The method and the system can effectively remove redundant content, can ensure that the selected image has high representativeness and optimization potential, thereby bringing clearer and smoother live broadcast watching experience for users, and fully embody the technical innovation and practical value of the live broadcast image optimization system in the aspects of improving live broadcast image quality and user experience.
In the next step, the redundant image content is retrieved from a plurality of live broadcast optimized images through an image optimization processing algorithm corresponding to each live broadcast optimized image, and the method comprises the following steps: the method comprises the steps of enabling an image optimization processing algorithm corresponding to a current optimized image frame in a plurality of live optimized images to realize redundancy discrimination with an image optimization processing algorithm corresponding to each live optimized image except the current optimized image frame in the plurality of live optimized images; and taking the current optimized image frame as redundant image content on the basis of the live optimized image which is successfully distinguished from the current optimized image frame in redundancy.
In the embodiment of the application, the live broadcast image optimization system searches redundant image content from a plurality of live broadcast optimized images through an image optimization processing algorithm respectively corresponding to each live broadcast optimized image, and the step is a key for ensuring the high efficiency of the image optimization flow and avoiding unnecessary repeated work.
Firstly, the system selects one of a plurality of live broadcast optimized images as a current optimized image frame, and the current optimized image frame is a reference for redundancy judgment of the system.
And then, the live broadcast image optimization system adopts a unique redundancy discrimination mechanism, and the mechanism compares an image optimization processing algorithm corresponding to the current optimized image frame with an image optimization processing algorithm corresponding to each live broadcast optimized image except the current optimized image frame, and the comparison process is used for identifying whether live broadcast optimized images which are highly similar or repeated in content with the current optimized image frame exist or not.
In the redundancy judging process, the system deeply analyzes the specific change and effect of each image optimization processing algorithm on the images so as to judge whether redundancy exists between different images. For example, if both images are subjected to similar color correction or sharpening processes and these processes produce similar effects visually, then both images may be considered redundant.
When the system finds that a live optimized image successfully distinguished from the current optimized image frame in redundancy, the system marks the current optimized image frame as redundant image content, which means that the image frame is highly similar to other image frame or image frames in content and optimizing effect, and therefore can be regarded as redundant in subsequent processing and processed correspondingly.
Through the step, the live image optimization system can effectively identify and clear redundant image contents, so that the efficiency and accuracy of image processing are improved, occupation of storage space is reduced, and unnecessary repeated processing of similar or repeated images in a subsequent optimization process can be avoided.
Therefore, through a unique redundancy discrimination mechanism, the live image optimization system can intelligently identify and clear redundant image contents, so that the efficiency and accuracy of image processing are improved. The technical innovation has important significance for improving the quality and the optimization effect of the live broadcast image, and simultaneously brings smoother and efficient live broadcast watching experience for users.
In still other embodiments, the image optimization processing algorithm includes a face beautifying algorithm variable, a style migration algorithm variable, and a color noise reduction algorithm variable, where the performing redundancy discrimination between the image optimization processing algorithm corresponding to a current optimized image frame in the plurality of live optimized images and the image optimization processing algorithm corresponding to each live optimized image except the current optimized image frame in the plurality of live optimized images includes: for each live broadcast optimized image except the current optimized image frame in a plurality of live broadcast optimized images, determining whether a face beautifying algorithm variable corresponding to the current optimized image frame is consistent with a face beautifying algorithm variable corresponding to the corresponding live broadcast optimized image; determining whether a style migration algorithm variable corresponding to the current optimized image frame is consistent with a style migration algorithm variable corresponding to the corresponding live broadcast optimized image; determining whether a color noise reduction algorithm variable corresponding to the current optimized image frame is consistent with a color noise reduction algorithm variable corresponding to the corresponding live broadcast optimized image; and taking the live broadcast optimized image corresponding to and consistent with the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable of the current optimized image frame as a live broadcast optimized image successfully distinguished with the redundancy of the current optimized image frame.
In still other embodiments, live image optimization systems are particularly concerned with several key variables in the image optimization processing algorithm when making redundancy decisions: the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable play a vital role in the image optimization process, and the final optimization effect of the image is directly affected by the arrangement of the variables.
Firstly, a live broadcast image optimization system selects a live broadcast optimized image as a current optimized image frame and uses the live broadcast optimized image as a reference for redundancy discrimination. Next, the system compares the consistency of the current optimized image frame with the other live optimized images one by one on the face beautifying algorithm variable. The face beautifying algorithm variable can relate to adjustment of face outline, improvement of skin texture and the like, and is a common technical means in live image optimization.
Second, the system checks the consistency of the style migration algorithm variables. The style migration algorithm variables determine the degree and effect of the style conversion of an image, which can convert the style of one image into another designated style, thereby enriching the visual effect of a live image.
Finally, the consistency of the color noise reduction algorithm variable is carefully checked by the system, and the variable is mainly used for removing noise points in the image and improving the color purity and definition of the image.
In the comparison process, if a certain live broadcast optimized image is consistent with the current optimized image frame in terms of the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable, the image can be regarded as being successfully distinguished from the current optimized image frame in a redundancy manner.
Through the careful comparison mode, the live image optimization system can accurately identify which images are redundant in optimization processing, so that unnecessary repeated optimization work is avoided, the processing efficiency is improved, and the diversity and quality of live images can be ensured. Meanwhile, the redundancy judging mechanism provides more flexible and intelligent optimization strategy selection for the live image optimizing system, so that the system can accurately optimize the image according to actual requirements.
Therefore, by comprehensively considering the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable, the live image optimization system can realize more accurate and efficient redundancy judgment. The technical innovation not only improves the optimization efficiency of the system, but also ensures the quality and diversity of the live broadcast images, thereby bringing more excellent and personalized live broadcast viewing experience for users.
Under some optional design ideas, the sampling operation is performed on each intermediate live broadcast optimized image by using at least one of an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or a superdivision state of each intermediate live broadcast optimized image, so as to obtain at least one target live broadcast optimized image to be processed, including: performing at least one type of operation in time sequence feature sampling operation or style updating sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed; the time sequence feature sampling operation is realized based on the image optimization processing algorithm corresponding to each intermediate live broadcast optimized image, and the style updating sampling operation is realized based on the superminute state of each intermediate live broadcast optimized image.
Under some optional design ideas, when the live image optimization system performs sampling operation, two different sampling modes are adopted: the method comprises a time sequence feature sampling operation and a style updating sampling operation, wherein the two sampling operations are performed based on the features of the intermediate live broadcast optimized image so as to ensure that the finally obtained target live broadcast optimized image to be processed has representativeness and optimized value.
Firstly, the time sequence feature sampling operation is realized based on the image optimization processing algorithm corresponding to each intermediate live broadcast optimization image. The system further analyzes the image optimization processing algorithms applied to each intermediate live optimized image, and particularly focuses on the timing characteristics of these algorithms during image processing. The time sequence characteristics can include the sequence of algorithm processing, the length of processing time, the stability of processing effect and the like. By comparing and analyzing these timing characteristics, the system can pick out an intermediate live broadcast optimized image that is excellent in the process flow and has a representative appearance as a target to be processed.
And secondly, the style updating sampling operation is realized based on the superminute state of each intermediate live broadcast optimized image. The super-resolution state refers to the definition and detail performance of the image after super-resolution processing. The system evaluates the superdivision state of each intermediate live broadcast optimized image, and particularly focuses on the visual effect and detail richness of the image after superdivision processing. By the sampling mode, the system can select the intermediate live broadcast optimized images with excellent performance and unique style after the super-division processing, so as to ensure that the final target live broadcast optimized images are representative in visual effect and style.
In actual operation, the live broadcast image optimization system can flexibly select the least one type of operation in the time sequence feature sampling operation or the style updating sampling operation according to actual requirements and application scenes, and the flexible selection mechanism enables the system to select the most suitable target live broadcast optimization image to be processed according to different conditions and requirements.
Through the combined application of the two sampling operations, the live broadcast image optimization system can ensure that the selected target live broadcast optimization image to be processed is excellent in processing flow, is representative in visual effect and style, and the comprehensive sampling strategy is beneficial to improving the optimization efficiency and the image quality of the system, so that more excellent live broadcast watching experience is brought to users. Meanwhile, the design thought also embodies the technical innovation and flexibility of the live image optimization system, so that the live image optimization system can adapt to different application scenes and requirements.
In other embodiments, the image optimization algorithm includes a face beautification algorithm variable, a style migration algorithm variable, and a color noise reduction algorithm variable, and the step of the timing feature sampling operation includes: determining interactive live broadcast images from the intermediate live broadcast optimized images; for each intermediate live broadcast optimized image except the interactive live broadcast image, determining whether a face beautifying algorithm variable corresponding to the interactive live broadcast image is consistent with a face beautifying algorithm variable corresponding to the corresponding intermediate live broadcast optimized image; determining whether the style migration algorithm variables corresponding to the interactive live broadcast image and the corresponding intermediate live broadcast optimized image are consistent; determining whether the color noise reduction algorithm variable corresponding to the interactive live broadcast image contains the color noise reduction algorithm variable corresponding to the corresponding intermediate live broadcast optimized image; and determining the intermediate live broadcast optimized image which corresponds to and is consistent with the face beautifying algorithm variable and the style migration algorithm variable of the interactive live broadcast image and is contained by the color noise reduction algorithm variable of the interactive live broadcast image as a time sequence associated live broadcast image, and cleaning the time sequence associated live broadcast image to obtain at least one target live broadcast optimized image to be processed.
It can be understood that when the live image optimization system performs the time sequence feature sampling operation, the live image optimization system can pay special attention to the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable in the image, the variables play a vital role in the image optimization process, and the setting of the variables can directly influence the optimization effect of the image.
First, the system determines an interactive live image from each intermediate live optimized image. Interactive live images may refer to those images with high interactivity with the viewer, such as a close-up of the anchor, pictures of the interactive links, etc., which have a high degree of attention in the live broadcast and are therefore selected as the basis for the timing feature sampling operation.
Next, the system compares the consistency of the interactive live image with other intermediate live optimized images on the face beautifying algorithm variable one by one. The face beautifying algorithm variable relates to adjustment of face outline, improvement of skin texture and the like, and is a common technical means in live image optimization. By comparison, the system can find out other images consistent with the interactive live image in the face beautifying process.
Second, the system checks the consistency of the style migration algorithm variables. The style migration algorithm variables determine the degree and effect of image style conversion, which is capable of converting the style of one image to another specified style. The system looks for other images that are similar in style migration processing to the interactive live image to ensure style diversity of the sampling results.
The system then focuses on the color noise reduction algorithm variables. The color noise reduction algorithm is mainly used for removing noise points in the image and improving the color purity and definition of the image. The system judges whether the color noise reduction algorithm variable of other intermediate live broadcast optimized images is contained by the color noise reduction algorithm variable of the interactive live broadcast image or not so as to find out the image similar to the interactive live broadcast image in color noise reduction processing.
In the comparison process, if a certain intermediate live broadcast optimized image is consistent with an interactive live broadcast image in terms of a face beautifying algorithm variable and a style migration algorithm variable, and the color noise reduction algorithm variable is contained by the color noise reduction algorithm variable of the interactive live broadcast image, the image is regarded as a time sequence-related live broadcast image, and the time sequence-related live broadcast image and the interactive live broadcast image have higher similarity in terms of optimization processing, so that the image can be regarded as a redundant image for cleaning.
By cleaning the time sequence-associated live broadcast images, the system can obtain at least one target live broadcast optimized image to be processed, wherein the target live broadcast optimized images are representative images in time sequence characteristics and can reflect key moments and highlight moments in the live broadcast process, so that the viewing experience of audiences is improved.
Therefore, by comprehensively considering the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable, the live image optimization system can realize more accurate time sequence feature sampling operation, and the technical innovation not only improves the optimization efficiency of the system, but also ensures the quality and diversity of live images. Meanwhile, through cleaning the time sequence associated live broadcast images, the system can further reduce the interference of redundant information, and brings clearer and smoother live broadcast watching experience for audiences.
In some alternative embodiments, the step of style update sampling operation includes: determining superprocessing iteration values of the intermediate live broadcast optimized images by using superdivision states of the intermediate live broadcast optimized images; and taking the intermediate live broadcast optimized image with the superdivision processing iteration value meeting the set iteration requirement as a target live broadcast optimized image.
Based on the embodiment, the live broadcast image optimization system pays special attention to the superminute state of each intermediate live broadcast optimized image when performing style updating sampling operation. The super-resolution state refers to the quality and definition level of the image after being processed by the super-resolution technology, and is an important index for evaluating the image optimization effect.
Firstly, the system analyzes the supersplit state of each intermediate live broadcast optimized image, specifically, the detail performance, edge sharpness and overall sharpness of the images after super-resolution processing are checked, and the information can help the system to know the performance of each image in the optimizing process and serve as the basis of subsequent sampling.
Next, the system determines a superdivision processing iteration value for each intermediate live broadcast optimized image according to the superdivision state, wherein the iteration value reflects the optimization degree and effect of the image in the super-resolution processing process, and the higher the numerical value is, the better the optimization effect of the image is.
After determining the oversubscription iteration values for each intermediate live optimized image, the system sets an iteration requirement, which may be a specific numerical range or a relative criterion, such as requiring that the iteration value must be above a certain threshold or within a certain interval.
And then, screening out intermediate live broadcast optimized images with superdivision processing iteration values meeting the set iteration requirements by the system, wherein the images show a good optimizing effect in the super-resolution processing process, so that the images are selected as target live broadcast optimized images.
Through the style updating sampling operation, the live broadcast image optimization system can ensure that the selected target live broadcast optimized image is representative in visual effect and style, and meanwhile, the quality and definition of the image are ensured.
Therefore, by focusing on the superdivision state and the superdivision processing iteration value of the middle live broadcast optimized image, the live broadcast image optimizing system can implement effective style updating sampling operation, and select the image with the best optimizing effect as the target live broadcast optimized image, so that the whole quality of the live broadcast image is improved, and clearer and vivid visual experience is provided for users.
In an alternative embodiment, the performing at least one of the time sequence feature sampling operation or the style updating sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed includes: performing time sequence feature sampling operation on the intermediate live broadcast optimized images by using an image optimization processing algorithm respectively corresponding to each intermediate live broadcast optimized image; performing style updating sampling operation on the middle live broadcast optimized images by using the superdivision state of each middle live broadcast optimized image; and determining at least one target live broadcast optimized image to be processed based on the live broadcast optimized image after the time sequence feature sampling operation and the live broadcast optimized image after the style updating sampling operation.
Based on the embodiment, the live image optimization system adopts a comprehensive sampling strategy which combines time sequence characteristic sampling operation and style updating sampling operation to ensure that the target live optimized image with the most representative and optimizing potential is selected.
Firstly, the system uses the image optimization processing algorithm corresponding to each intermediate live broadcast optimized image to sample the time sequence characteristics of the images, and the step mainly focuses on the time sequence characteristics of the images in the processing process, such as the application sequence of the algorithm, the processing time, the effect stability and the like. In this way, the system is able to identify images that perform well in the process flow with critical timing features.
Secondly, the system uses the superminute state of each intermediate live broadcast optimized image to carry out style updating sampling operation, wherein the superminute state refers to the definition and quality level of the image after super-resolution processing. The system evaluates the superminute state of each image, and in particular focuses on their visual effects and style characteristics after the super-resolution processing. In this way, the system can pick out images that are unique in visual effect and style and have a significant optimizing effect.
After the time sequence characteristic sampling operation and the style updating sampling operation are completed, the system determines at least one target live broadcast optimized image to be processed based on the results of the two sampling operations. Specifically, the system comprehensively considers the time sequence characteristics and the wind grid characteristics, and selects images which are excellent in both aspects as target images, wherein the images are not only representative in the processing flow, but also unique in the visual effect and style, so that the system has high optimizing potential.
Through the comprehensive sampling strategy, the live broadcast image optimization system can ensure that the selected target live broadcast optimized image not only has representativeness in time sequence characteristics, but also embodies unique style characteristics, thereby being beneficial to improving the optimization efficiency of the system, and ensuring that the optimized image reaches higher standards in quality and visual effect.
Therefore, by comprehensively considering the time sequence characteristics and the style characteristics, more comprehensive and more accurate sampling of the intermediate live broadcast optimized image is realized, the overall optimization effect of the live broadcast image is improved, and more colorful visual experience is brought to users.
In still other alternative embodiments, the performing at least one of a time sequence feature sampling operation or a style update sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed includes: performing time sequence feature sampling operation on the intermediate live broadcast optimized images by using an image optimization processing algorithm respectively corresponding to each intermediate live broadcast optimized image; and carrying out style updating sampling operation on the live broadcast optimized images after the time sequence feature sampling operation according to the superdivision state of each live broadcast optimized image after the time sequence feature sampling operation to obtain at least one target live broadcast optimized image to be processed.
Based on this embodiment, the live image optimization system performs a complex series of sampling operations to determine the target live optimized image with the most optimal potential.
Firstly, the system uses the image optimization processing algorithm corresponding to each intermediate live broadcast optimized image to sample the time sequence characteristics of the images, which means that the system deeply analyzes the algorithm used by each image in the processing process and considers how the algorithms influence the time sequence characteristics of the images. The purpose of the timing feature sampling operation is to identify images with critical time nodes or important algorithm applications in the process flow, which may be representative or specific in timing.
Next, the system evaluates the superminute state of each live optimized image after the timing characteristic sampling operation. The super-resolution state is an important index for measuring the definition and quality of an image after super-resolution processing. The system judges the optimizing effect and visual expressive force of each image according to the superdivision state of each image.
Then, based on the evaluation results of the superminute states, the system performs style updating sampling operation on the live broadcast optimized image after the time sequence feature sampling operation, and the aim of the step is to further screen out images which are not only representative in time sequence features, but also excellent in style and visual effect. The style update sampling operation comprehensively considers the superminute state and style characteristics of the images to ensure that the selected images have unique visual styles and optimized quality while maintaining the time sequence characteristics.
Finally, through the series of integrated sampling operations, the live image optimization system is able to determine at least one target live optimized image to be processed, which is not only representative in time sequence characteristics, but also meets higher standards in style and visual effect.
Therefore, by comprehensively utilizing the time sequence feature sampling operation and the style updating sampling operation, the high-efficiency screening of the intermediate live broadcast optimized image is realized, the optimized quality and visual effect of the live broadcast image are improved, the dual representativeness of the selected image in time sequence and style is ensured, and the strategy is favorable for presenting more wonderful and high-quality live broadcast content to users and improving the overall viewing experience.
In an alternative embodiment, the image optimization processing algorithm includes a face beautifying algorithm variable, a style migration algorithm variable, and a color noise reduction algorithm variable; the determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimizing processing algorithm and the image optimizing processing algorithm corresponding to each target live broadcast optimized image respectively includes: determining a first linkage live broadcast optimized image with a face beautifying algorithm variable consistent with a current face beautifying algorithm variable of an image optimizing client and a style migration algorithm variable consistent with a current style migration algorithm variable of the image optimizing client from at least one target live broadcast optimized image; for each first linkage live broadcast optimized image, determining the confidence weight of each noise suppression task in the color noise reduction algorithm variable corresponding to the current first linkage live broadcast optimized image based on the current optimizing processing label of the image optimizing client; and determining a core live broadcast optimized image from the first linkage live broadcast optimized image according to the confidence weight of each noise suppression task.
In the embodiment of the application, when the live broadcast image optimization system determines the core live broadcast optimized image, various algorithm variables and optimization processing labels are comprehensively considered, and the process involves various factors such as face beautifying algorithm variables, style migration algorithm variables, color noise reduction algorithm variables and the like.
First, the system screens out a first linked live optimized image in which the face beautification algorithm variable and the style migration algorithm variable are consistent with the current settings of the image optimization client from at least one target live optimized image, which means that these images are matched with the current requirements of the client in face beautification and style migration.
Specifically, the system checks whether the face beautification algorithm variable of each target live broadcast optimized image is consistent with the current face beautification algorithm variable of the image optimization client, and checks whether the style migration algorithm variable is consistent with the current style migration algorithm variable of the client. Only if both conditions are met will the image be selected as the first linked live optimized image.
Next, for each first linked live broadcast optimized image, the system determines a confidence weight for each noise suppression task in the color noise reduction algorithm variable corresponding to the current image based on the current optimization processing tag of the image optimization client. The optimization processing tag may include a variety of different noise suppression tasks such as removing noise, reducing color distortion, etc. The system analyzes the importance of each task and assigns a confidence weight to each noise suppression task that reflects the priority and importance of the task in the current optimization process.
And finally, according to the confidence weights of each noise suppression task, the system determines a core live broadcast optimized image from the first linkage live broadcast optimized image, and in the process, the system comprehensively considers the confidence weights of various noise suppression tasks and the performance of each image in terms of color noise reduction, so that the core live broadcast optimized image which best meets the current optimization requirement is selected.
Therefore, the live broadcast image optimization system can dynamically select the most suitable core live broadcast optimization image according to the current setting and optimization requirements of the client, the pertinence and the efficiency of image optimization are improved, and the optimization result is ensured to be more in line with the expectations and the requirements of users. Meanwhile, by comprehensively considering various algorithm variables and optimization processing labels, the system can evaluate the optimization potential of each image more comprehensively, so that the image with the highest quality is selected as the core live broadcast optimization image.
In other examples, the image optimization processing algorithm includes: face beautification algorithm variables, style migration algorithm variables and color noise reduction algorithm variables. The determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimization processing algorithm and the image optimization processing algorithm corresponding to each target live broadcast optimized image respectively includes: determining that a face beautifying algorithm variable is consistent with a current face beautifying algorithm variable of an image optimizing client from at least one target live broadcast optimized image, a style migration algorithm variable is consistent with a current style migration algorithm variable of the image optimizing client, and a second linkage live broadcast optimized image of a noise suppression task incompatible with a current optimizing processing label of the image optimizing client does not exist in a color noise reduction algorithm variable; and selecting the second linkage live broadcast optimized image with the largest superprocessing iteration value from the second linkage live broadcast optimized images as a core live broadcast optimized image.
Based on the example, the live image optimization system adopts a finer screening method when determining the core live optimized image, and the method relates to a plurality of factors such as a face beautifying algorithm variable, a style migration algorithm variable, a color noise reduction algorithm variable, an over-processing iteration value and the like.
Firstly, the system further screens out images of which the face beautifying algorithm variable is consistent with the current face beautifying algorithm variable of the image optimizing client and the style migration algorithm variable is consistent with the current style migration algorithm variable of the image optimizing client from at least one target live broadcast optimized image. This is to ensure that the selected image remains highly consistent with the client's current settings in terms of face beautification and style migration.
Then, the system continues to filter the filtered images to exclude images in which noise suppression tasks incompatible with the current optimization processing labels of the image optimization clients exist in the color noise reduction algorithm variables. This step is to ensure that the selected image also meets the client's specific requirements in terms of color noise reduction.
Through the two rounds of screening, the system obtains a group of second linkage live broadcast optimized images which not only meet the requirements of face beautification and style migration, but also meet the requirements of color noise reduction.
And finally, the system selects the image with the largest super-processing iteration value from the second linkage live broadcast optimized images as a core live broadcast optimized image. The super-resolution processing iteration value is an important index, and reflects the optimization degree and effect of the image in the super-resolution processing process. Selecting the image with the largest iteration value means that the system tends to select those images that undergo more optimization iterations, thereby achieving higher definition and quality as core live optimized images.
Therefore, the selected core live broadcast optimized image can be ensured to be highly matched with the current setting and optimizing requirements of the client in aspects of face beautification, style migration, color noise reduction and the like, and the definition and quality of the image can be further improved by selecting the image with the largest superdivision processing iteration value. The refined screening method enables the live image optimization system to provide more accurate and high-quality image optimization service for users.
It should be noted that the super-processing of the core live broadcast optimized image to adapt to the image quality optimization requirement corresponding to the target image optimization request may include the following contents:
obtaining image quality optimization parameters specified in a target image optimization request, wherein the parameters comprise, but are not limited to, resolution improvement times, sharpness enhancement levels, color saturation adjustment values and the like;
Determining an algorithm model and an optimization strategy of super-resolution processing based on the image quality optimization parameters, wherein the algorithm model can be a pre-trained deep learning model, such as a Convolutional Neural Network (CNN) or a generation countermeasure network (GAN), and is specially used for image super-resolution reconstruction;
Preprocessing the core live optimized image, including but not limited to denoising, contrast enhancement, and edge sharpening, to enhance the base quality of the image prior to the super processing;
Inputting the preprocessed core live broadcast optimized image into a selected superdivision processing algorithm model, reconstructing the super resolution according to the algorithm model and an optimization strategy, and adapting to the image quality optimization requirement of a target image optimization request by increasing the details and definition of the image;
in the super-processing process, the processing effect and performance are monitored in real time, and parameters and settings of an algorithm model are dynamically adjusted to ensure that the processing result meets the expected image quality optimization requirement;
after the super-resolution processing is completed, post-processing is carried out on the generated super-resolution image, including color correction, contrast fine adjustment and the like, so as to further improve the visual effect of the image;
and finally outputting the processed super-resolution image to meet the image quality optimization requirement corresponding to the target image optimization request, and providing the image quality optimization requirement for a user.
It can be understood that when the live broadcast image optimization system performs super-division processing on the core live broadcast optimized image, a series of fine steps are performed to adapt to the image quality optimization requirement corresponding to the target image optimization request.
First, the system acquires image quality optimization parameters explicitly specified in the target image optimization request. These parameters are critical because they directly determine the goal and effect of the superdivision process. For example, the resolution improvement factor indicates how much an image needs to be magnified, the sharpness enhancement level determines the sharpness of the edges of the image, and the color saturation adjustment value affects the vividness of the image color.
Next, based on these image quality optimization parameters, the system determines the most appropriate superdivision processing algorithm model and optimization strategy. This algorithmic model may be a pre-trained deep learning model, such as a Convolutional Neural Network (CNN) or a Generative Antagonism Network (GAN), which performs well in terms of image super-resolution reconstruction.
Before formally performing the super-processing, the system also performs necessary preprocessing on the core live optimized image. The aim of this step is to improve the basic quality of the image, laying a good foundation for the subsequent superprocessing. The preprocessing operations may include denoising to eliminate spurious noise in the image, contrast enhancement to enhance the bright-dark contrast of the image, and edge sharpening to enhance the contours and details of the image.
After the preprocessing is completed, the system inputs the processed core live broadcast optimized image into a superdivision processing algorithm model selected before. According to the algorithm model and the optimization strategy, the system performs super-resolution reconstruction, namely, the image quality optimization requirement of the target image optimization request is adapted by increasing the details and the definition of the image.
In the whole super-processing process, the system monitors the processing effect and performance in real time so as to discover and solve the problems in time. If the processing result is found to be inconsistent with the expected image quality optimization requirement, the system dynamically adjusts parameters and settings of the algorithm model to ensure that the final processing result can meet the requirement of a user.
After the super-resolution processing is completed, the system also performs post-processing on the generated super-resolution image. This step is mainly to further enhance the visual effect of the image, and may include color correction to adjust the color balance of the image, fine contrast adjustment to optimize the shading level of the image, and the like.
Finally, after the series of fine processing steps, the system outputs the processed super-resolution image to meet the image quality optimization requirement corresponding to the target image optimization request and provide the image quality optimization requirement for the user. In this way, the live image optimization system is able to provide high quality, personalized image optimization services to users.
Therefore, the image quality can be accurately optimized according to the specific requirements of users, and super-resolution reconstruction and detail enhancement of the image can be realized through the strong capability of the deep learning model. Meanwhile, the system can ensure that the processing result always meets the expected requirement through real-time monitoring and adjusting the processing process. This provides a more flexible and efficient image optimization solution for the user.
In other independent embodiments, after the core live broadcast optimized image is subjected to the super-processing to adapt to the image quality optimization requirement corresponding to the target image optimization request, the method further includes: and processing the core live broadcast optimized image subjected to the superdivision processing according to the special effect configuration requirement to obtain the special effect live broadcast image.
In detail, the processing the core live broadcast optimized image after the superdivision processing according to the special effect configuration requirement to obtain the special effect live broadcast image comprises the following steps: acquiring pixel points, pixel clusters and image block layers in the core live broadcast optimized image subjected to superdivision processing based on the special effect configuration requirements; generating a hierarchical special effect pyramid, wherein the hierarchical special effect pyramid comprises mapping units corresponding to the pixel points, the pixel clusters and the image block layers, and the mapping units corresponding to the pixel points, the pixel clusters and the image block layers with the involved states are connected through directional pointers; performing cross vector mining processing on an upstream mapping unit and a downstream mapping unit based on the hierarchical special effect pyramid to obtain a mapping unit special effect matching vector of a mapping unit in the hierarchical special effect pyramid; and carrying out special effect configuration on an object to be configured in the core live broadcast optimized image subjected to super-division processing based on the special effect matching vector of the mapping unit in the hierarchical special effect pyramid to obtain a special effect live broadcast image, wherein the object to be configured comprises at least one of the pixel points, the pixel clusters and the image block layers.
In the live broadcast image optimization system, the super-resolution processing of the core live broadcast optimized image is a key link, and the resolution and the definition of the image can be effectively improved, so that the requirements of audience on high-definition image quality are met. However, the functionality of the system is not limited thereto, and in order to meet a more diversified visual experience, the system also provides the functionality of special effects configuration.
After the over-processing of the core live broadcast optimized image is completed, the system further processes the image according to the preset or user-defined special effect configuration requirement. This process is very elaborate and technically powerful to implement.
Firstly, the system acquires each element in the core live broadcast optimized image which is subjected to super-division processing, wherein the elements comprise pixel points, pixel clusters and image block layers. These are basic units constituting an image, and various complex image effects can be achieved by processing them.
Next, the system generates a hierarchical effect pyramid. This pyramid is a data structure that contains mapping units corresponding to the elements in the image. The mapping units are arranged hierarchically in the pyramid, each layer representing a different level of special effects processing. In particular, those having inter-related pixel points, pixel clusters, and tile layers, the mapping units in the pyramid are connected by directional pointers, which means that they need to take into account each other's interactions in the effect processing.
The system then performs in-depth data mining on this hierarchical effect pyramid. By means of cross vector mining between the upstream and downstream mapping units, the system can find the best matching mode in special effect processing between the elements, namely the so-called mapping unit special effect matching vector. The vector not only considers the special effect requirement of single elements, but also considers the mutual influence among the elements, and ensures the harmony and consistency of the whole special effect.
And finally, carrying out special effect configuration on the object to be configured in the image by the system according to the mapping unit special effect matching vector obtained by mining. The object to be configured here may be any one or more of a pixel point, a pixel cluster or a tile layer. Through accurate special effect configuration, the system can generate special effect live broadcast images with unique visual effects, and more colorful visual experience is brought to audiences.
Therefore, the live image optimization system not only improves the image quality of live images, but also increases the visual effect of the images through the two steps of superdivision processing and special effect configuration. The processing mode not only maintains the original information of the image, but also increases the ornamental and artistic properties of the image through special effect configuration, so that the live image is more vivid and interesting.
It is worth mentioning that, based on the mapping unit special effect matching vector of the mapping unit in the hierarchical special effect pyramid, special effect configuration is performed on the object to be configured in the core live broadcast optimized image after the super-division processing, so as to obtain a special effect live broadcast image, including:
Determining special effect processing priority and special effect type of an object to be configured (comprising pixel points, pixel clusters or image block layers) according to the special effect matching vector of the mapping unit in the hierarchical special effect pyramid;
For each object to be configured, sequentially performing special effect configuration according to the special effect processing priority of the object to be configured; in the configuration process, special effect transformation is carried out on the object by utilizing special effect parameters related to the object in the special effect matching vector of the mapping unit;
Special effect transformations may include, but are not limited to, color adjustment, shape transformation, dynamic effect addition, etc., to achieve a rich and varied visual effect; meanwhile, according to the directed pointer relation between the mapping units, the coordination and consistency of the special effect among different objects are ensured;
when special effect configuration is carried out, the processing performance and effect are monitored in real time, and special effect parameters and configuration strategies are dynamically adjusted according to the monitoring result, so that the finally generated special effect live image can meet the requirement of visual effect, and higher processing efficiency can be maintained;
after the special effect configuration of all the objects to be configured is completed, the overall effect evaluation and optimization are carried out on the generated special effect live image so as to ensure the perfect combination of the image quality and the special effect;
and finally outputting the live image subjected to special effect configuration, and providing live content with unique visual experience for the user.
In the embodiment of the application, after the superdivision processing of the core live broadcast optimized image is completed, the live broadcast image optimization system can enter a special effect configuration stage, and the aim of the stage is to add various visual special effects for the image based on the hierarchical special effect pyramid so as to provide richer and more attractive live broadcast content.
First, the system refers to the mapping unit effect matching vectors of the mapping units in the hierarchical effect pyramid. These vectors have been derived by previous data mining processes, which not only indicate the type of special effects that should be applied to each object to be configured in the image (e.g., pixel point, pixel cluster, or tile layer), but also determine the priority of the special effects process. This means that the system first handles those objects that have the greatest impact on the overall visual effect.
And then, the system carries out special effect configuration on the objects to be configured one by one according to the priority order. In this process, the system uses the special effect parameters associated with the object in the mapping unit special effect matching vector to perform special effect transformation. These transforms may include color adjustments to change the hue, saturation, and brightness of the object; shape transformation such as twisting, stretching or compressing; and the addition of dynamic effects such as blinking, fading, or animation. These transformations aim to create attractive and distinctive visual effects.
Meanwhile, the system ensures the coordination consistency of the special effects among different objects by utilizing the directed pointer relation among the mapping units in the hierarchical special effect pyramid. This means that when the special effect of one object changes, the other objects associated with it will also adjust their special effects accordingly to maintain harmonious unification of the overall visual effect.
In the special effect configuration process, the system monitors the processing performance and effect in real time. If the processing speed is reduced or the special effect is not ideal, the system dynamically adjusts special effect parameters and configuration strategies. For example, it may reduce the complexity of certain effects, to increase processing speed, or adjust the intensity and range of effects to achieve a more desirable visual effect.
And after all the objects to be configured complete special effect configuration, the system carries out overall effect evaluation and tuning on the generated special effect live image. The purpose of this step is to ensure that a perfect combination of image quality and special effects is achieved. If the system detects any unsatisfied place, it will fine tune until the desired effect is achieved.
Finally, the live image with the careful special effect configuration is output and presented to the audience. These images not only have high definition, but also incorporate various attractive special effects, providing a unique and impressive visual experience for the viewer.
Therefore, the live image optimization system realizes the precise special effect configuration of the live image through the application of the grading special effect pyramid and the mapping unit special effect matching vector. The processing mode not only improves the visual effect of the image, but also ensures the high efficiency of processing and the stability of the image quality. Thus, the viewer can enjoy more colorful, attractive live content.
Further, fig. 2 is a schematic structural diagram of a live image optimization system 200 according to an embodiment of the present application. The live image optimization system 200 as shown in fig. 2 includes a processor 210, and the processor 210 may call and run a computer program from a memory to implement the method in an embodiment of the present application.
Optionally, as shown in fig. 2, the live image optimization system 200 may also include a memory 230. Wherein the processor 210 may call and run a computer program from the memory 230 to implement the method in an embodiment of the application.
Wherein the memory 230 may be a separate device from the processor 210 or may be integrated into the processor 210.
Optionally, as shown in fig. 2, the live image optimization system 200 may further include a transceiver 220, and the processor 210 may control the transceiver 220 to interact with other devices, and in particular, may send information or data to other devices, or receive information or data sent by other devices.
Optionally, the live image optimization system 200 may implement the storage engine or a component (such as a processing module) in the storage engine or a corresponding flow corresponding to a device in which the storage engine is deployed in each method of the embodiments of the present application, which is not described herein for brevity.
It should be appreciated that the processor of an embodiment of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The Processor may be a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an Application SPECIFIC INTEGRATED Circuit (ASIC), an off-the-shelf programmable gate array (Field Programmable GATE ARRAY, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
It will be appreciated that the memory in embodiments of the application may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (STATIC RAM, SRAM), dynamic random access memory (DYNAMIC RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate Synchronous dynamic random access memory (Double DATA RATE SDRAM, DDR SDRAM), enhanced Synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and Direct memory bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
It should be appreciated that the above memory is exemplary but not limiting, and for example, the memory in the embodiments of the present application may also be static random access memory (STATIC RAM, SRAM), dynamic random access memory (DYNAMIC RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (doubledata RATE SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCH LINK DRAM, SLDRAM), direct Rambus RAM (DR RAM), and the like. That is, the memory in embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.
On the basis of the above, a computer readable storage medium is provided, on which a computer program is stored, which computer program, when run, implements the method described above.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art.
Claims (9)
1. A live image optimization method based on artificial intelligence, characterized in that it is applied to a live image optimization system, the method comprising:
Acquiring a plurality of live broadcast optimized images obtained by operating the target live broadcast images in a quality optimized state according to target optimization indication features; the live broadcast optimized images are generated through at least two image optimization processing algorithms;
Based on an image optimization processing algorithm corresponding to each live broadcast optimized image, comprehensively evaluating and sampling the live broadcast optimized images to obtain at least one target live broadcast optimized image to be processed;
if a target image optimization request matched with the target optimization indicating characteristic is obtained under the condition that the target live image is in a quality optimization state, determining a current image optimization processing algorithm;
Determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimized processing algorithm and the image optimized processing algorithm respectively corresponding to each target live broadcast optimized image;
performing super-processing on the core live broadcast optimized image to adapt to the image quality optimization requirement corresponding to the target image optimization request;
The step of comprehensively evaluating and sampling the plurality of live broadcast optimized images based on the image optimization processing algorithm respectively corresponding to each live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed, comprises the following steps:
Searching redundant image contents from a plurality of live broadcast optimized images through an image optimization processing algorithm respectively corresponding to each live broadcast optimized image;
performing cleaning treatment on redundant image content in the live broadcast optimized images to obtain at least one style of intermediate live broadcast optimized image;
Sampling each intermediate live broadcast optimized image by using at least one of an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or the superdivision state of each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed;
the method for searching redundant image content from a plurality of live broadcast optimized images through an image optimization processing algorithm respectively corresponding to each live broadcast optimized image comprises the following steps:
The method comprises the steps of enabling an image optimization processing algorithm corresponding to a current optimized image frame in a plurality of live optimized images to realize redundancy discrimination with an image optimization processing algorithm corresponding to each live optimized image except the current optimized image frame in the plurality of live optimized images;
When a live broadcast optimized image which is successfully distinguished from the redundancy of the current optimized image frame exists, taking the current optimized image frame as redundant image content;
The image optimization processing algorithm comprises a face beautifying algorithm variable, a style migration algorithm variable and a color noise reduction algorithm variable, wherein the image optimization processing algorithm corresponding to a current optimized image frame in a plurality of live broadcast optimized images and the image optimization processing algorithm corresponding to each live broadcast optimized image except the current optimized image frame in the plurality of live broadcast optimized images realize redundancy discrimination, and the method comprises the following steps:
for each live broadcast optimized image except the current optimized image frame in a plurality of live broadcast optimized images, determining whether a face beautifying algorithm variable corresponding to the current optimized image frame is consistent with a face beautifying algorithm variable corresponding to the corresponding live broadcast optimized image;
Determining whether a style migration algorithm variable corresponding to the current optimized image frame is consistent with a style migration algorithm variable corresponding to the corresponding live broadcast optimized image;
determining whether a color noise reduction algorithm variable corresponding to the current optimized image frame is consistent with a color noise reduction algorithm variable corresponding to the corresponding live broadcast optimized image;
and taking the live broadcast optimized image corresponding to and consistent with the face beautifying algorithm variable, the style migration algorithm variable and the color noise reduction algorithm variable of the current optimized image frame as a live broadcast optimized image successfully distinguished with the redundancy of the current optimized image frame.
2. The method according to claim 1, wherein the sampling operation is performed on each intermediate live broadcast optimized image by using at least one of an image optimization processing algorithm corresponding to each intermediate live broadcast optimized image or a superdivision state of each intermediate live broadcast optimized image, to obtain at least one target live broadcast optimized image to be processed, including:
performing at least one type of operation in time sequence feature sampling operation or style updating sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed;
the time sequence feature sampling operation is realized based on the image optimization processing algorithm corresponding to each intermediate live broadcast optimized image, and the style updating sampling operation is realized based on the superminute state of each intermediate live broadcast optimized image.
3. The method of claim 2, wherein the image optimization processing algorithm includes a face beautification algorithm variable, a style migration algorithm variable, and a color noise reduction algorithm variable, and wherein the step of timing feature sampling operation includes:
Determining interactive live broadcast images from the intermediate live broadcast optimized images;
for each intermediate live broadcast optimized image except the interactive live broadcast image, determining whether a face beautifying algorithm variable corresponding to the interactive live broadcast image is consistent with a face beautifying algorithm variable corresponding to the corresponding intermediate live broadcast optimized image;
determining whether the style migration algorithm variables corresponding to the interactive live broadcast image and the corresponding intermediate live broadcast optimized image are consistent;
determining whether the color noise reduction algorithm variable corresponding to the interactive live broadcast image contains the color noise reduction algorithm variable corresponding to the corresponding intermediate live broadcast optimized image;
And determining the intermediate live broadcast optimized image which corresponds to and is consistent with the face beautifying algorithm variable and the style migration algorithm variable of the interactive live broadcast image and is contained by the color noise reduction algorithm variable of the interactive live broadcast image as a time sequence associated live broadcast image, and cleaning the time sequence associated live broadcast image to obtain at least one target live broadcast optimized image to be processed.
4. The method of claim 2, wherein the step of style update sampling operation comprises:
determining superprocessing iteration values of the intermediate live broadcast optimized images by using superdivision states of the intermediate live broadcast optimized images;
And taking the intermediate live broadcast optimized image with the superdivision processing iteration value meeting the set iteration requirement as a target live broadcast optimized image.
5. The method according to claim 2, wherein performing at least one of a time sequence feature sampling operation or a style update sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed comprises:
Performing time sequence feature sampling operation on the intermediate live broadcast optimized images by using an image optimization processing algorithm respectively corresponding to each intermediate live broadcast optimized image;
Performing style updating sampling operation on the middle live broadcast optimized images by using the superdivision state of each middle live broadcast optimized image;
And determining at least one target live broadcast optimized image to be processed based on the live broadcast optimized image after the time sequence feature sampling operation and the live broadcast optimized image after the style updating sampling operation.
6. The method according to claim 2, wherein performing at least one of a time sequence feature sampling operation or a style update sampling operation on each intermediate live broadcast optimized image to obtain at least one target live broadcast optimized image to be processed comprises:
Performing time sequence feature sampling operation on the intermediate live broadcast optimized images by using an image optimization processing algorithm respectively corresponding to each intermediate live broadcast optimized image;
And carrying out style updating sampling operation on the live broadcast optimized images after the time sequence feature sampling operation according to the superdivision state of each live broadcast optimized image after the time sequence feature sampling operation to obtain at least one target live broadcast optimized image to be processed.
7. The method of claim 1, wherein the image optimization processing algorithm comprises a face beautification algorithm variable, a style migration algorithm variable, and a color noise reduction algorithm variable;
The determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimizing processing algorithm and the image optimizing processing algorithm corresponding to each target live broadcast optimized image respectively includes:
Determining a first linkage live broadcast optimized image with a face beautifying algorithm variable consistent with a current face beautifying algorithm variable of an image optimizing client and a style migration algorithm variable consistent with a current style migration algorithm variable of the image optimizing client from at least one target live broadcast optimized image;
For each first linkage live broadcast optimized image, determining the confidence weight of each noise suppression task in the color noise reduction algorithm variable corresponding to the current first linkage live broadcast optimized image based on the current optimizing processing label of the image optimizing client;
and determining a core live broadcast optimized image from the first linkage live broadcast optimized image according to the confidence weight of each noise suppression task.
8. The method of claim 1, wherein the image optimization processing algorithm comprises: a face beautification algorithm variable, a style migration algorithm variable and a color noise reduction algorithm variable;
The determining a core live broadcast optimized image from the at least one target live broadcast optimized image based on the current image optimizing processing algorithm and the image optimizing processing algorithm corresponding to each target live broadcast optimized image respectively includes:
Determining that a face beautifying algorithm variable is consistent with a current face beautifying algorithm variable of an image optimizing client from at least one target live broadcast optimized image, a style migration algorithm variable is consistent with a current style migration algorithm variable of the image optimizing client, and a second linkage live broadcast optimized image of a noise suppression task incompatible with a current optimizing processing label of the image optimizing client does not exist in a color noise reduction algorithm variable;
And selecting the second linkage live broadcast optimized image with the largest superprocessing iteration value from the second linkage live broadcast optimized images as a core live broadcast optimized image.
9. A live image optimization system comprising at least one processor and memory; the memory stores computer-executable instructions; the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410643084.4A CN118233675B (en) | 2024-05-23 | 2024-05-23 | Live image optimization method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410643084.4A CN118233675B (en) | 2024-05-23 | 2024-05-23 | Live image optimization method and system based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118233675A CN118233675A (en) | 2024-06-21 |
| CN118233675B true CN118233675B (en) | 2024-07-19 |
Family
ID=91498253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410643084.4A Active CN118233675B (en) | 2024-05-23 | 2024-05-23 | Live image optimization method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118233675B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118590602A (en) * | 2024-07-30 | 2024-09-03 | 山东浪潮超高清智能科技有限公司 | Method and system applied to video image quality enhancement and ultra-high resolution |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934988A (en) * | 2022-12-28 | 2023-04-07 | 广州方硅信息技术有限公司 | Live cover image extraction method, device, equipment and medium |
| CN117896540A (en) * | 2023-12-19 | 2024-04-16 | 广州酷狗计算机科技有限公司 | Image quality optimization method and device applied to live broadcast system, terminal and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111355967A (en) * | 2020-03-11 | 2020-06-30 | 叠境数字科技（上海）有限公司 | Video live broadcast processing method, system, device and medium based on free viewpoint |
| CN116320597A (en) * | 2022-09-06 | 2023-06-23 | 北京字跳网络技术有限公司 | Live image frame processing method, device, equipment, readable storage medium and product |
-
2024
- 2024-05-23 CN CN202410643084.4A patent/CN118233675B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934988A (en) * | 2022-12-28 | 2023-04-07 | 广州方硅信息技术有限公司 | Live cover image extraction method, device, equipment and medium |
| CN117896540A (en) * | 2023-12-19 | 2024-04-16 | 广州酷狗计算机科技有限公司 | Image quality optimization method and device applied to live broadcast system, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118233675A (en) | 2024-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN118233675B (en) | Live image optimization method and system based on artificial intelligence | |
| Rana et al. | Deep tone mapping operator for high dynamic range images | |
| US8606042B2 (en) | Blending of exposure-bracketed images using weight distribution functions | |
| CN108347643B (en) | Subtitle superposition screenshot realization method based on deep learning | |
| CN107464217B (en) | Image processing method and device | |
| CN109685726A (en) | Scene of game processing method, device, electronic equipment and storage medium | |
| JP2018101406A (en) | Using image analysis algorithm for providing training data to neural network | |
| CN110717868B (en) | Video high dynamic range inverse tone mapping model construction and mapping method and device | |
| CN113284064B (en) | Cross-scale context low-illumination image enhancement method based on attention mechanism | |
| CN110795858A (en) | Method and device for generating home decoration design drawing | |
| CN113297937B (en) | Image processing method, device, equipment and medium | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN112200736A (en) | Image processing method based on reinforcement learning and model training method and device | |
| CN116630464A (en) | Image style migration method and device based on stable diffusion | |
| CN109618228A (en) | Video source modeling control method, device and electronic equipment | |
| CN111524060B (en) | System, method, storage medium and device for blurring portrait background in real time | |
| Zhang et al. | A real-time semi-supervised deep tone mapping network | |
| CN111415319A (en) | Video image restoration method based on time sequence network prediction and pyramid fusion | |
| CN110232730A (en) | A kind of three-dimensional face model textures fusion method and computer-processing equipment | |
| CN110689478B (en) | Image stylization processing method and device, electronic equipment and readable medium | |
| CN117058598B (en) | Cloud video frame high-quality optimization method and device and computer equipment | |
| CN117611467A (en) | Low-light image enhancement method capable of balancing details and brightness of different areas simultaneously | |
| US20120301042A1 (en) | Image processing apparatus and program | |
| Unemi | SBArt4 for an automatic evolutionary art | |
| US12094089B2 (en) | Intelligent video enhancement system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |