EP3195202A1 - Système et procédé de restauration de plage dynamique d'images et de vidéos - Google Patents

Système et procédé de restauration de plage dynamique d'images et de vidéos

Info

Publication number
EP3195202A1
EP3195202A1 EP14901826.9A EP14901826A EP3195202A1 EP 3195202 A1 EP3195202 A1 EP 3195202A1 EP 14901826 A EP14901826 A EP 14901826A EP 3195202 A1 EP3195202 A1 EP 3195202A1
Authority
EP
European Patent Office
Prior art keywords
media content
dimensional
data set
vector data
dimensional vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP14901826.9A
Other languages
German (de)
English (en)
Inventor
William L. Gaddy
Vidhya Seran
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
A2Zlogix Inc
Original Assignee
A2Zlogix Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by A2Zlogix Inc filed Critical A2Zlogix Inc
Publication of EP3195202A1 publication Critical patent/EP3195202A1/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform

Definitions

  • Embodiments of the present disclosure relate to image processing, and, more particularly, to the field of restoration of dynamic range within video or images without directly coupling to an upstream encoding or decoding process.
  • a common artifact typically present in statistical, predictive, and perceptual image compression systems is a reduction of color or brightness dynamic range. This is intentional, in most cases, since the quantization applied to reduce the dynamic range is not meant to be perceivable by most humans under normal viewing conditions.
  • typical encoders— and accordingly their matched decoders - introduce several objectionable artifacts at the final viewing end under these constrained conditions.
  • U.S. Patent Application Publication No. 2014/0079335 discloses a high-dynamic- range imaging system that analyses relative exposure levels by using motion analysis. However, this approach only works for video and moving imagery, and is not suitable for single images.
  • U.S. Patent Application Publication No. 2013/0107956 discloses a method to generate high dynamic range images from low dynamic range images.
  • the method disclosed in the '956 publication employs a predictive mapping that is closely coupled to the image encoder and decoder, which requires changes to an end-user device decoder. This renders the method disclosed in the '956 publication infeasible to employ for most mobile applications.
  • the method disclosed in the '956 publication requires high-dynamic range reference imagery such as YUV444, which is usually not available.
  • U.S. Patent Application Publication No. 2014/0177706 discloses a method and system that provides super-resolution of quantized images and video. While the method disclosed in the '706 publication could be altered to restore dynamic range instead of spatial resolution, the method disclosed in the '706 publication is similarly closely coupled to the encoder and decoder processes, rendering the method disclosed in the '706 publication infeasible for mobile applications.
  • U.S. Patent No. 8,582,913 discloses a post-processing controller to generate high-dynamic range images from low dynamic range images. While the controller as disclosed is amenable to real-time operations, it uses a brightness enhancement function tied to the angular spatial frequency represented by features in an image, which requires that the disclosed process have information about the display width and height, and the intended distance of the viewer from the display, limiting its application to professional applications and rendering the method disclosed in the '913 patent unsuitable for general consumer applications.
  • the method disclosed in the '913 patent uses simple thresholds in its masking operation, meaning that it is prone to false positives such as smoothing areas with no banding but legitimate image detail, and false negatives such as not eliminating contouring artifacts where they should be treated.
  • Such banding and contouring artifacts may be treated in the frequency domain by selective convolution, however the process of converting to and from the frequency domain renders selective convolution ineffective for processing full-resolution imagery on contemporaneous mobile consumer devices.
  • the foregoing methods are generally applied to a single purpose, namely the reduction of image banding and contouring artifacts due to limited dynamic range of digital sensors or compression-related quantization.
  • the foregoing methods have no general applicability to other related problems in the art such as quantization errors of vertex location of 3D models, and the restoration of dynamic range of other D-dimensional datasets.
  • a media content processor of a processing device may generate a Gaussian pyramid from input image data or video data.
  • the media content processor may generate a structure tensor field comprising a set of structure tensors for each pixel of the input image data or video data.
  • the media content processor may compute an eigensystem analysis for each of the structure tensors of each pixel of the input image data or video data, resulting in a plurality of eigenvalue fields.
  • the media content processor may generate a Gaussian pyramid from each of the plurality of eigenvalue fields.
  • the color channel processor may select one or more of the eigenvalues of each element of the lowest resolution level of computed Gaussian pyramids from each of the eigenvalue fields to assign a linear weight to each pixel of the lowest resolution Gaussian pyramid computed from each of the eigenvalue fields.
  • the color channel processor may interpolate, using a corresponding linear weight, between (1) each pixel of the lowest resolution pyramid of the Gaussian pyramid computed from the input image data or video data, and (2) a corresponding pixel of the plurality of pixels of the input image data or video data using a corresponding computed weight to produce a restored dynamic range image or video.
  • the color channel processor may display the restored dynamic range image data or video data on a display or transmit the restored dynamic range image or video to one or more downstream devices.
  • the linear weighting may be scaled according to a region-of-support used for the structure tensor analysis.
  • the structure tensor analysis may be performed with only one channel - such as the luminance of each pixel of an input image— while the weighted interpolation may jointly be performed on all channels of a multichannel image — such as all 3 channels (luminance and Cb, Cr) of a YCbCr image.
  • the input image Gaussian pyramid and the eigenvalue field Gaussian pyramids may be calculated in parallel.
  • a media content processor of a processing device may receive, from an upstream device, a D-dimensional vector data set of media content, wherein D is greater than or equal to one.
  • the media content processor may apply a low-frequency-pass method to the received D-dimensional vector data set of media content to generate a low-frequency D-dimensional vector data set of media content.
  • the media content processor may obtain a structure tensor field comprising a set of D- dimensional structure tensors corresponding to each vector of the received D-dimensional vector data set of media content.
  • the media content processor may perform an eigensystem analysis for each structure tensor in the field of structure tensors to generate a plurality of D eigenvalue fields comprising D eigenvalues for each vector of the received D-dimensional vector data set of media content.
  • the media content processor may interpolate between each vector of the received D-dimensional vector data set of media content and a corresponding vector from the low-frequency D-dimensional vector data set of media content in view of one or more of the D eigenvalues for each vector of the plurality of D eigenvalue fields to produce a restored-dynamic range D-dimensional vector data set of media content.
  • the media content processor may output the restored-dynamic range D-dimensional vector data set of media content to a display or downstream device.
  • the media content may comprises at least one of audio, one or more two-dimensional still images, one or more three-dimensional still images, two-dimensional video, three-dimensional video, magnetic resonance imaging (MRI) data, computed tomography (CT)-scan data, geometric data, seismic data, holographic data, or meteorological data.
  • MRI magnetic resonance imaging
  • CT computed tomography
  • the low-frequency-pass method may comprise the media content processor generating a D-dimensional Gaussian pyramid from the received D-dimensional vector data set of media content and selecting D-dimensional vector data from the lowest dimension pyramid level of the D-dimensional Gaussian pyramid to generate the low- frequency D-dimensional vector data set of media content.
  • the media content processor obtaining a set of D-dimensional structure tensors may comprise the media content processor computing a set of D- dimensional directional gradients for each vector of the received D-dimensional vector data set of media content.
  • the media content processor obtaining a set of D- dimensional structure tensors may comprise the media content processor performing a structure tensor analysis with one element of each vector of the received D-dimensional vector data set of media content.
  • interpolating may comprise the media content processor linearly interpolating between each vector of the received D-dimensional vector data set of media content and the corresponding vector from the low-frequency D-dimensional vector data set of media content.
  • interpolating may comprise the media content processor generating a Gaussian pyramid from each of the plurality of D eigenvalue fields.
  • the media content processor may further employ one or more of the D eigenvalues to assign linear weights to each vector of the lowest resolution plurality of D eigenvalue fields.
  • the media content processor may further employ the linear weights to interpolate between each vector of the received D-dimensional vector data set of media content and the corresponding vector from the low-frequency D-dimensional vector data set of media to produce the restored- dynamic range D-dimensional vector data set of media content.
  • the linear weights may be assigned in view of a region-of-support used for obtaining the structure tensor field.
  • the media content processor employing one or more of the D eigenvalues to assign linear weights to each vector of the lowest resolution plurality of D eigenvalue fields may comprise the media content processor selecting an eigenvalue of the D eigenvalues indicative of a homogenous region of support about a vector of the received D-dimensional vector data as the lower bound for assigning the linear weights.
  • the media content processor may further select one of the remaining eigenvalues of the D eigenvalues indicative of an edge or highly textured region of support about a vector of the received D- dimensional vector data as the upper bound for assigning the linear weights.
  • interpolating may comprise the media content processor performing an interpolation jointly between all element of each vector of the received D-dimensional vector data set of media content and the corresponding elements of each vector from the low-frequency D-dimensional vector data set of media content.
  • the interpolating and applying a low-frequency-pass method may be performed in parallel.
  • each vector of the received D-dimensional vector data of media content, the low-frequency D-dimensional vector data set of media content, and the restored- dynamic range D-dimensional vector data of media content may each comprise N- dimensional scalar values where N is equal to or greater than 1.
  • the received D-dimensional vector data set of media content, the low-frequency D-dimensional vector data set of media content, and the restored-dynamic range D-dimensional vector data set of media content may each comprise D-dimensional pixel values.
  • the pixel values may comprise color values or luminance values or both color values and luminance values.
  • Figure 1 is a block diagram of an example computing system that receives and restores the dynamic range of images and video without close coupling to an upstream encoding or decoding process in which examples of the present disclosure may operate.
  • Figure 2 is a flow diagram illustrating an example of a method for restoration of the dynamic range of images or video without close coupling to an upstream encoding, decoding, or quantization process using the system of Figure 1.
  • Figure 3 is a block diagram of another example computing system that receives and restores the dynamic range of D-dimensional vector data set of media content, wherein D is greater than or equal to one, without close coupling to an upstream encoding or decoding process in which examples of the present disclosure may operate.
  • Figure 4 is a flow diagram illustrating one example of a method for restoration of the dynamic range of D-dimensional media content without close coupling to an upstream encoding, decoding, or quantization process using the system of Figure 3.
  • Figure 5 shows an example input image.
  • Figure 6 shows a schematic of a constructed Gaussian Pyramid.
  • Figure 7 shows an example Gaussian Pyramid computed from the example input image of Figure 6.
  • Figure 8 shows a computed gradient field (dX, dY) from the example input image of Figure 6.
  • Figure 10 shows the two eigenvalue fields (lambdai and lambda 2 ) computed from the structure tensors of the example input image of Figure 6, normalized for visualization in the figure.
  • Figure 11 shows a close up of part of the example input image of Figure 6 with severe degradation of dynamic range, notably banding in the shoulder and arm.
  • Figure 12 shows a Gaussian pyramid constructed from the lambdai eigenvalue field of Figure 11.
  • Figure 13 shows a related art attempted reconstruction of the dynamic range of the image in Figure 12 having loss of detail in the hair and face regions, plus severe contouring is not eliminated.
  • Figure 14 shows reconstruction of the dynamic range of the image of Figure 12 using examples of the present disclosure.
  • Figure 15 illustrates a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • examples of the present disclosure in other fields such as medical imaging, image analysis and classification, and general signal dynamic range restoration for such signal types as audio, 3D datasets, holographic datasets, and other media and data types constrained by dynamic range, by way of non-limiting examples.
  • examples of the present disclosure may be described in terms of images and video, the restoration of dynamic range for other D-dimensional datasets where D is greater than 1 as in images and video, such as 3D scans, meteorological and seismic data, can employ concepts disclosed herein to improve upon acquisition system limits with respect to dynamic range.
  • Examples of the present disclosure may be configured to restore the dynamic range of images, video and other D-dimensional data sets without close coupling to an upstream encoding, decoding, or other quantization process.
  • Examples of the present disclosure are efficient enough to run on real time for HD scale video on end-user devices such as cellphones and set top boxes, robustly without degrading image detail and high-frequency spatial information such as fine textures, detail, and edge information.
  • Eigensystem analysis is known in the art for such tasks as principal component analysis, fluid dynamics calculations, and material stress calculations. Eigensystem analysis can provide a very robust D-dimensional masking function for a post-processing method to restore dynamic range in images and other D-dimensional data sets.
  • U.S. Patent No. 8,355,534 discloses eigensyetem analysis of spatio-temporal structure tensors of input images, but the eigensyetem analysis is employed primarily to determine the orientation of edge features in a very accurate way so as to avoid the well- known aperture problem in motion estimation.
  • Another use of eigensystem analysis includes the computation of the principal component(s) of a color distribution in an image, such as taught in US Patent Application Publication No. 2014/0098263 (hereinafter "the '263 publication").
  • the '534 patent and the '263 do not benefit from the use of a structure tensor's eigenvalue fields as a smooth masking function for dynamic range enhancement.
  • FIG. 1 is a block diagram of an example computing system 100 that receives and restores the dynamic range of images and video without close coupling to an upstream encoding or decoding process in which examples of the present disclosure may operate.
  • the computing system 100 may be configured to receive encoded image data or video data from one or more data sources 105.
  • the computing system 100 may also include a computing platform 115.
  • the computing platform 115 may comprise a host system 120, which may comprise, for example, a processing device 125, such as one or more central processing units 130a- 130n.
  • the processing device 125 may be coupled to a host memory 135.
  • the host memory 135 may store the encoded image data or video data received from the one or more data sources 105 in an image data buffer 150.
  • the encoded image data or video data may be received by a receiver 160, decoded by a decoder 365, and passed to an image data buffer 150.
  • the receiver 160 may receive the encoded image data or video data either directed from the one or more data sources 105 or over a network 110.
  • one or both of the receiver 360 or the decoder 365 may be external to the processing device 325 or the computing platform 115.
  • one or both of the receiver 160 or the decoder 165 may be integrated with the processing device 125 or the computing platform 115.
  • the processing device 125 may further implement a graphics processing unit 140 (GPU).
  • GPU graphics processing unit 140
  • other co-processor architectures may be utilized besides GPUs, such as, but not limited to, DSPs, FPGAs, or ASICs, or adjunct fixed-function features of the processing device 125 itself.
  • the GPU 140 may be collocated on the same physical chip or logical device as the central processing units 130a- 130n, also known as an "APU", such as found on mobile phones and tablets. Separate GPU and CPU functions may be found on computer server systems where the GPU is a physical expansion card, and personal computer systems and laptops.
  • the GPU 140 may comprise a GPU memory 137.
  • the host memory 135 and GPU memory 137 may also be collocated on the same physical chip(s) or logical device, such as on an APU. It will further be appreciated by those skilled in the art that the decoding processing device 125 may be partially or wholly integrated with the encoding processing device 125 into the computing system 100 of Figure 1 to provide both encoding and decoding functionality.
  • the processing device 125 may be configured to receive encoded image data or video data from the data source 105 through the receiver 160.
  • the processing device 125 may be configured to create the image data buffer 150 based on the received encoded image data or video data to a decoder 165 to decode the image buffer 150.
  • the decoder 165 may be configured to transfer image data or video data stored in the image buffer 150 to the GPU memory 137 as image data or video data stored in the image buffer 155.
  • the processing device 125 may be configured to implement a color channel processor 145 to receive the image data or video data stored in the image buffer 155 and received from the decoder 165, a demultiplexer (not shown) or an unwrapper (not shown). In one example, the processing device 125 may implement the color channel processor 145 as a component of the GPU 140.
  • the color channel processor 145 may be configured to compute a Gaussian pyramid from the image data or video data stored in the image buffer 155.
  • the color channel processor 145 may be configured to create structure tensors for each of the plurality of pixels of image data or video data stored in the image buffer 155.
  • the color channel processor 145 may be configured to compute an eigensystem analysis of structure tensors corresponding each of the plurality of pixels of image data or video data stored in the image buffer 155 to generate eigenvalue fields from the structure tensors.
  • the color channel processor 145 may be configured to compute Gaussian pyramids from each of the eigenvalue fields.
  • the color channel processor 145 may be configured to select one or more of the eigenvalues of each element of the lowest resolution level of computed Gaussian pyramids from each of the eigenvalue fields to assign a linear weight to each pixel of the lowest resolution Gaussian pyramid computed from each of the eigenvalue fields.
  • the color channel processor may be configured to interpolate, using a corresponding linear weight, between (1) each pixel of the lowest resolution pyramid of the Gaussian pyramid computed from the image data or video data stored in the image buffer 155, and (2) a corresponding pixel of the plurality of pixels of input image data or video data stored in the image buffer 155 using a corresponding computed weight to produce a restored dynamic range image or video.
  • the color channel processor 145 may be configured to convey the restored dynamic range image or video into the image buffer 155, suitable for display output on a display 170 or for transmission to one or more downstream devices 175.
  • Figure 2 is a flow diagram illustrating an example of a method 200 for restoration of the dynamic range of images or video without close coupling to an upstream encoding, decoding, or quantization process.
  • the method 200 may be performed by a computer system 100 of Figure 1 and may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device), or a combination thereof.
  • the method 100 may be performed primarily by the color channel processor 145 of the computing system 100 of Figure 1.
  • the receiver 160 may receive encoded image data or video data from the data source 105.
  • the decoder 165 may decode the encoded image data or video data, and may place the decoded image data or video data in an image data buffer 150 of the processing device 125.
  • the processing device 125 may transfer the decoded image data or video data in the image data buffer 150 to GPU image buffer 155 of the GPU memory 137.
  • the color channel processor 145 may receive the decoded image data or video data in the image data buffer 155 from GPU memory 137.
  • the decoded image data or video data in the image data buffer 155 may comprise a plurality of pixels, wherein each pixel comprises one or more color channels.
  • the color channel processor 145 may compute a Gaussian pyramid from the decoded image data or video data in the image data buffer 155.
  • color channel processor 145 may create structure tensors for each of the plurality of pixels in the input image buffer 155.
  • the color channel processor 145 may compute an eigensystem analysis of the structure tensors to generate eigenvalue fields from the structure tensors.
  • the color channel processor 145 may compute Gaussian pyramids from each of the eigenvalue fields.
  • the color channel processor 145 may select one or more of the eigenvalues of each element of the lowest resolution level of computed Gaussian pyramids from each of the eigenvalue fields to assign a linear weight to each pixel of the lowest resolution Gaussian pyramid computed from each of the eigenvalue fields.
  • the color channel processor interpolates, using a corresponding linear weight, between (1) each pixel of the lowest resolution pyramid of the Gaussian pyramid computed from the image data or video data stored in the image buffer 155, and (2) a corresponding pixel of the plurality of pixels of input image data or video data stored in the image buffer 155 using a corresponding computed weight to produce a restored dynamic range image or video.
  • the color channel processor 145 may convey the restored dynamic range image or video into the image buffer 155, suitable, at block 245, for the color channel processor 145 to display as output on a display 170 or to transmit to one or more downstream devices 175.
  • FIG. 3 is a block diagram of another example computing system 300 that receives and restores the dynamic range of D-dimensional vector data set of media content (collectively "D-dimensional media content"), wherein D is greater than or equal to one, without close coupling to an upstream encoding or decoding process in which examples of the present disclosure may operate.
  • D-dimensional media content D-dimensional vector data set of media content
  • the D-dimensional media content may comprise at least one of audio, one or more two-dimensional still images, one or more three-dimensional still images, two- dimensional video, three-dimensional video, magnetic resonance imaging (MRI) data, computed tomography (CT)-scan data, geometric data, seismic data, holographic data, or meteorological data.
  • MRI magnetic resonance imaging
  • CT computed tomography
  • the computing system 300 may be configured to receive encoded D-dimensional media content from one or more media content sources 305.
  • the computing system 300 may also include a computing platform 315.
  • the computing platform 315 may comprise a host system 320.
  • the host system 120 may comprise, for example, a processing device 325, such as the one or more central processing units 330a- 330n.
  • the processing device 325 may be coupled to the host memory 335.
  • the host memory 335 may store the encoded D-dimensional media content received from the one or more media content sources 305 in a media content buffer 350.
  • the encoded D-dimensional media content may be received by a receiver 360, decoded by a decoder 365, and passed to the media content buffer 350.
  • the receiver 360 may receive the encoded D-dimensional media content either directly from the one or more media content sources 305 or over a network 310.
  • one or both of the receiver 360 or the decoder 365 may be external to the processing device 325 or the computing platform 315.
  • one or both of the receiver 360 or the decoder 365 may be integrated with the processing device 325 or the computing platform 315.
  • the processing device 325 may further implement a graphics processing unit 340 (GPU).
  • GPU graphics processing unit
  • other co-processor architectures may be utilized besides GPUs, such as, but not limited to, DSPs, FPGAs, or ASICs, or adjunct fixed-function features of the processing device 325 itself.
  • the GPU 340 may be collocated on the same physical chip or logical device as the central processing units 330a-330n, also known as an "APU", such as found on mobile phones and tablets. Separate GPU and CPU functions may be found on computer server systems where the GPU 340 may be a physical expansion card, and personal computer systems and laptops.
  • the GPU 340 may comprise a GPU memory 337.
  • the host memory 335 and GPU memory 337 may also be collocated on the same physical chip(s) or logical device, such as on an APU. It will further be appreciated by those skilled in the art that the decoding processing device 325 may be partially or wholly integrated with the encoding processing device 325 into the computing system 300 of Figure 3 to provide both encoding and decoding functionality.
  • the processing device 325 may be configured to receive the encoded D-dimensional media content from the media content source 305 through the receiver 360.
  • the processing device 325 may be configured to create the media content buffer 350 from the encoded D- dimensional media content and transmit the encoded D-dimensional media content to a decoder 365 to decode the D-dimensional media content stored in the media content buffer 350.
  • the decoder 365 may be configured to transfer the D-dimensional media content stored in the media content buffer 350 to the GPU memory 337 as input D-dimensional media content stored in the media content buffer 355.
  • the processing device 325 may be configured to implement a media content processor 345 to receive the input D-dimensional media content stored in the media content buffer 355 from the decoder 365, demultiplexer (not shown) or unwrapper (not shown). In one example, the processing device 325 may implement the media content processor 345 as a component of the GPU 340.
  • the media content processor 345 may be configured to apply a low- frequency-pass method to the D-dimensional media content stored in the media content buffer 155 to generate a low-frequency D-dimensional vector data set of media content (hereinafter "low frequency D-dimensional media content").
  • the low- frequency-pass method may comprise the media content processor 345 generating a D- dimensional Gaussian pyramid from the received D-dimensional media content stored in the media content buffer 355 and selecting a D-dimensional vector data from the lowest dimension pyramid level of the D-dimensional Gaussian pyramid to generate the low- frequency D-dimensional media content.
  • the media content processor 345 may be configured to obtain a structure tensor field comprising a set of D-dimensional structure tensors corresponding to each vector of the input D-dimensional media content stored in the media content buffer 355.
  • the media content processor 345 obtaining a set of D-dimensional structure tensors may comprise the media content processor 345 computing a set of D- dimensional directional gradients for each vector of the received D-dimensional media content.
  • the media content processor 345 obtaining a set of D-dimensional structure tensors may comprise the media content processor 345 performing a structure tensor analysis with one element of each vector of the received D-dimensional media content.
  • the media content processor 345 may be configured to perform an eigensystem analysis for each structure tensor in the field of structure tensors to generate a plurality of D eigenvalue fields comprising D eigenvalues for each vector of the received D-dimensional media content.
  • the media content processor 345 may be configured to interpolate between each vector of the received D-dimensional media content stored in the media content buffer 355 and a corresponding vector from the low-frequency D-dimensional media content in view of one or more of the D eigenvalues for each vector of the plurality of D eigenvalue fields to produce a restored-dynamic range D-dimensional vector data set of media content (hereinafter "restored-dynamic range D-dimensional media content).
  • interpolating may comprise the media content processor 345 linearly interpolating between each vector of the received D-dimensional media content and the corresponding vector from the low-frequency D-dimensional media content.
  • interpolating may comprise the media content processor 345 generating a Gaussian pyramid from each of the plurality of D eigenvalue fields.
  • the media content processor 345 may further employ one or more of the D eigenvalues to assign linear weights to each vector of the lowest resolution plurality of D eigenvalue fields.
  • the media content processor 345 may further employ the linear weights to interpolate between each vector of the received D- dimensional media content and the corresponding vector from the low-frequency D- dimensional media content to produce the restored-dynamic range D-dimensional media content.
  • the linear weights may be assigned in view of a region-of-support used for obtaining the structure tensor field.
  • the media content processor 345 employing one or more of the D eigenvalues to assign linear weights to each vector of the lowest resolution plurality of D eigenvalue fields may comprise the media content processor selecting an eigenvalue of the D eigenvalues indicative of a homogenous region of support about a vector of the received D-dimensional media content as the lower bound for assigning the linear weights.
  • the media content processor may further select one of the remaining eigenvalues of the D eigenvalues indicative of an edge or highly textured region of support about a vector of the received D- dimensional media content as the upper bound for assigning the linear weights.
  • interpolating may comprise the media content processor 345 performing an interpolation jointly between all element of each vector of the received D- dimensional media content and the corresponding elements of each vector from the low- frequency D-dimensional media content.
  • the media content processor 345 may be configured to convey the restored-dynamic range D-dimensional media content into the media content buffer 355, suitable for display output on a display 370 or for transmission to one or more downstream devices 375.
  • each vector of the received D-dimensional vector data set of media content, the low-frequency D-dimensional vector data set of media content, and the restored- dynamic range D-dimensional vector data set of media content may each comprise N- dimensional scalar values where N is equal to or greater than 1.
  • the received D-dimensional vector data set of media content, the low-frequency D-dimensional vector data set of media content, and the restored-dynamic range D-dimensional vector data of media content may each comprise D-dimensional pixel values.
  • the pixel values may comprise color values or luminance values or both color values and luminance values.
  • Figure 4 is a flow diagram illustrating one example of a method 400 for restoration of the dynamic range of D-dimensional media content without close coupling to an upstream encoding, decoding, or quantization process.
  • the method 400 may be performed by a computer system 300 of Figure 3 and may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device), or a combination thereof.
  • the method 400 may be performed primarily by the media content processor 345 of the computing system 300 of Figure 3.
  • the receiver 360 may receive encoded D-dimensional media content from the data source 305.
  • the D-dimensional media content may comprise at least one of audio, one or more two-dimensional still images, one or more three- dimensional still images, two-dimensional video, three-dimensional video, magnetic resonance imaging (MRI) data, computed tomography (CT)-scan data, geometric data, seismic data, holographic data, or meteorological data.
  • MRI magnetic resonance imaging
  • CT computed tomography
  • the decoder 365 may decode the encoded D-dimensional media content, and may place the decoded D-dimensional media content in the media content buffer 350 of the processing device 325.
  • the processing device 325 may transfer the decoded D-dimensional media content in the media content buffer 350 to GPU media content buffer 355 of the GPU memory 337.
  • the media content processor 345 may receive the decoded D-dimensional media content in the media content buffer 355 from GPU memory 337.
  • the media content processor 345 may apply a low-frequency-pass method to the D-dimensional media content stored in the media content buffer 355 to generate low frequency D-dimensional media content.
  • the low-frequency- pass method may comprise the media content processor 345 generating a D-dimensional Gaussian pyramid from the received D-dimensional media content stored in the media content buffer 355 and selecting a D-dimensional vector data from the lowest dimension pyramid level of the D-dimensional Gaussian pyramid to generate the low-frequency D- dimensional media content.
  • the media content processor 345 may obtain a structure tensor field comprising a set of D-dimensional structure tensors corresponding to each vector of the decoded D-dimensional media content stored in the media content buffer 355.
  • the media content processor 345 obtaining a set of D-dimensional structure tensors may comprise the media content processor 345 computing a set of D-dimensional directional gradients for each vector of the received D-dimensional media content.
  • the media content processor 345 obtaining a set of D-dimensional structure tensors may comprise the media content processor 345 performing a structure tensor analysis with one element of each vector of the received D-dimensional media content.
  • the media content processor 345 may perform an eigensystem analysis for each structure tensor in the field of structure tensors to generate a plurality of D eigenvalue fields comprising D eigenvalues for each vector of the decoded D-dimensional media content.
  • the media content processor 345 may interpolate between each vector of the received D-dimensional media content stored in the media content buffer 355 and a corresponding vector from the low-frequency D-dimensional media content in view of one or more of the D eigenvalues for each vector of the plurality of D eigenvalue fields to produce restored-dynamic range D-dimensional media content.
  • interpolating may comprise the media content processor 345 linearly interpolating between each vector of the received D-dimensional media content and the corresponding vector from the low-frequency D-dimensional media content.
  • interpolating may comprise the media content processor 345 generating a Gaussian pyramid from each of the plurality of D eigenvalue fields.
  • the media content processor 345 may further employ one or more of the D eigenvalues to assign linear weights to each vector of the lowest resolution plurality of D eigenvalue fields.
  • the media content processor 345 may further employ the linear weights to interpolate between each vector of the received D- dimensional media content and the corresponding vector from the low-frequency D- dimensional media content to produce the restored-dynamic range D-dimensional media content.
  • the linear weights may be assigned in view of a region-of-support used for obtaining the structure tensor field.
  • the media content processor 345 employing one or more of the D eigenvalues to assign linear weights to each vector of the lowest resolution plurality of D eigenvalue fields may comprise the media content processor selecting an eigenvalue of the D eigenvalues indicative of a homogenous region of support about a vector of the received D-dimensional media content as the lower bound for assigning the linear weights.
  • the media content processor may further select one of the remaining eigenvalues of the D eigenvalues indicative of an edge or highly textured region of support about a vector of the received D- dimensional media content as the upper bound for assigning the linear weights.
  • interpolating may comprise the media content processor 345 performing an interpolation jointly between all element of each vector of the received D- dimensional media content and the corresponding elements of each vector from the low- frequency D-dimensional media content.
  • the media content processor 345 may convey the restored-dynamic range D- dimensional media content into the media content buffer 355, suitable, at block 435, for the media content processor 345 to display output on a display 370 or to transmit to one or more downstream devices 375.
  • the interpolating and applying a low-frequency-pass method may be performed in parallel.
  • each vector of the received D-dimensional vector data set of media content, the low-frequency D-dimensional vector data set of media content, and the restored- dynamic range D-dimensional vector data set of media content may each comprise N- dimensional scalar values where N is equal to or greater than 1.
  • the received D-dimensional vector data set of media content, the low-frequency D-dimensional vector data set of media content, and the restored-dynamic range D-dimensional vector data of media content may each comprise D-dimensional pixel values.
  • the pixel values may comprise color values or luminance values or both color values and luminance values.
  • Figure 5 shows an example of input D-dimensional media content (e.g., an image).
  • FIG. 6 shows a schematic of a constructed Gaussian Pyramid.
  • an input image may be subjected to Gaussian filtering, and subsequent octave-by- octave decimation.
  • Each successive stage of the Gaussian pyramid is a low pass, resolution- decimated version of the original image with successively smaller image sizes.
  • the advantage of this approach is that it generates an approximation of an expensive, low-pass filtered image without using a single pass of a Gaussian filter with an extraordinarily large region of support size.
  • Gaussian Pyramids operate in the pixel domain, which permits very low frequency low-pass filtration operations without resorting to convolution in the frequency domain - an expensive process.
  • Figure 7 shows an example Gaussian Pyramid computed from the example input image of Figure 5, according to examples of the present disclosure.
  • Figure 8 shows a computed gradient field from the example input image of Figure 5.
  • Examples of the present disclosure are intended to increase the dynamic range of images or data sets. A naive way to accomplish this is by spatial interpolation, such as by blurring the images. This increases the sampling dynamic range, but at the cost of high-frequency detail such as texture and edges. In many cases, the reduced dynamic range is not visible or mathematically important around high frequencies; however, it is most noticeable in smooth gradients over large, nearly homogenous regions. Distinguishing what type of region exists within a given part of an image is a difficult problem.
  • Certain related art approaches may employ edge-directed smoothing - which may not be robust in regions of fine texture without strong edges.
  • Other approaches may employ bilateral filtering or sigma filtering, which is an improvement, but suffer from needing an arbitrary threshold - setting the threshold too low results in over-smoothing in textured regions, and setting the threshold too high results in no increase of dynamic range in the areas of an image that need it.
  • Examples of the present disclosure employ the properties of the structure tensor in order to distinguish textured, edge, detail, and homogenous regions, with the intent that only homogenous regions will be treated, while other regions are left in their original state.
  • the first step is to compute gradients along each dimension of the data set. In the case of images, the gradients dX and dY are computed with a discrete approximation of the gradient function.
  • Eq. 1 is one of many appropriate discrete approximations of the gradient of each pixel.
  • the 2- dimensional structure tensor for each pixel created using the gradients is shown in Eq. 2:
  • wlk'iv ⁇ , ⁇ ⁇ , and ⁇ for a 3x3 region-of- support are defined by Eq. 3: dl (n, m) dl (n, m)
  • weight(n,m) (0.125, 0.75, 0.125 ⁇ x ⁇ 0.125, 0.75, 0.125 f
  • the eigenvalue fields (two, in the case of a 2D data set such as an image) may be calculated for each tensor of each gradient of each pixel as in Eq. 5:
  • the two eigenvalues give structural information about the region- of-support surrounding the associated pixel.
  • One high and one low eigenvalue indicate tendency to a strong edge.
  • Two high or medium eigenvalues imply highly or moderately textured regions, respectively.
  • Figure 10 shows the two eigenvalue fields (lambdai and lambda 2 ) computed from the structure tensors of the example input image in Figure 5, normalized for visualization in the figure.
  • the limited region of support means that any dynamic range expansion utilizing this analysis will be limited to loss of dynamic range covering a very small area.
  • Figure 11 shows a close up of part of the original image of Figure 5 with severe degradation of dynamic range. Note banding in the shoulder and arm. The banding in this case involves a random distribution of dots of two different quantized values. The region of transition spans much more than a 3x3 region. To permits the analysis to be sensitive to texture and edge features further away than a 3x3 region, a Gaussian pyramid of the eigenvalue fields is constructed.
  • Figure 12 shows the Gaussian pyramid constructed from the lambdai eigenvalue field of Figure 11.
  • the Gaussian pyramids of all D eigenvalue fields and given floating point precision, one has a coarse but otherwise accurate approximation of the overall structure of the region of the image or dataset surrounding a given corresponding pixel, or data element.
  • the 4 th pyramid level of both eigenvalue fields gives an indication of the overall structure of not just the 3x3 surrounding a given pixel of the input image, but the surrounding 16x16 region.
  • the overall strategy behind examples of the disclosure is to select, for each of the plurality of pixels of the input image or data set of media content, between two possible pixel or data set values.
  • the first choice is a very accurately blurred, or multi-octave low- pass image, which effectively has increased the dynamic range of the quantized image - provided by block 215, by way of the image Gaussian pyramid.
  • the 4 th pyramid level suffices generated by block 215 for a sufficient approximation of a very low pass filter. While this low-pass image has increased dynamic range, it is provided at the expense of other high frequency detail such as edges and textures.
  • the other choice is to select from the original image or dataset, which has other high frequency information, but at a lower dynamic range, including low-energy edges due to quantization.
  • the choice for each of the plurality of pixels between the input image from block 210 and the low-pass pyramid image from block 215 should not be a binary one. To ensure smooth transitions between these regions, a linear weighting factor is used.
  • the linear weighting factor is used to smoothly interpolate between the input and low-pass choices in regions of the image or data set where the structure is transitioning from homogenous to structured.
  • Each output pixel value has a corresponding weight that is used to linearly interpolate between the input image and the low-pass image.
  • Figure 13 shows an attempted reconstruction in the related art of the dynamic range of the image in Figure 12 - note the loss of detail in the hair and face regions, plus the severe contouring is not eliminated.
  • Edge-directed, bilateral, and frequency-domain dynamic range reconstruction techniques are prone to false positives such as smoothing areas that should not be smoothed, and false negatives such as missing banding and dynamic range quantization artifacts altogether.
  • Figure 14 shows reconstruction of the dynamic range of the image of Figure
  • the dynamic range restoration process of examples of the present disclosure is efficient enough to perform in greater-than-real-time for 4K video resolution video at 30 fps on contemporaneous, commercial, mass-marketed computer hardware, and multiple images and video at a variety of resolutions in real-time and near-real-time. Additionally, the dynamic range restoration process of embodiments of the present disclosure is efficient enough to perform on any end-user device with a GPU, CPU, or APU at full HD resolutions for single instances of video and images, such as feature phones, smart phones, tablets, laptops, PCs, set-top boxes, and televisions.
  • This combination of efficiencies at the decoder proximity employing examples of the present disclosure opens up new applications.
  • These applications include, but are not limited to, real-time improved video coder efficiency for over-the-top video delivery, cost-effective real-time reduction of public radio-access-network congestion when both uploading and downloading video and image data from mobile devices, increased realtime pass-band television delivery capacity, increase of satellite transponder capacity, reduction of storage costs for content management systems and network DVR architectures, and high-throughput treatment of images and video at the distribution network core, all by means of permitting existing encoding processes to transmit and store images, video, and other datasets with less dynamic range than has been previously possible.
  • Figure 15 illustrates a diagrammatic representation of a machine in the example form of a computer system 1500 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • the machine may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, or the Internet.
  • the machine may operate in the capacity of a server machine in client-server network environment.
  • the machine may be a personal computer (PC), a set-top box (STB), a server, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • PC personal computer
  • STB set-top box
  • server a server
  • network router switch or bridge
  • the example computer system 1500 includes a processing device (processor)
  • main memory 1504 e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)
  • DRAM dynamic random access memory
  • static memory 1506 e.g., flash memory, static random access memory (SRAM)
  • data storage device 1516 which communicate with each other via a bus 1508.
  • Processor 1502 represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like. More particularly, the processor 1502 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLrW) microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets.
  • the processor 1502 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like.
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • DSP digital signal processor
  • the color channel processor 145 and the media content processor 345 shown in Figures 1 and 3, respectively, may be executed by processor 1502 configured to perform the operations and steps discussed herein.
  • the computer system 1500 may further include a network interface device
  • the computer system 1500 also may include a video display unit 1510 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 1512 (e.g., a keyboard), a cursor control device 1514 (e.g., a mouse), and a signal generation device 1520 (e.g., a speaker).
  • a video display unit 1510 e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)
  • an alphanumeric input device 1512 e.g., a keyboard
  • a cursor control device 1514 e.g., a mouse
  • signal generation device 1520 e.g., a speaker
  • a drive unit 1516 may include a computer-readable medium 1524 on which is stored one or more sets of instructions (e.g., instructions of the color channel processor 145 and the media content processor 345) embodying any one or more of the methodologies or functions described herein.
  • the instructions of the media content processor 145 may also reside, completely or at least partially, within the main memory 1504 and/or within the processor 1502 during execution thereof by the computer system 1500, the main memory 1504 and the processor 1502 also constituting computer-readable media.
  • the instructions of the color channel processor 145 and the media content processor 345 may further be transmitted or received over a network via the network interface device 1522.
  • computer-readable storage medium 1524 is shown in an example to be a single medium, the term “computer-readable storage medium” should be taken to include a single non-transitory medium or multiple non-transitory media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
  • the term “computer-readable storage medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure.
  • the term “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.
  • Examples of the disclosure also relate to an apparatus for performing the operations herein.
  • This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer.
  • the system and method as disclosed herein restoration of the dynamic range of images, video and other D-dimensional data sets without close coupling to an upstream encoding, decoding, or other quantization process improves the perceptual quality and/or the transmission or storage efficiency of existing image and video compression or transmission systems and methods solves problems in many fields, such as real-time efficiency for over-the-top video delivery, cost-effective real-time reduction of public radio-access-network congestion when both uploading and downloading video and image data from mobile devices, increased real-time pass-band television delivery capacity, increase of satellite transponder capacity, reduction of storage costs for content management systems and network DVR architectures, and high-throughput treatment of images and video at the distribution network core as but a few examples.
  • Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions.
  • a computer readable storage medium such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un dispositif de traitement qui reçoit, d'un dispositif en amont, un ensemble de données de vecteurs de dimension D d'un contenu multimédia, D étant supérieur ou égal à un. Le dispositif de traitement applique un procédé de passage basse fréquence au contenu multimédia de dimension D reçu afin de générer un ensemble de données de vecteurs de dimension D basse fréquence d'un contenu multimédia. Le dispositif de traitement obtient un champ de tenseur de structure comprenant un ensemble de tenseurs de structure en dimension D correspondant à chaque vecteur du contenu multimédia en dimension D reçu. Le dispositif de traitement effectue une analyse de système de valeurs propres pour chaque tenseur de structure dans le champ de tenseurs de structure afin de générer une pluralité de champs de valeurs propres D comprenant des valeurs propres D pour chaque vecteur du contenu multimédia en dimension D reçu.
EP14901826.9A 2014-09-19 2014-09-19 Système et procédé de restauration de plage dynamique d'images et de vidéos Withdrawn EP3195202A1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2014/056559 WO2016043774A1 (fr) 2014-09-19 2014-09-19 Système et procédé de restauration de plage dynamique d'images et de vidéos

Publications (1)

Publication Number Publication Date
EP3195202A1 true EP3195202A1 (fr) 2017-07-26

Family

ID=55533651

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14901826.9A Withdrawn EP3195202A1 (fr) 2014-09-19 2014-09-19 Système et procédé de restauration de plage dynamique d'images et de vidéos

Country Status (3)

Country Link
EP (1) EP3195202A1 (fr)
CA (1) CA2961719A1 (fr)
WO (1) WO2016043774A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241949B (zh) * 2020-01-03 2021-12-14 中科智云科技有限公司 图像识别方法、装置、电子设备及可读存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10506735A (ja) * 1995-03-24 1998-06-30 ユナイテッド パーセル サービス オブ アメリカ，インコーポレイテッド 循環パターンを含む画像において重なった線を除去しピクセル値を復元するための方法及び装置
US7039252B2 (en) * 1999-02-25 2006-05-02 Ludwig Lester F Iterative approximation environments for modeling the evolution of an image propagating through a physical medium in restoration and other applications
US6650476B1 (en) * 1999-02-25 2003-11-18 Lester Frank Ludwig Image processing utilizing non-positive-definite transfer functions via fractional fourier transform
US7460130B2 (en) * 2000-09-26 2008-12-02 Advantage 3D Llc Method and system for generation, storage and distribution of omni-directional object views
KR100682889B1 (ko) * 2003-08-29 2007-02-15 삼성전자주식회사 영상에 기반한 사실감 있는 3차원 얼굴 모델링 방법 및 장치
US8243805B2 (en) * 2006-11-14 2012-08-14 Microsoft Corporation Video completion by motion field transfer
US8374446B2 (en) * 2007-09-28 2013-02-12 Vsevolod Yurievich Mokrushin Encoding and decoding of digital signals based on compression of hierarchical pyramid
JP5506273B2 (ja) * 2009-07-31 2014-05-28 富士フイルム株式会社 画像処理装置及び方法、データ処理装置及び方法、並びにプログラム

Also Published As

Publication number Publication date
WO2016043774A1 (fr) 2016-03-24
CA2961719A1 (fr) 2016-03-24

Similar Documents

Publication Publication Date Title
US10750179B2 (en) Decomposition of residual data during signal encoding, decoding and reconstruction in a tiered hierarchy
CN105374005B (zh) 数据处理系统及其操作方法、计算机可读存储介质
KR102165147B1 (ko) 계층형 신호 디코딩 및 신호 복원
US8639056B2 (en) Contrast enhancement
EP2230640B1 (fr) Verfahren zum Filtern von Tiefenbildern
US20120269457A1 (en) Method for Synthesizing a Virtual Image from a Reduced Resolution Depth Image
JP6137699B2 (ja) 空間的にスケーリング可能なビデオコーディングのための適応型アップサンプリングの方法、プログラム及びコンピュータシステム
CN114868401A (zh) 使用下采样/上采样和深度图非线性滤波对视频数据的编码方案
CN113313774A (zh) 图像处理方法、装置、电子设备及存储介质
US8982950B1 (en) System and method for restoration of dynamic range of images and video
Ma et al. Learning-based image restoration for compressed images
CN111083478A (zh) 一种视频帧重构方法、装置及终端设备
US11792453B2 (en) Coding scheme for immersive video with asymmetric down-sampling and machine learning
Kim et al. Example-based learning for single-image super-resolution and JPEG artifact removal
WO2016043774A1 (fr) Système et procédé de restauration de plage dynamique d'images et de vidéos
WO2022237899A1 (fr) Procédé, dispositif et support pour traitement vidéo
CN115375539A (zh) 图像分辨率增强、多帧图像超分辨率系统和方法
Liu et al. Arbitrary-scale super-resolution via deep learning: A comprehensive survey
CN110830806A (zh) 一种视频帧预测方法、装置及终端设备
Ogawa et al. Adaptive subspace-based inverse projections via division into multiple sub-problems for missing image data restoration
Mosleh et al. Image and video spatial super-resolution via bandlet-based sparsity regularization and structure tensor
Tanaka et al. Directional image decomposition using retargeting pyramid
Ye et al. Learning multi-granularity semantic interactive representation for joint low-light image enhancement and super-resolution
JP2017520871A (ja) 画像データ内の間引きされた色チャネルの送信、受信、及び復元を行うシステム及び方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20170418

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20170727