WO2022183358A1 - 视频图像去交错方法和视频图像去交错装置 - Google Patents

视频图像去交错方法和视频图像去交错装置 Download PDF

Info

Publication number
WO2022183358A1
WO2022183358A1 PCT/CN2021/078598 CN2021078598W WO2022183358A1 WO 2022183358 A1 WO2022183358 A1 WO 2022183358A1 CN 2021078598 W CN2021078598 W CN 2021078598W WO 2022183358 A1 WO2022183358 A1 WO 2022183358A1
Authority
WO
WIPO (PCT)
Prior art keywords
resolution
field data
image
resolutions
numbered field
Prior art date
Application number
PCT/CN2021/078598
Other languages
English (en)
French (fr)
Inventor
朱丹
段然
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US17/625,714 priority Critical patent/US11711491B2/en
Priority to CN202180000385.XA priority patent/CN115868157A/zh
Priority to PCT/CN2021/078598 priority patent/WO2022183358A1/zh
Publication of WO2022183358A1 publication Critical patent/WO2022183358A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
    • H04N7/012—Conversion between an interlaced and a progressive signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0102—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving the resampling of the incoming video signal

Definitions

  • the embodiments of the present disclosure relate to the technical field of image processing, and in particular, to a video image deinterleaving method and a video image deinterleaving device.
  • the interlaced scanning video image needs to be deinterlaced to obtain a progressive scanning video image.
  • de-interlacing an interlaced video image how to improve the de-interlacing effect is an urgent problem to be solved.
  • an embodiment of the present disclosure provides a video image deinterlacing method, including:
  • N is a positive integer greater than or equal to 2.
  • inputting images of N resolutions including the original video image and the down-sampled images of the N-1 resolutions into a deinterlacing network for deinterlacing processing to obtain a deinterlaced image includes:
  • the up-sampled image of the i-th resolution and the down-sampled image of the i-th resolution are spliced to obtain a mosaic image of the i-th resolution; the mosaic image of the i-th resolution is input to the i-th resolution.
  • the de-interlaced image of the i-th resolution is obtained in the interleaving sub-network; the de-interlaced image of the i-th resolution is up-sampled to obtain an up-sampled image of the i-1-th resolution; wherein, i is greater than or an integer equal to 2 and less than N;
  • merging the odd-numbered field data and the even-numbered field data with the same resolution in the N-1 odd-numbered field data with different resolutions and the even-numbered field data with the same resolution in the N-1 even-numbered field data with different resolutions includes:
  • the odd-numbered field data and the even-numbered field data having the same resolution are arranged at line intervals.
  • the N is 3 or 4.
  • the resolutions of images of adjacent resolutions among the N resolution images are in a 2-fold relationship.
  • a bicubic interpolation method is used to perform up-sampling and/or down-sampling.
  • the N series-connected deinterleaving sub-networks have the same structure and different parameters.
  • each of the deinterleaving sub-networks includes a plurality of filters in series, each filter includes a plurality of convolution kernels in series, and in the plurality of filters in series, the resolutions of every two filters are the same. , the output of each filter except the last serves as the input to the next filter and the filter with the same resolution.
  • the method further includes:
  • the training of the deinterlaced network to be trained includes:
  • the training images of N resolutions including the video images for training and the downsampling images for training of the N-1 resolutions into the deinterlacing network to be trained for deinterlacing processing to obtain outputs of N resolutions image, among the images of the N resolutions, the resolutions from the image of the Nth resolution to the image of the first resolution are sequentially increased, and the deinterlacing network to be trained includes N deinterlacing sub-networks connected in series , the images processed by the N concatenated deinterlacing sub-networks are generated based on the images of the N resolutions respectively;
  • the loss is L2 loss.
  • the total loss is equal to the sum of the losses of the output images of the N resolutions, or equal to the weighted sum of the losses of the output images of the N resolutions.
  • an embodiment of the present disclosure provides a video image deinterlacing apparatus, including:
  • the first acquisition module is used to acquire a single-frame original video image containing odd-even field information
  • the first extraction module is used for extracting odd-numbered field data and even-numbered field data in the original video image
  • the first down-sampling module is used to downsample the odd-numbered field data N-1 times to obtain N-1 odd-numbered field data of different resolutions, and perform downsampling N-1 times on the even-numbered field data to obtain N- 1 even-numbered field data with different resolutions; the odd-numbered field data and even-numbered field data with the same resolution in N-1 odd-numbered field data with different resolutions and N-1 even-numbered field data with different resolutions are processed. Merge to get N-1 downsampled images of different resolutions;
  • a de-interlacing module configured to input images of N resolutions including the original video image and the down-sampled images of the N-1 resolutions into a de-interlacing network to perform de-interlacing processing to obtain a de-interlaced image, the In the images of N resolutions, the resolutions of the images of the Nth resolution increase sequentially from the image of the Nth resolution to the image of the first resolution, and the deinterlacing network includes N series deinterlacing sub-networks, and the N series deinterlacing sub-networks are The images processed by the deinterlacing sub-network are generated based on the N resolution images respectively;
  • N is a positive integer greater than or equal to 2.
  • an embodiment of the present disclosure provides an electronic device, including a processor, a memory, and a program or instruction stored on the memory and executable on the processor, where the program or instruction is processed by the processor.
  • an embodiment of the present disclosure provides a non-volatile computer-readable storage medium, where a program or an instruction is stored on the non-volatile computer-readable storage medium, and the program or instruction is implemented when executed by a processor The steps of the video image deinterlacing method described in the first aspect above.
  • FIG. 1 is a schematic flowchart of a video image deinterlacing method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of extracting odd-numbered field data and even-numbered field data from an original video image according to an embodiment of the present disclosure
  • FIG. 3 and 4 are schematic diagrams of an image downsampling method according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic diagram of a deinterleaving network according to an embodiment of the disclosure.
  • FIG. 6 is a schematic structural diagram of a deinterleaving sub-network according to an embodiment of the present disclosure
  • FIG. 7 is a schematic flowchart of a method for training a deinterlacing network according to an embodiment of the present disclosure
  • FIG. 8 is a schematic diagram of a method for calculating the total loss of a deinterleaving network according to an embodiment of the present disclosure
  • FIG. 9 is a schematic structural diagram of a video image deinterlacing apparatus according to an embodiment of the disclosure.
  • FIG. 10 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
  • an embodiment of the present disclosure provides a video image deinterlacing method, including:
  • Step 11 Obtain a single-frame original video image containing parity field information
  • the original video image is a video image obtained by interlaced scanning.
  • the electron beam first scans all odd-numbered lines from left to right and from top to bottom to form a field of image data, and then the electron beam returns to the top again. Scan all even lines from left to right and top to bottom to form another field of image data.
  • the scanning fields displayed in these two vertical directions alternately constitute the complete video image of each frame.
  • Step 12 Extract odd field data and even field data in the original video image
  • FIG. 2 is a schematic diagram of extracting odd field data and even field data from an original video image according to an embodiment of the present disclosure.
  • Step 13 Perform N-1 downsampling on the odd-numbered field data to obtain N-1 odd-numbered field data with different resolutions, and perform N-1 downsampling on the even-numbered field data to obtain N-1 different resolutions
  • the odd-numbered field data and the even-numbered field data with the same resolution in the N-1 odd-numbered field data with different resolutions and the odd-numbered field data and the even-numbered field data with the same resolution in the N-1 even-numbered field data with different resolutions are combined to obtain N- 1 downsampled image of different resolutions;
  • the odd-numbered field data and the even-numbered field data of the original video image are down-sampled twice, please refer to Figure 3 and Figure 4, which are down-sampling two times and four times respectively, assuming the resolution of odd-numbered field data and even-numbered field data.
  • the rate is 256 ⁇ 256, down-sampling 2 times to obtain odd-numbered field data and even-numbered field data of 128 ⁇ 128 resolution, and down-sampling 4 times to obtain odd-numbered field data and even-numbered field data of 64 ⁇ 64 resolution.
  • the odd-numbered field data and the even-numbered field data after downsampling may be combined according to the splicing manner of the odd-numbered field data and the even-numbered field data in the original video image.
  • the odd-numbered field data and the even-numbered field data with the same resolution are arranged according to the line interval, that is, interleaved and combined.
  • Step 14 Input images of N resolutions including the original video image and the down-sampled images of the N-1 resolutions into a deinterlacing network for deinterlacing to obtain a deinterlaced image, the N resolution images.
  • the resolutions of the images of the Nth resolution increase sequentially from the image of the Nth resolution to the image of the first resolution
  • the deinterlacing network includes N series deinterlacing sub-networks, and the N series The images processed by the sub-network are respectively generated based on the images of the N resolutions;
  • N is a positive integer greater than or equal to 2.
  • the N deinterlacing networks are respectively used to process images of one of the N resolutions.
  • the deinterlaced image finally output by the deinterlacing network is a progressive scan video image.
  • the odd-numbered field data and the even-numbered field data of the extracted original video image are downsampled to multiple resolutions, and deinterlaced at multiple resolutions, which can ensure that the original video image will not be damaged during downsampling.
  • the information of the video image and the progressive deinterlacing can achieve a better deinterlacing effect.
  • images of N resolutions including the original video image and the downsampled images of N-1 resolutions are input into a deinterlacing network for deinterlacing processing to obtain deinterlacing Images include:
  • For the image with the smallest resolution perform the following operations: stitch the two down-sampled images of the Nth resolution to obtain the stitched image of the Nth resolution; input the stitched image of the Nth resolution to the In the Nth concatenated deinterlacing sub-network, a deinterlaced image of the Nth resolution is obtained; the deinterlaced image of the Nth resolution is up-sampled to obtain an upsampling image of the N-1th resolution ;
  • For the image with the largest resolution perform the following operations: stitch the upsampled image of the first resolution and the original video image of the first resolution to obtain a stitched image of the first resolution;
  • the stitched image at the rate is input to the first deinterlacing network to obtain a deinterlacing image of the first resolution, which is used as the output image of the deinterleaving network.
  • the odd-numbered field data and the even-numbered field data of the output original video image are down-sampled, which can ensure that the information of the original video image will not be destroyed during down-sampling.
  • the N is 3 or 4, so as to achieve a better de-interleaving effect, and can effectively reduce the implementation cost.
  • N is other values greater than or equal to 2.
  • the resolutions of images of adjacent resolutions among the N resolution images are in a 2-fold relationship.
  • the resolutions of the N resolution images are respectively 256 ⁇ 256, 128 ⁇ 128, 64 ⁇ 64.
  • a bicubic interpolation method is used to perform up-sampling and/or down-sampling, so as to preserve better image details.
  • other interpolation methods such as bilinear interpolation, can also be used for up-sampling and/or down-sampling.
  • FIG. 5 is a schematic diagram of a deinterleaving network according to an embodiment of the present disclosure.
  • the deinterleaving network includes three deinterleaving sub-networks: deinterleaving sub-network 1, de-interleaving sub-network 2 and de-interleaving sub-network Subnet 3.
  • the deinterleaving network works as follows:
  • the downsampling is 2 times and 4 times, respectively, to obtain the down-sampling images Down_x2 odd , Down_x2 even , Down_x4 odd of the odd field data and the even field data And Down_x4 even , combine Down_x2 odd and Down_x2 even to get Down_x2, combine Down_x4 odd and Down_x4 even to get Down_x4;
  • the N series-connected deinterleaving sub-networks have the same structure and different parameters.
  • FIG. 6 is a schematic structural diagram of a deinterleaving sub-network according to an embodiment of the present disclosure.
  • the de-interleaving sub-network includes: a plurality of filters connected in series, and each filter includes a plurality of convolutional convolutions connected in series Nuclei (vertical bars in Figure 6).
  • each filter includes four convolution kernels connected in series.
  • the number of convolution kernels in the filter is not limited to four.
  • the vertical bars filled with diagonal stripes represent downsampling
  • the vertical bars filled with dots represent upsampling.
  • every two filters have the same resolution, and except for the last filter, the output of each of the other filters is used as the next filter and has the same resolution. input of the filter.
  • the deinterleaving sub-network includes 6 filters in series, wherein the first filter has the same resolution as the sixth filter, and the second filter has the same resolution as the fifth filter.
  • the resolution of the third filter is the same as that of the fourth filter, and the output of the first filter is used as the second filter and the sixth filter (same resolution as the first filter the same) input, the output of the second filter as the input of the third and fifth filter (same resolution as the second filter), the output of the third filter as the fourth filter Input to the filter (same resolution as the third filter).
  • the video image deinterlacing method further includes: training the deinterlacing network to be trained to obtain the deinterlacing network; please refer to FIG. 7 , the training of the deinterlacing network to be trained includes:
  • Step 71 Obtain a single-frame training video image containing parity field information
  • Step 72 Extract the odd-numbered field data and the even-numbered field data for training in the training video image
  • Step 73 Perform N-1 downsampling on the training odd-numbered field data to obtain N-1 odd-numbered field data with different resolutions, and perform N-1 downsampling on the training even-numbered field data to obtain N-1 Even field data with different resolutions; combine odd field data and even field data with the same resolution in N-1 odd field data with different resolutions and N-1 even field data with different resolutions , to obtain N-1 down-sampled images of different resolutions for training;
  • Step 74 Input the training images of N resolutions including the video images for training and the downsampling images for training of the N-1 resolutions into the deinterlacing network to be trained for deinterlacing to obtain N resolutions.
  • the output images of the N resolutions are sequentially increased in resolution from the Nth resolution image to the first resolution image, and the to-be-trained deinterlacing network includes N series deinterlacing an interleaving sub-network, where the images processed by the N series-connected de-interlacing sub-networks are generated based on the images of the N resolutions respectively;
  • Step 75 Calculate the loss of the output images of the N resolutions, and calculate the total loss of the deinterlacing network to be trained according to the loss of the output images of the N resolutions, and optimize the to-be-trained deinterlacing network according to the total loss. Train the parameters of the deinterlacing network to get the trained deinterleaving network.
  • the loss is an L2 loss.
  • L2 loss any loss that is possible.
  • the total loss is equal to the sum of the losses of the output images of the N resolutions, or equal to the weighted sum of the losses of the output images of the N resolutions.
  • the loss of each output image is obtained according to the output image and the corresponding ground-truth image of the output image.
  • the total loss of the deinterlacing network is equal to the sum of the losses of the output images Output, Out_x2 and Out_x4 of the three deinterlacing sub-networks, or, equal to the three deinterlacing sub-networks.
  • the weighted sum of the losses of the output images Output, Out_x2 and Out_x4 of the interleaving sub-network is calculated, and the sum of the total losses is calculated, and the parameters of the deinterleaving network are updated according to the calculated total losses.
  • an embodiment of the present disclosure further provides a video image deinterlacing apparatus 90, including:
  • the first acquisition module 91 is used to acquire a single-frame original video image containing parity field information
  • the first extraction module 92 is used for extracting odd-numbered field data and even-numbered field data in the original video image
  • the first down-sampling module 93 is configured to perform N-1 downsampling on the odd-numbered field data to obtain N-1 odd-numbered field data with different resolutions, and perform N-1 downsampling on the even-numbered field data to obtain N-1.
  • -1 even field data of different resolutions odd field data and even field data with the same resolution in N-1 odd field data of different resolutions and N-1 even field data of different resolutions Merge to obtain N-1 downsampled images of different resolutions;
  • the de-interlacing module 94 is configured to input images of N resolutions including the original video image and the down-sampled images of the N-1 resolutions into a de-interlacing network to perform de-interlacing processing to obtain a de-interlaced image, where In the images of the N resolutions, the resolutions of the images of the Nth resolution to the images of the first resolution are sequentially increased, and the deinterlacing network includes N deinterlacing sub-networks connected in series. The images processed by the concatenated deinterlacing sub-networks are respectively generated based on the images of the N resolutions;
  • N is a positive integer greater than or equal to 2.
  • the odd-numbered field data and the even-numbered field data of the extracted original video image are downsampled to multiple resolutions, and deinterlaced at multiple resolutions, which can ensure that the original video image will not be damaged during downsampling.
  • the information of the video image and the progressive deinterlacing can achieve a better deinterlacing effect.
  • the deinterleaving module 94 includes:
  • the first deinterlacing submodule is used for splicing two down-sampled images of the Nth resolution; inputting the spliced images of the Nth resolution into the Nth serial deinterlacing sub-network, Obtaining a deinterlaced image of the Nth resolution; upsampling the deinterlaced image of the Nth resolution to obtain an upsampled image of the N-1th resolution;
  • the second de-interlacing submodule is used for splicing the up-sampled image of the ith resolution and the down-sampled image of the ith resolution to obtain the stitched image of the ith resolution;
  • the spliced image is input into the i-th deinterlacing sub-network to obtain a de-interlaced image of the i-th resolution;
  • the de-interlaced image of the i-th resolution is up-sampling to obtain the i-1th resolution.
  • Sampling image wherein, i is an integer greater than or equal to 2 and less than N;
  • the third de-interlacing sub-module is used to stitch the upsampled image of the first resolution and the original video image of the first resolution to obtain a stitched image of the first resolution;
  • the stitched image is input to the first de-interlacing network to obtain a de-interlacing image of the first resolution, which is used as the output image of the de-interlacing network.
  • the N is 3 or 4.
  • the resolutions of images of adjacent resolutions among the N resolution images are in a 2-fold relationship.
  • a bicubic interpolation method is used to perform up-sampling and/or down-sampling.
  • the N series-connected deinterleaving sub-networks have the same structure and different parameters.
  • each of the de-interleaving sub-networks includes a plurality of filters in series, each filter includes a plurality of convolution kernels in series, and among the plurality of filters in series, every two The filters are of the same resolution, and the output of each filter except the last serves as the input for the next filter and the filter with the same resolution.
  • the video image deinterlacing apparatus further includes:
  • a training module for training the deinterlacing network to be trained to obtain the deinterleaving network
  • the training of the deinterlaced network to be trained includes:
  • the training images of N resolutions including the video images for training and the downsampling images for training of the N-1 resolutions into the deinterlacing network to be trained for deinterlacing processing to obtain outputs of N resolutions image, among the images of the N resolutions, the resolutions from the image of the Nth resolution to the image of the first resolution are sequentially increased, and the deinterlacing network to be trained includes N deinterlacing sub-networks connected in series , the images processed by the N concatenated deinterlacing sub-networks are generated based on the images of the N resolutions respectively;
  • the loss is an L2 loss.
  • the total loss is equal to the sum of the losses of the output images of the N resolutions, or equal to the weighted sum of the losses of the output images of the N resolutions.
  • an embodiment of the present application further provides an electronic device 100 , including a processor 101 , a memory 102 , a program or instruction stored in the memory 102 and executable on the processor 101 , the program or instruction When executed by the processor 101, each process of the above embodiments of the video image deinterlacing method is implemented, and the same technical effect can be achieved. To avoid repetition, details are not described here.
  • the embodiments of the present application further provide a non-volatile computer-readable storage medium, where a program or an instruction is stored on the non-volatile computer-readable storage medium, and when the program or instruction is executed by a processor, the above-mentioned video image removal is realized.
  • the various processes in the embodiments of the interleaving method can achieve the same technical effect, and are not repeated here in order to avoid repetition.
  • the processor is the processor in the terminal described in the foregoing embodiment.
  • the non-volatile computer-readable storage medium such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Systems (AREA)

Abstract

提供了一种视频图像去交错方法和视频图像去交错装置，方法包括：获取单帧原始视频图像（11）；提取原始视频图像中的奇数场数据和偶数场数据（12）；对奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将具有相同分辨率的奇数场数据和偶数场数据进行合并，得到下采样图像（13）；将原始视频图像和下采样图像输入至去交错网络中进行去交错处理（14）。

Description

视频图像去交错方法和视频图像去交错装置 技术领域
本公开实施例涉及图像处理技术领域，尤其涉及一种视频图像去交错方法和视频图像去交错装置。
背景技术
现有技术中，针对逐行扫描的显示装置，如果接收到的视频图像为隔行扫描的视频图像，需要将隔行扫描的视频图像进行去交错处理，得到逐行扫描的视频图像。在对隔行扫描的视频图像进行去交错处理时，如何提高去交错效果，是亟待解决的问题。
发明内容
第一方面，本公开实施例提供了一种视频图像去交错方法，包括：
获取包含奇偶场信息的单帧原始视频图像；
提取所述原始视频图像中的奇数场数据和偶数场数据；
对所述奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的下采样图像；
将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
其中，N为大于或等于2的正整数。
可选的，将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像包括：
将两个所述第N个分辨率的下采样图像进行拼接，得到第N个分辨率的拼接图像；将所述第N个分辨率的拼接图像输入至第N个串联的去交错子网络中，得到第N个分辨率的去交错图像；对所述第N个分辨率的去交错图像进行上采样，得到第N-1个分辨率的上采样图像；
将第i个分辨率的上采样图像和第i个分辨率的下采样图像进行拼接，得到第i个分辨率的拼接图像；将所述第i个分辨率的拼接图像输入至第i个去交错子网络中得到第i个分辨率的去交错图像；对所述第i个分辨率的去交错图像进行上采样处理，得到第i-1分辨率的上采样图像；其中，i为大于或等于2且小于N的整数；
将第一个分辨率的上采样图像和第一个分辨率的原始视频图像进行拼接，得到第一个分辨率的拼接图像；将所述第一个分辨率的拼接图像输入至第一个去交错网络中得到第一个分辨率的去交错图像，作为所述去交错网络的输出图像。
可选的，将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并包括：
将具有相同分辨率的奇数场数据和偶数场数据按照行间隔排布。
可选的，所述N为3或4。
可选的，所述N个分辨率的图像中相邻分辨率的图像的分辨率呈2倍关系。
可选的，采用双三次插值方法进行上采样和/或下采样。
可选的，所述N个串联的去交错子网络的结构相同，参数不同。
可选的，每个所述去交错子网络包括多个串联的滤波器，每个滤波器包括多个串联的卷积核，多个串联的滤波器中，每两个滤波器的分辨率相同，除最后一个滤波器之外，其余每个滤波器的输出作为下一个滤波器以及与其具有相同分辨率的滤波器的输入。
可选的，所述方法还包括：
对待训练去交错网络进行训练，得到所述去交错网络；
其中，对待训练去交错网络进行训练包括：
获取包含奇偶场信息的单帧训练用视频图像；
提取所述训练用视频图像中的训练用奇数场数据和偶数场数据；
对所述训练用奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述训练用偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的训练用下采样图像；
将包含所述训练用视频图像和所述N-1个分辨率的训练用下采样图像的N个分辨率的训练图像输入至待训练去交错网络中进行去交错处理得到N个分辨率的输出图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述待训练去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
计算所述N个分辨率的输出图像的损失，并根据所述N个分辨率的输出图像的损失计算所述待训练去交错网络的总损失，根据所述总损失优化所述待训练去交错网络的参数，得到训练后的去交错网络。
可选的，所述损失为L2损失。
可选的，所述总损失等于所述N个分辨率的输出图像的损失之和，或者，等于所述N个分辨率的输出图像的损失加权后之和。
第二方面，本公开实施例提供了一种视频图像去交错装置，包括：
第一获取模块，用于获取包含奇偶场信息的单帧原始视频图像；
第一提取模块，用于提取所述原始视频图像中的奇数场数据和偶数场数据；
第一下采样模块，用于对所述奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的下采样图像；
去交错模块，用于将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错 图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
其中，N为大于或等于2的正整数。
第三方面，本公开实施例提供了一种电子设备，包括处理器，存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现上述第一方面所述的视频图像去交错方法的步骤。
第四方面，本公开实施例提供了一种非易失性计算机可读存储介质，所述非易失性计算机可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现上述第一方面所述的视频图像去交错方法的步骤。
附图说明
图1为本公开实施例的视频图像去交错方法的流程示意图；
图2为本公开实施例的从原始视频图像中提取奇数场数据和偶数场数据的示意图；
图3和图4为本公开实施例的图像的下采样方法的示意图；
图5为本公开一实施例的去交错网络的示意图；
图6为本公开一实施例的去交错子网络的结构示意图；
图7为本公开实施例的去交错网络的训练方法的流程示意图；
图8为本公开实施例的去交错网络的总损失的计算方法示意图；
图9为本公开实施例的视频图像去交错装置的结构示意图；
图10为本公开实施例的电子设备的结构示意图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
请参考图1，本公开实施例提供一种视频图像去交错方法，包括：
步骤11：获取包含奇偶场信息的单帧原始视频图像；
所述原始视频图像为通过隔行扫描方式获取的视频图像，隔行扫描时电子束首先从左到右、从上到下扫描所有的奇数行形成一场图像数据，然后电子束又回到顶端，再次从左到右、从上到下扫描所有的偶数行形成另一场图像数据。这两个垂直方向交换显示的扫描场构成每一帧完整的视频图像。
步骤12：提取所述原始视频图像中的奇数场数据和偶数场数据；
请参考图2，图2为本公开实施例的从原始视频图像中提取奇数场数据和偶数场数据的示意图。
步骤13：对所述奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的下采样图像；
举例来说，分别对原始视频图像的奇数场数据和偶数场数据下采样2次，请参考图3和图4，分别为下采样2倍和4倍，假设奇数场数据和偶数场数据的分辨率均为256×256，下采样2倍得到128×128分辨率的奇数场数据和偶数场数据，下采样4倍得到64×64分辨率的奇数场数据和偶数场数据。
本公开实施例中，可以按照原始视频图像中的奇数场数据和偶数场数据的拼接方式，合并下采样之后的奇数场数据和偶数场数据。
可选的，请参考图3和图4，将具有相同分辨率的奇数场数据和偶数场数据按照行间隔排布，即交错合并。
步骤14：将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
其中，N为大于或等于2的正整数。
本公开实施例中，所述N个去交错网络分别用于处理所述N个分辨率中 的其中一种分辨率的图像。
本公开实施例中，所述去交错网络最终输出的去交错图像为逐行扫描的视频图像。
本公开实施例中，对提取出的原始视频图像的奇数场数据和偶数场数据进行下采样至多个分辨率上，并在多个分辨率上进行去交错，可以保证在下采样时不会破坏原始视频图像的信息，并且进行递进式去交错，能够达到更好的去交错效果。
下面对本公开实施例的去交错网络的工作过程进行说明。
本公开实施例中，可选的，将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像包括：
对于最小分辨率的图像执行以下操作：将两个所述第N个分辨率的下采样图像进行拼接，得到第N个分辨率的拼接图像；将所述第N个分辨率的拼接图像输入至第N个串联的去交错子网络中，得到第N个分辨率的去交错图像；对所述第N个分辨率的去交错图像进行上采样，得到第N-1个分辨率的上采样图像；
对于中间分辨率的图像执行以下操作：将第i个分辨率的上采样图像和第i个分辨率的下采样图像进行拼接，得到第i个分辨率的拼接图像；将所述第i个分辨率的拼接图像输入至第i个去交错子网络中得到第i个分辨率的去交错图像；对所述第i个分辨率的去交错图像进行上采样处理，得到第i-1分辨率的上采样图像；其中，i为大于或等于2且小于N的整数；
对于最大分辨率的图像执行以下操作：将第一个分辨率的上采样图像和第一个分辨率的原始视频图像进行拼接，得到第一个分辨率的拼接图像；将所述第一个分辨率的拼接图像输入至第一个去交错网络中得到第一个分辨率的去交错图像，作为所述去交错网络的输出图像。
本公开实施例中，对视频图像进行去交错处理时，从低分辨率开始，不断提升分辨率，进行递进式去交错，从而达到更好的去交错效果，且由于下采样时分别对提取出的原始视频图像的奇数场数据和偶数场数据进行下采样，可以保证在下采样时不会破坏原始视频图像的信息。
本公开实施例中，可选的，所述N为3或4，以达到较好的去交错效果，且能够有效降低实现成本。当然，也不排除N为其他大于或等于2的数值。
本公开实施例中，可选的，所述N个分辨率的图像中相邻分辨率的图像的分辨率呈2倍关系，举例来说，所述N个分辨率的图像的分辨率分别为256×256，128×128，64×64。
本公开实施例中，可选的，采用双三次插值方法进行上采样和/或下采样，以保留更好的图像细节。当然，上采样和/或下采样也可以采用其他插值方法，例如双线性插值等。
请参考图5，图5为本公开一实施例的去交错网络的示意图，该实施例中，去交错网络包括3个去交错子网络：去交错子网络1、去交错子网络2和去交错子网络3。
该去交错网络的工作过程如下：
1)提取原始视频图像中的奇数场数据和偶数场数据；
2)分别对原始视频图像的奇数场数据和偶数场数据下采样2次，分别为下采样2倍和4倍，得到奇数场数据和偶数场数据的下采样图像Down_x2 奇数、Down_x2 偶数、Down_x4 奇数和Down_x4 偶数，将Down_x2 奇数和Down_x2 偶数合并得到Down_x2，将Down_x4 奇数和Down_x4 偶数合并得到Down_x4；
3)将两个Down_x4进行拼接，得到拼接图像，将拼接图像输入至去交错子网络3中进行去交错处理得到去交错图像out_x4；
4)将去交错图像out_x4上采样2倍，得到Up_x2，将Up_x2和Down_x2进行拼接，得到拼接图像，将拼接图像输入至去交错子网络2中进行去交错处理得到去交错图像out_x2；
5)将去交错图像out_x2上采样2倍，得到Up_x1，将Up_x1和与原始视频图像进行拼接，得到拼接图像，将拼接图像输入至去交错子网络1中进行去交错处理得到最终的输出。
本公开实施例中，可选的，所述N个串联的去交错子网络的结构相同，参数不同。
请参考图6，图6为本公开一实施例的去交错子网络的结构示意图，该去交错子网络包括：多个串联的滤波器(filter)，每个滤波器包括多个串联的卷积核(图6中的竖条)。图6所示的实施例中，每个滤波器包括四个串联的 卷积核，当然，在本公开的其他一些实施例中，滤波器中的卷积核的个数也不限于4个。图6所示的实施例中，填充有斜纹的竖条表示下采样，填充有点状的竖条表示上采样。本公开实施例中，多个串联的滤波器中，每两个滤波器的分辨率相同，除最后一个滤波器之外，其余每个滤波器的输出作为下一个滤波器以及与其具有相同分辨率的滤波器的输入。图6所示的实施例中，去交错子网络包括6个串联的滤波器，其中，第一个滤波器与第六个滤波器的分辨率相同，第二个滤波器与第五个滤波器的分辨率相同，第三个滤波器与第四个滤波器的分辨率相同，第一个滤波器的输出作为第二个滤波器和第六个滤波器(与第一个滤波器的分辨率相同)的输入，第二个滤波器的输出作为第三个滤波器和第五个滤波器(与第二个滤波器的分辨率相同)的输入，第三个滤波器的输出作为第四个滤波器(与第三个滤波器的分辨率相同)的输入。
本公开实施例中，可选的，所述视频图像去交错方法还包括：对待训练去交错网络进行训练，得到所述去交错网络；请参考图7，对待训练去交错网络进行训练包括：
步骤71：获取包含奇偶场信息的单帧训练用视频图像；
步骤72：提取所述训练用视频图像中的训练用奇数场数据和偶数场数据；
步骤73：对所述训练用奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述训练用偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的训练用下采样图像；
步骤74：将包含所述训练用视频图像和所述N-1个分辨率的训练用下采样图像的N个分辨率的训练图像输入至待训练去交错网络中进行去交错处理得到N个分辨率的输出图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述待训练去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
步骤75：计算所述N个分辨率的输出图像的损失，并根据所述N个分辨 率的输出图像的损失计算所述待训练去交错网络的总损失，根据所述总损失优化所述待训练去交错网络的参数，得到训练后的去交错网络。
本公开实施例中，可选的，所述损失为L2损失。当然，也可以为其他类型的损失。
本公开实施例中，可选的，所述总损失等于所述N个分辨率的输出图像的损失之和，或者，等于所述N个分辨率的输出图像的损失加权后之和。每个输出图像的损失，根据该输出图像和该输出图像对应的真值图像得到。
以图5中的去交错网络的训练为例，请参考图8，去交错网络的总损失等于三个去交错子网络的输出图像Output、Out_x2和Out_x4的损失之和，或者，等于三个去交错子网络的输出图像Output、Out_x2和Out_x4的损失加权后之和，计算出总损失之和，根据计算得到的总损失更新去交错网络的参数。
请参考图9，本公开实施例还提供一种视频图像去交错装置90，包括：
第一获取模块91，用于获取包含奇偶场信息的单帧原始视频图像；
第一提取模块92，用于提取所述原始视频图像中的奇数场数据和偶数场数据；
第一下采样模块93，用于对所述奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的下采样图像；
去交错模块94，用于将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
其中，N为大于或等于2的正整数。
本公开实施例中，对提取出的原始视频图像的奇数场数据和偶数场数据 进行下采样至多个分辨率上，并在多个分辨率上进行去交错，可以保证在下采样时不会破坏原始视频图像的信息，并且进行递进式去交错，能够达到更好的去交错效果。
本公开实施例中，可选的，所述去交错模块94包括：
第一去交错子模块，用于将两个所述第N个分辨率的下采样图像进行拼接；将所述第N个分辨率的拼接图像输入至第N个串联的去交错子网络中，得到第N个分辨率的去交错图像；对所述第N个分辨率的去交错图像进行上采样，得到第N-1个分辨率的上采样图像；
第二去交错子模块，用于将第i个分辨率的上采样图像和第i个分辨率的下采样图像进行拼接，得到第i个分辨率的拼接图像；将所述第i个分辨率的拼接图像输入至第i个去交错子网络中得到第i个分辨率的去交错图像；对所述第i个分辨率的去交错图像进行上采样处理，得到第i-1分辨率的上采样图像；其中，i为大于或等于2且小于N的整数；
第三去交错子模块，用于将第一个分辨率的上采样图像和第一个分辨率的原始视频图像进行拼接，得到第一个分辨率的拼接图像；将所述第一个分辨率的拼接图像输入至第一个去交错网络中得到第一个分辨率的去交错图像，作为所述去交错网络的输出图像。
本公开实施例中，可选的，所述N为3或4。
本公开实施例中，可选的，所述N个分辨率的图像中相邻分辨率的图像的分辨率呈2倍关系。
本公开实施例中，可选的，采用双三次插值方法进行上采样和/或下采样。
本公开实施例中，可选的，所述N个串联的去交错子网络的结构相同，参数不同。
本公开实施例中，可选的，每个所述去交错子网络包括多个串联的滤波器，每个滤波器包括多个串联的卷积核，多个串联的滤波器中，每两个滤波器的分辨率相同，除最后一个滤波器之外，其余每个滤波器的输出作为下一个滤波器以及与其具有相同分辨率的滤波器的输入。
本公开实施例中，可选的，所述视频图像去交错装置还包括：
训练模块，用于对待训练去交错网络进行训练，得到所述去交错网络；
其中，对待训练去交错网络进行训练包括：
获取包含奇偶场信息的单帧训练用视频图像；
提取所述训练用视频图像中的训练用奇数场数据和偶数场数据；
对所述训练用奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述训练用偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的训练用下采样图像；
将包含所述训练用视频图像和所述N-1个分辨率的训练用下采样图像的N个分辨率的训练图像输入至待训练去交错网络中进行去交错处理得到N个分辨率的输出图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述待训练去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
计算所述N个分辨率的输出图像的损失，并根据所述N个分辨率的输出图像的损失计算所述待训练去交错网络的总损失，根据所述总损失优化所述待训练去交错网络的参数，得到训练后的去交错网络。
本公开实施例中，可选的，所述损失为L2损失。
本公开实施例中，可选的，所述总损失等于所述N个分辨率的输出图像的损失之和，或者，等于所述N个分辨率的输出图像的损失加权后之和。
如图10所示，本申请实施例还提供一种电子设备100，包括处理器101，存储器102，存储在存储器102上并可在所述处理器101上运行的程序或指令，该程序或指令被处理器101执行时实现上述视频图像去交错方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本申请实施例还提供一种非易失性计算机可读存储介质，所述非易失性计算机可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述视频图像去交错方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的终端中的处理器。所述非易失 性计算机可读存储介质，如计算机只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等。
上面结合附图对本公开的实施例进行了描述，但是本公开并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本公开的启示下，在不脱离本公开宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本公开的保护之内。

Claims (14)

  1. 一种视频图像去交错方法，其特征在于，包括：
    获取包含奇偶场信息的单帧原始视频图像；
    提取所述原始视频图像中的奇数场数据和偶数场数据；
    对所述奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的下采样图像；
    将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
    其中，N为大于或等于2的正整数。
  2. 根据权利要求1所述的方法，其特征在于，将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像包括：
    将两个所述第N个分辨率的下采样图像进行拼接，得到第N个分辨率的拼接图像；将所述第N个分辨率的拼接图像输入至第N个串联的去交错子网络中，得到第N个分辨率的去交错图像；对所述第N个分辨率的去交错图像进行上采样，得到第N-1个分辨率的上采样图像；
    将第i个分辨率的上采样图像和第i个分辨率的下采样图像进行拼接，得到第i个分辨率的拼接图像；将所述第i个分辨率的拼接图像输入至第i个去交错子网络中得到第i个分辨率的去交错图像；对所述第i个分辨率的去交错图像进行上采样处理，得到第i-1分辨率的上采样图像；其中，i为大于或等于2且小于N的整数；
    将第一个分辨率的上采样图像和第一个分辨率的原始视频图像进行拼接， 得到第一个分辨率的拼接图像；将所述第一个分辨率的拼接图像输入至第一个去交错网络中得到第一个分辨率的去交错图像，作为所述去交错网络的输出图像。
  3. 根据权利要求1或2所述的方法，其特征在于，将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并包括：
    将具有相同分辨率的奇数场数据和偶数场数据按照行间隔排布。
  4. 根据权利要求1或2所述的方法，其特征在于，所述N为3或4。
  5. 根据权利要求1或2所述的方法，其特征在于，所述N个分辨率的图像中相邻分辨率的图像的分辨率呈2倍关系。
  6. 根据权利要求2所述的方法，其特征在于，采用双三次插值方法进行上采样和/或上采样。
  7. 根据权利要求1或2所述的方法，其特征在于，所述N个串联的去交错子网络的结构相同，参数不同。
  8. 根据权利要求1或2所述的方法，其特征在于，每个所述去交错子网络包括多个串联的滤波器，每个滤波器包括多个串联的卷积核，多个串联的滤波器中，每两个滤波器的分辨率相同，除最后一个滤波器之外，其余每个滤波器的输出作为下一个滤波器以及与其具有相同分辨率的滤波器的输入。
  9. 根据权利要求1所述的方法，其特征在于，还包括：
    对待训练去交错网络进行训练，得到所述去交错网络；
    其中，对待训练去交错网络进行训练包括：
    获取包含奇偶场信息的单帧训练用视频图像；
    提取所述训练用视频图像中的训练用奇数场数据和偶数场数据；
    对所述训练用奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述训练用偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的训练用下采样图像；
    将包含所述训练用视频图像和所述N-1个分辨率的训练用下采样图像的 N个分辨率的训练图像输入至待训练去交错网络中进行去交错处理得到N个分辨率的输出图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述待训练去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
    计算所述N个分辨率的输出图像的损失，并根据所述N个分辨率的输出图像的损失计算所述待训练去交错网络的总损失，根据所述总损失优化所述待训练去交错网络的参数，得到训练后的去交错网络。
  10. 根据权利要求9所述的方法，其特征在于，所述损失为L2损失。
  11. 根据权利要求9所述的方法，其特征在于，所述总损失等于所述N个分辨率的输出图像的损失之和，或者，等于所述N个分辨率的输出图像的损失加权后之和。
  12. 一种视频图像去交错装置，其特征在于，包括：
    第一获取模块，用于获取包含奇偶场信息的单帧原始视频图像；
    第一提取模块，用于提取所述原始视频图像中的奇数场数据和偶数场数据；
    第一下采样模块，用于对所述奇数场数据进行N-1次下采样得到N-1个不同分辨率的奇数场数据，对所述偶数场数据进行N-1次下采样得到N-1个不同分辨率的偶数场数据；将N-1个不同分辨率的奇数场数据中和N-1个不同分辨率的偶数场数据中的具有相同分辨率的奇数场数据和偶数场数据进行合并，得到N-1个不同分辨率的下采样图像；
    去交错模块，用于将包含所述原始视频图像和所述N-1个分辨率的下采样图像的N个分辨率的图像输入至去交错网络中进行去交错处理得到去交错图像，所述N个分辨率的图像中，从第N个分辨率的图像至第一个分辨率的图像的分辨率依次递增，所述去交错网络包括N个串联的去交错子网络，所述N个串联的去交错子网络所处理的图像分别基于所述N个分辨率的图像生成；
    其中，N为大于或等于2的正整数。
  13. 一种电子设备，其特征在于，包括处理器，存储器及存储在所述存 储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1至11任一项所述的视频图像去交错方法的步骤。
  14. 一种非易失性计算机可读存储介质，其特征在于，所述非易失性计算机可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至11任一项所述的视频图像去交错方法的步骤。
PCT/CN2021/078598 2021-03-02 2021-03-02 视频图像去交错方法和视频图像去交错装置 WO2022183358A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/625,714 US11711491B2 (en) 2021-03-02 2021-03-02 Video image de-interlacing method and video image de-interlacing device
CN202180000385.XA CN115868157A (zh) 2021-03-02 2021-03-02 视频图像去交错方法和视频图像去交错装置
PCT/CN2021/078598 WO2022183358A1 (zh) 2021-03-02 2021-03-02 视频图像去交错方法和视频图像去交错装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/078598 WO2022183358A1 (zh) 2021-03-02 2021-03-02 视频图像去交错方法和视频图像去交错装置

Publications (1)

Publication Number Publication Date
WO2022183358A1 true WO2022183358A1 (zh) 2022-09-09

Family

ID=83153710

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/078598 WO2022183358A1 (zh) 2021-03-02 2021-03-02 视频图像去交错方法和视频图像去交错装置

Country Status (3)

Country Link
US (1) US11711491B2 (zh)
CN (1) CN115868157A (zh)
WO (1) WO2022183358A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060077809A (ko) * 2004-12-31 2006-07-05 삼성전자주식회사 다른 해상도에 기반한 최적화 필터를 이용하는디인터레이싱 장치 및 방법
EP1931136A1 (en) * 2006-12-08 2008-06-11 Matsushita Electric Industrial Co., Ltd. Block-based line combination algorithm for de-interlacing
US20100177241A1 (en) * 2009-01-12 2010-07-15 Himax Technologies Limited Apparatus and method for motion adaptive de-interlacing with chroma up-sampling error remover
CN102045531A (zh) * 2010-12-31 2011-05-04 福州瑞芯微电子有限公司 一种移动终端的去交错视频显示的实现方法
CN102665060A (zh) * 2012-04-25 2012-09-12 中国科学技术大学 从交错式格式视频到渐进式格式视频的转换方法
CN103905769A (zh) * 2012-12-26 2014-07-02 苏州赛源微电子有限公司 无本地帧缓存的视频去隔行算法及解决方案
CN109688360A (zh) * 2018-12-07 2019-04-26 成都东方盛行电子有限责任公司 一种隔行视频缩放采样方法
CN109714547A (zh) * 2018-12-17 2019-05-03 珠海亿智电子科技有限公司 一种基于模式识别的隔行视频转换方法

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8218091B2 (en) 2006-04-18 2012-07-10 Marvell World Trade Ltd. Shared memory multi video channel display apparatus and methods
US20080174694A1 (en) * 2007-01-22 2008-07-24 Horizon Semiconductors Ltd. Method and apparatus for video pixel interpolation
JP5427785B2 (ja) 2007-09-28 2014-02-26 ドルビー ラボラトリーズ ライセンシング コーポレイション ビデオ圧縮技法及びビデオ伝達技法
ES2395363T3 (es) 2008-12-25 2013-02-12 Dolby Laboratories Licensing Corporation Reconstrucción de vistas desentrelazadas, utilizando interpolación adaptativa basada en disparidades entre las vistas para un muestro ascendente
CN101827206B (zh) 2009-03-02 2011-09-07 奇景光电股份有限公司 具有cue移除器的动态适应性去交错装置与方法
WO2010123855A1 (en) 2009-04-20 2010-10-28 Dolby Laboratories Licensing Corporation Filter selection for video pre-processing in video applications
CN107241606B (zh) 2011-12-17 2020-02-21 杜比实验室特许公司 解码系统、方法和设备以及计算机可读介质
CN104662896B (zh) 2012-09-06 2017-11-28 诺基亚技术有限公司 用于图像处理的装置和方法
CA2886174C (en) 2012-10-07 2018-07-10 Numeri Ltd. Video compression method
US9661340B2 (en) 2012-10-22 2017-05-23 Microsoft Technology Licensing, Llc Band separation filtering / inverse filtering for frame packing / unpacking higher resolution chroma sampling formats
JP6560230B2 (ja) 2014-01-02 2019-08-14 ヴィド スケール インコーポレイテッド インターレースおよびプログレッシブ混合のコンテンツを用いるスケーラブルビデオコーディングのための方法およびシステム
US10991281B2 (en) 2016-02-22 2021-04-27 Dolby Laboratories Licensing Corporation Apparatus and method for encoding high frame rate content in standard frame rate video using temporal interlacing
CN108111833A (zh) 2016-11-24 2018-06-01 阿里巴巴集团控股有限公司 用于立体视频编解码的方法、装置及系统
CN110855903A (zh) 2019-11-19 2020-02-28 长沙品先信息技术有限公司 一种多路视频实时拼接方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060077809A (ko) * 2004-12-31 2006-07-05 삼성전자주식회사 다른 해상도에 기반한 최적화 필터를 이용하는디인터레이싱 장치 및 방법
EP1931136A1 (en) * 2006-12-08 2008-06-11 Matsushita Electric Industrial Co., Ltd. Block-based line combination algorithm for de-interlacing
US20100177241A1 (en) * 2009-01-12 2010-07-15 Himax Technologies Limited Apparatus and method for motion adaptive de-interlacing with chroma up-sampling error remover
CN102045531A (zh) * 2010-12-31 2011-05-04 福州瑞芯微电子有限公司 一种移动终端的去交错视频显示的实现方法
CN102665060A (zh) * 2012-04-25 2012-09-12 中国科学技术大学 从交错式格式视频到渐进式格式视频的转换方法
CN103905769A (zh) * 2012-12-26 2014-07-02 苏州赛源微电子有限公司 无本地帧缓存的视频去隔行算法及解决方案
CN109688360A (zh) * 2018-12-07 2019-04-26 成都东方盛行电子有限责任公司 一种隔行视频缩放采样方法
CN109714547A (zh) * 2018-12-17 2019-05-03 珠海亿智电子科技有限公司 一种基于模式识别的隔行视频转换方法

Also Published As

Publication number Publication date
US11711491B2 (en) 2023-07-25
US20230156147A1 (en) 2023-05-18
CN115868157A (zh) 2023-03-28

Similar Documents

Publication Publication Date Title
US20180315165A1 (en) Apparatus for upscaling an image, method for training the same, and method for upscaling an image
JPH07118627B2 (ja) 飛越し型デジタル・ビデオ入力フイルタ／間引き器および／または伸張器／補間器フイルタ
CN102868890B (zh) 图像处理设备、成像设备和图像处理方法
CN1832564A (zh) 图像处理装置和方法
US8115864B2 (en) Method and apparatus for reconstructing image
KR20190059157A (ko) 영상 화질 개선방법 및 그 장치
CN112017116A (zh) 基于非对称卷积的图像超分辨率重建网络及其构建方法
WO2022183358A1 (zh) 视频图像去交错方法和视频图像去交错装置
JPH07212652A (ja) ビデオ特殊効果装置
US8902474B2 (en) Image processing apparatus, control method of the same, and program
US8879864B2 (en) Generating sub-frame images for driving display
KR101979584B1 (ko) 디인터레이싱 방법 및 장치
US20080165277A1 (en) Systems and Methods for Deinterlacing Video Data
JP2011040004A (ja) 画像処理装置及び画像処理方法
JP5762897B2 (ja) 撮像装置および評価値生成装置
JP5058720B2 (ja) 映像高解像度化装置及び方法
CN112668619A (zh) 图像处理方法、装置、终端及存储介质
JP2006510977A (ja) 画像スケーリング
US8130299B2 (en) Method of performing interlaced preview for a CMOS sensor
Morsy Performance analysis of the effects of non-adaptive image scaling on image edges
JP5087845B2 (ja) 画像ディスプレイ回路および画像表示処理方法
JP2015061178A (ja) 撮像システム、撮像システムの駆動方法、撮像装置、撮像装置の駆動方法
US7961971B2 (en) Method and apparatus for performing parallel box filtering in region based image processing
CN116363247B (zh) 高像素图像的ct重建方法、装置及存储介质
JP7274367B2 (ja) フレームレート変換モデル学習装置およびフレームレート変換装置、ならびに、それらのプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21928449

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21928449

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15-01-2024)