CN111294545A - Image data interpolation method and device, storage medium and terminal - Google Patents

Image data interpolation method and device, storage medium and terminal Download PDF

Info

Publication number
CN111294545A
CN111294545A CN201910188662.9A CN201910188662A CN111294545A CN 111294545 A CN111294545 A CN 111294545A CN 201910188662 A CN201910188662 A CN 201910188662A CN 111294545 A CN111294545 A CN 111294545A
Authority
CN
China
Prior art keywords
interpolated
point
pixel
pixel value
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910188662.9A
Other languages
Chinese (zh)
Other versions
CN111294545B (en
Inventor
孙振燕
罗小伟
李�荣
林福辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Spreadtrum Communications Shanghai Co Ltd
Original Assignee
Spreadtrum Communications Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Spreadtrum Communications Shanghai Co Ltd filed Critical Spreadtrum Communications Shanghai Co Ltd
Priority to CN201910188662.9A priority Critical patent/CN111294545B/en
Publication of CN111294545A publication Critical patent/CN111294545A/en
Application granted granted Critical
Publication of CN111294545B publication Critical patent/CN111294545B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

An image data interpolation method and device, a storage medium and a terminal are provided, the method comprises the following steps: carrying out interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, wherein each field comprises a plurality of points to be interpolated and a plurality of pixel points; calculating the current to-be-interpolated point in each field by adopting a time domain interpolation method to obtain a predicted pixel value of the current to-be-interpolated point; determining a pixel value of a first pixel point adjacent to a current point to be interpolated, wherein the horizontal coordinate of the first pixel point is the same as that of the current point to be interpolated, and the first pixel point and the current point to be interpolated are positioned in the same field; performing difference detection on at least the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point to obtain a difference detection result; and when the difference detection result exceeds a first preset threshold value, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method. The technical scheme provided by the invention can improve the interpolation accuracy and the performance of the de-interlacing algorithm.

Description

Image data interpolation method and device, storage medium and terminal
Technical Field
The invention relates to the technical field of image processing, in particular to an image data interpolation method and device, a storage medium and a terminal.
Background
At present, most television systems and Digital Video Disk (DVD) playing devices adopt an interlaced scanning technology. The interlaced scanning can effectively save the bandwidth required by transmission, but can cause the phenomena of line-to-line flicker, field-to-field flicker, sawtooth effect and the like in the display process, thereby seriously influencing the sensory experience of users. In order to improve user experience, de-interlacing (de-interlace) algorithm is widely applied as a video post-processing link. The de-interlacing algorithm can fill in the pixel values of missing points (also called to-be-interpolated points) introduced by interlaced scanning through prediction, so that the display effect is improved.
The de-interlacing algorithm can be divided into three categories: single field de-interlacing algorithms, inter-field de-interlacing algorithms, and hybrid de-interlacing algorithms. The single field de-interlacing algorithm utilizes spatial correlation and adopts modes of linear filtering, line replication, interpolation algorithm based on boundary direction and the like to carry out interpolation. The linear filtering and line copying method is simple and easy to implement, but the phenomena of sawtooth and the like are easily caused. The interpolation algorithm based on the boundary direction mainly utilizes the visual characteristics of human eyes to predict and interpolate along the boundary direction, thereby improving the subjective quality of the image. The inter-field de-interlacing algorithm predicts the missing point by utilizing the correlation between a spatial domain and a temporal domain, generally by adopting technologies such as spatial interpolation and temporal interpolation combination, motion detection and spatial interpolation combination, motion compensation and the like, wherein the motion compensation technology can effectively predict a motion region, but has relatively high calculation complexity and is not suitable for real-time processing and display. Hybrid de-interlacing algorithms generally refer to dynamic adaptive de-interlacing, where an image is divided into static and dynamic regions by a motion detection algorithm and then processed separately. The effect of the hybrid de-interlacing algorithm mainly depends on motion detection, and if a more accurate motion detection result is obtained, multi-field data needs to be referred to, so that the requirement on hardware equipment is higher. The space-time domain interpolation-based inter-field de-interlacing algorithm is moderate in calculation complexity and widely applied due to small hardware overhead.
In the spatial domain interpolation method, an interpolation algorithm based on a boundary cannot judge whether a point to be interpolated is also positioned at an image boundary, which may cause the image boundary to be fuzzy and lose detail information at a rich texture position. In the time domain interpolation method, field averaging, median filtering and the like are suitable for static scenes, but a sawtooth phenomenon can be generated in a motion area, most of the existing de-interlacing algorithms are based on theoretical research, the used test sequences are open source standard test sequences, and the problem of frame loss possibly existing in data transmission in practical application is not considered. Frame loss may cause a large difference between pixels of two adjacent fields of data, and a screen blooming phenomenon may occur by using a time domain interpolation method.
Therefore, how to optimize the de-interleaving algorithm to effectively combine the space-time domain information and improve the performance of the de-interleaving algorithm needs to be further researched.
Disclosure of Invention
The invention solves the technical problem of how to improve the interpolation accuracy rate so as to improve the resolution of the image data after interpolation and improve the performance of a de-interlacing algorithm.
To solve the foregoing technical problem, an embodiment of the present invention provides an image data interpolation method, including: carrying out interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, wherein each field comprises a plurality of points to be interpolated and a plurality of pixel points; calculating a prediction pixel value of a current point to be interpolated in each field by adopting a time domain interpolation method; determining a pixel value of a first pixel point adjacent to the current point to be interpolated, wherein the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field; at least carrying out difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point to obtain a difference detection result; and when the difference detection result exceeds a first preset threshold value, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method.
Optionally, the image data interpolation method further includes: when the difference detection result does not exceed the first preset threshold, carrying out boundary detection on the current point to be interpolated; if the current point to be interpolated is a boundary point, performing median filtering based on a predicted pixel value of the current point to be interpolated and a pixel value of a second pixel point to determine the pixel value of the current point to be interpolated; otherwise, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method, wherein the second pixel point and the current point to be interpolated are positioned in the same field, and the distance between any pixel point in the second pixel point and the current point to be interpolated is smaller than a first preset distance.
Optionally, the total number of the second pixel points is 2, and performing median filtering based on the predicted pixel value of the current to-be-interpolated point and the pixel value of the second pixel point to determine the pixel value of the current to-be-interpolated point includes: performing median filtering by adopting the following formula to determine the pixel value of the current point to be interpolated: p(x,y,t)＝mid(Ptemp(x,y,t),F0,F1) (ii) a Wherein, P(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y,t)Representing the predicted pixel value of the current point to be interpolated, x representing the coordinate in the horizontal direction, y representing the coordinate in the vertical direction, t representing the coordinate in the time domain field, F0And F1A pixel value representing said second pixel point, mid (P)temp(x,y,t),F0,F1) Represents a pair F0、F1And Ptemp(x,y,t)And performing median filtering.
Optionally, the performing boundary detection on the current point to be interpolated includes: determining a plurality of third pixel points, wherein each third pixel point and the current to-be-interpolated point are located in the same field, the distance between each third pixel point and the current to-be-interpolated point is smaller than a second preset distance, and the second preset distance is larger than the first preset distance; calculating the absolute value of the pixel difference between each third pixel point and the current point to be interpolated; and when the maximum value of all the calculated absolute values of the pixel difference values is smaller than a second preset threshold value, determining the current point to be interpolated as a non-boundary point, otherwise, determining the current point to be interpolated as the boundary point.
Optionally, the following formula is adopted to calculate each pixel difference absolute value between any one of the third pixel points and the current to-be-interpolated point: diff ═ Ptemp(x,y,t)-F(x+i,y+j,t)|,i＝-2,-1,0,1,2；j＝-1,1；t>0; wherein diff represents the absolute value of the pixel difference, x represents the coordinate in the horizontal direction, y represents the coordinate in the vertical direction, t represents the coordinate in the time domain field, Ptemp(x,y,t)A predicted pixel value, F, representing said current point to be interpolated(x+i,y+j,t)And representing the pixel value of any one of the third pixel points.
Optionally, the image data interpolation method further includes: for each field of the input image data, determining pixel values of other points to be interpolated in each field except the current point to be interpolated and pixel values of each pixel point to obtain field data of each field; based on the field data of each field, output image data is obtained.
Optionally, the obtaining, by computing the current to-be-interpolated point in each field by using a time domain interpolation method, a predicted pixel value of the current to-be-interpolated point includes: for each field, dividing the field into a plurality of strips, and performing data interpolation on points to be interpolated in the plurality of strips, wherein each strip comprises partial points to be interpolated and partial pixel points of the field; and calculating the current to-be-interpolated point in each strip by adopting a time domain interpolation method to obtain the predicted pixel value of the current to-be-interpolated point.
Optionally, after the prediction pixel value of the current point to be interpolated is obtained by computing using a time-domain interpolation method, the data interpolation method further includes: determining a plurality of other points to be interpolated adjacent to the current point to be interpolated, wherein the plurality of other points to be interpolated and the current point to be interpolated are positioned in the same field, the horizontal direction coordinates of the plurality of other points to be interpolated are the same as the horizontal direction coordinates of the current point to be interpolated, and the distance between any one of the plurality of other points to be interpolated and the current point to be interpolated is less than a third preset distance; and calculating by adopting a time domain interpolation method to obtain respective predicted pixel values of the other points to be interpolated.
Optionally, the performing, at least, difference detection on the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point includes: and performing difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point, and performing difference detection on the predicted pixel values of the other points to be interpolated and the pixel value of the first pixel point.
Optionally, the number of the other to-be-interpolated points is 2, the fields arranged according to the time sequence include multiple odd fields and even fields, the current to-be-interpolated point and the other to-be-interpolated points are located in the even fields, and the predicted pixel values of the current to-be-interpolated point and the other to-be-interpolated points are determined by using the following formulas:
Figure BDA0001993675590000041
wherein, Ptemp(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y-2,t)、Ptemp(x,y+2,t)Respectively representing the predicted pixel values of the other points to be interpolated, x representing a horizontal coordinate, y representing a vertical coordinate, t representing a time domain field coordinate, F(x,y+j,t+1)J ═ 2, 0, 2 each denote a pixel value of a pixel point located in the odd field (t +1), F(x,y+j,t-1)And j ═ 2, 0, 2 each denote a pixel value of a pixel point located in the odd field (t-1).
Optionally, the performing difference detection on the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point, and performing difference detection on the predicted pixel values of the other to-be-interpolated points and the pixel value of the first pixel point includes: calculating a pixel difference absolute value between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point; and calculating the absolute value of the pixel difference between the predicted pixel value of any one point in the other points to be interpolated and the pixel value of the first pixel point.
Optionally, the number of the first pixel points is 2, and the absolute value of the pixel difference between the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point, and the absolute value of the pixel difference between the predicted pixel values of the other to-be-interpolated points and the first pixel point are calculated by using the following formulas:
Figure BDA0001993675590000051
wherein, diff0、diff1、diff2、diff3Respectively representing the absolute value of the pixel difference, F(x,y+j,t)And j is-1, and 1 represents the pixel value of the first pixel point.
To solve the above technical problem, an embodiment of the present invention further provides an image data interpolation apparatus, including: the scanning module is suitable for carrying out interlaced scanning on input image data to obtain a plurality of fields which are arranged according to a time sequence, and each field comprises a plurality of points to be interpolated and a plurality of pixel points; the calculation module is suitable for calculating the current to-be-interpolated point of each field by adopting a time domain interpolation method to obtain a predicted pixel value of the current to-be-interpolated point; the first determining module is suitable for determining a pixel value of a first pixel point adjacent to the current point to be interpolated, the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are positioned in the same field; the difference detection module is suitable for at least carrying out difference detection on the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point to obtain a difference detection result; and the second determining module is suitable for determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method when the difference detection result exceeds a first preset threshold value.
To solve the above technical problem, an embodiment of the present invention further provides a storage medium having stored thereon computer instructions, where the computer instructions execute the steps of the above method when executed.
In order to solve the foregoing technical problem, an embodiment of the present invention further provides a terminal, including a memory and a processor, where the memory stores computer instructions executable on the processor, and the processor executes the computer instructions to perform the steps of the foregoing method.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides an image data interpolation method, which comprises the following steps: carrying out interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, wherein each field comprises a plurality of points to be interpolated and a plurality of pixel points; calculating the current to-be-interpolated point of each field by adopting a time domain interpolation method to obtain a predicted pixel value of the current to-be-interpolated point; determining a pixel value of a first pixel point adjacent to the current point to be interpolated, wherein the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field; at least carrying out difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point to obtain a difference detection result; and when the difference detection result exceeds a first preset threshold value, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method. The embodiment of the invention can determine the reliability of the time domain reference information by detecting the difference between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point, and detect the frame loss condition possibly existing in the image data transmission process in practical application. When the time domain information is unreliable (for example, frame loss or violent motion exists), a spatial domain interpolation method can be adopted to calculate the point to be interpolated (namely, the missing point) in each field, so that the interpolation accuracy can be improved, the resolution of the image data after interpolation can be improved, and the performance of the de-interlacing algorithm can be improved.
Further, the image data interpolation method further includes: when the difference detection result does not exceed the first preset threshold, carrying out boundary detection on the current point to be interpolated; if the current point to be interpolated is a boundary point, performing median filtering based on a predicted pixel value of the current point to be interpolated and a pixel value of a second pixel point to determine the pixel value of the current point to be interpolated; otherwise, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method, wherein the second pixel point and the current point to be interpolated are positioned in the same field, and the distance between any pixel point in the second pixel point and the current point to be interpolated is smaller than a first preset distance. By the technical scheme provided by the embodiment of the invention, whether the current point to be interpolated belongs to the boundary point can be judged by using the predicted value obtained by the time domain interpolation method and the pixel value of the known pixel point under the condition that the time domain reference information is reliable. If not, adopting spatial interpolation; and if the boundary point belongs to the boundary point, comprehensively utilizing the space-time domain information for processing. The embodiment of the invention fully utilizes the space-time domain correlation, can further reduce the possible fuzzy phenomenon of the space-time interpolation at the image boundary, and improves the robustness of the algorithm in the real-time processing process, thereby effectively retaining the detail information and further improving the video quality.
Further, before the time domain interpolation method is adopted to calculate and obtain the predicted pixel value of the current point to be interpolated, the method further comprises the following steps: for each field, dividing the field into a plurality of strips, and performing data interpolation on points to be interpolated in the plurality of strips, wherein each strip comprises partial points to be interpolated and partial pixel points of the field. By dividing each field into a plurality of strips, the plurality of strips can be processed in parallel, time consumption is reduced, and therefore the de-interlacing processing performance can be further improved, the processing speed is effectively improved, and the method is applicable to real-time processing.
Drawings
Fig. 1 is a schematic diagram of an interlaced data stream obtained based on interlaced scanning in a prior art scheme;
FIG. 2 is a schematic diagram of a spatial interpolation method based on boundary direction in a prior art solution;
FIG. 3 is a schematic diagram of an embodiment of the prior art;
FIG. 4 is a flow chart illustrating a method for interpolating image data according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a principle of calculating a point to be interpolated based on a time domain interpolation method according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a pixel point for boundary detection according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating a point to be interpolated being located at a boundary according to an embodiment of the present invention;
FIG. 8 is a flow chart illustrating a further method for interpolating image data according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an image data interpolation apparatus according to an embodiment of the present invention.
Detailed Description
As mentioned in the background, the de-interlacing algorithm in the prior art has drawbacks and needs to be further improved.
To reduce bandwidth, current television systems typically employ interlaced scanning techniques. The data generated by each scan is a field, which is called an odd field (also called a top field) or an even field (also called a bottom field), respectively. Fig. 1 shows a data stream obtained by an interlaced scanning technique in a prior art solution. As shown in fig. 1, a black solid dot 101 represents a known pixel obtained by interlaced scanning, and an empty dot 102 represents a missing dot, which may also be referred to as a to-be-interpolated dot. Where the odd field 103 refers to a field containing only odd lines of known pixels and the even field 104 refers to a field containing only even lines.
Fig. 2 is a schematic diagram of a spatial domain interpolation method based on a boundary direction in a prior art scheme. Referring to FIG. 2, P(x,y)Indicates a point of absence, F(x+i,y+j)I-3, -2, -1,0,1,2,3, j-1, 1 indicates a known pixel. The space domain interpolation method is represented by P(x,y)As a center, each directional gradient diff is calculated separatelyiAnd gradient of direction diffiThe direction in which the value of (d) is the smallest is taken as the most likely boundary direction (diff)iThe value of i) in the minimum time) and the known pixel point in the boundary direction is taken as a reference for interpolation, the formula is as follows:
Figure BDA0001993675590000071
P(x,y)＝Pi(x,y),diffi＝min(diff0,diff±1,diff±2,...,diff±n)；
wherein x represents a horizontal coordinate, y represents a vertical coordinate, n is a positive integer, min (diff)0,diff±1,diff±2,...,diff±n) Representing the solution set { diff0,diff±1,diff±2,...,diff±nThe minimum value of (c).
The time domain interpolation method is a method for performing data interpolation by using time domain correlation. The simplest method is to combine the data in adjacent odd and even fields directly and output the combined data, which is called field combining.
Fig. 3 is a schematic diagram of an application of the prior art solution. Where the odd field 301 and the even field 302 are combined resulting in a combined field 303. In addition, median filtering is also a common time-domain interpolation method. The time domain interpolation method has a good effect in a static area, and can generate an obvious sawtooth phenomenon in a moving area.
The boundary-based airspace interpolation method takes pixel points as processing units, accords with the visual characteristics of human eyes, but cannot judge whether a point to be interpolated is positioned on a boundary. If the point to be interpolated is exactly at the boundary, the spatial domain interpolation method can cause the boundary to be fuzzy, and the detail information is lost. The time domain interpolation method has a good effect on a static area, but obvious saw teeth can be generated in an area with violent movement, and if a frame loss phenomenon occurs in practical application, the time domain interpolation method can cause image screen blooming and seriously affect the visual effect.
The embodiment of the invention provides an image data interpolation method, which comprises the following steps: carrying out interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, wherein each field comprises a plurality of points to be interpolated and a plurality of pixel points; calculating the current to-be-interpolated point of each field by adopting a time domain interpolation method to obtain a predicted pixel value of the current to-be-interpolated point; determining a pixel value of a first pixel point adjacent to the current point to be interpolated, wherein the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field; at least carrying out difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point to obtain a difference detection result; and when the difference detection result exceeds a first preset threshold value, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method.
The embodiment of the invention can determine the reliability of the time domain reference information by detecting the difference between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point, and detect the frame loss condition possibly existing in the image data transmission process in practical application. When the time domain information is unreliable (for example, frame loss or violent motion exists), a spatial domain interpolation method can be adopted to calculate the point to be interpolated (namely, the missing point) in each field, so that the interpolation accuracy can be improved, the resolution of the image data after interpolation can be improved, and the performance of the de-interlacing algorithm can be improved.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
Fig. 4 is a flowchart illustrating an image data interpolation method according to an embodiment of the present invention. The image data interpolation method may be performed by a terminal device (e.g., a television device). Specifically, the image data interpolation method may include the steps of:
step S101, carrying out interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, wherein each field comprises a plurality of points to be interpolated and a plurality of pixel points;
step S102, calculating a current to-be-interpolated point of each field by adopting a time domain interpolation method to obtain a predicted pixel value of the current to-be-interpolated point;
step S103, determining a pixel value of a first pixel point adjacent to the current point to be interpolated, wherein the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field;
step S104, at least carrying out difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point to obtain a difference detection result;
and S105, when the difference detection result exceeds a first preset threshold value, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method.
More specifically, in step S101, input image data may be interlaced to obtain a plurality of image data fields (hereinafter, simply referred to as fields) arranged in time series. The fields include odd and even fields. The odd-numbered lines of pixels in the odd field are known pixels, the pixel values of which are known, and the even-numbered lines of pixels are to-be-interpolated points (which can also be called missing points). And the odd-numbered line pixel points in the even field are to-be-interpolated points, the even-numbered line pixel points are known pixel points, and the pixel values of the even-numbered line pixel points are known.
In a specific implementation, in order to increase the interpolation speed of the point to be interpolated, each odd field and/or even field) may be divided into a plurality of stripes (slices), and the plurality of stripes are processed in parallel to perform data interpolation on the point to be interpolated in each stripe. Each stripe may comprise part of the field to be interpolated and part of the pixel points.
In step S102, for the current to-be-interpolated point in each field or each stripe, a time domain interpolation method may be used to calculate a predicted pixel value of the current to-be-interpolated point. The simplest time domain interpolation method can directly copy the pixel value of the corresponding known point in the field before the current point to be interpolated, so as to obtain the predicted pixel value of the current point to be interpolated.
In a specific implementation, the average value of pixels of corresponding known pixels in a field before and a field after the current point to be interpolated is also used as the predicted pixel value of the current point to be interpolated.
In step S103, a pixel value of a first pixel point adjacent to the current to-be-interpolated point may be determined. The horizontal direction coordinate of the first pixel point is the same as the horizontal direction coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field.
In a specific implementation, the first pixel point is a known pixel point in the odd field or the even field, and a pixel value thereof is also known.
Furthermore, a plurality of other points to be interpolated adjacent to the current point to be interpolated can also be determined. And the other points to be interpolated and the current point to be interpolated are positioned in the same field. The plurality of other points to be interpolated refer to other points to be interpolated, which have the same horizontal direction coordinates as the current point to be interpolated and have a distance from any point of the plurality of other points to be interpolated to the current point to be interpolated, which is smaller than a third preset distance. And the third preset distance is used for defining a range for selecting adjacent points to be interpolated from all other points to be interpolated in the field where the current point to be interpolated is located.
Then, the respective predicted pixel values of the other points to be interpolated can be calculated by adopting a time domain interpolation method.
Fig. 5 is a schematic diagram illustrating a principle of calculating a point to be interpolated based on a time domain interpolation method according to an embodiment of the present invention. Referring to fig. 5, black solid dots represent known pixel points obtained by scanning, hollow dots represent points to be interpolated, and vertical line dots represent the current points to be interpolated.
With continued reference to FIG. 5, the coordinates of the current point to be interpolated are (x, y, t), and are located in the even field F2, where the pixel value is represented by P(x,y,t)It is shown that the adjacent preceding field is an odd field F1 and the adjacent succeeding field is an odd field F3. The coordinates of two other points to be interpolated which are adjacent to the current point to be interpolated and have the same horizontal coordinate are (x, y +2, t), (x, y-2, t), and the pixel value is P(x,y+2,t)、P(x,y-2,t)And (4) showing.
It can be further seen that the pixel value of the known pixel point located in the odd field F1 and having the coordinate of (x, y, t-1) is F(x,y,t-1)(ii) a The pixel value of the known pixel point with the coordinate of (x, y-2, t-1) is F(x,y-2,t-1)(ii) a The pixel value of the known pixel point with the coordinate of (x, y +2, t-1) is F(x,y+2,t-1). The pixel value of a known pixel point located in the odd field F3 and having coordinates of (x, y, t +1) is F(x,y,t+1)(ii) a The pixel value of the known pixel point with the coordinate of (x, y-2, t +1) is F(x,y-2,t+1)(ii) a The pixel value of the known pixel point with the coordinate of (x, y +2, t +1) is F(x,y+2,t+1)。
For the magnetic field at even field,The current point to be interpolated with coordinates (x, y, t) may be a known pixel point with coordinates (x, y-1, t) and coordinates (x, y +1, t) as the first pixel point, and it is assumed that the pixel values are F respectively(x,y-1,t)、F(x,y+1,t)。
Further, the current point to be interpolated P can be calculated(x,y,t)And other points P to be interpolated(x,y+2,t)、P(x,y-2,t)Are respectively expressed as Ptemp(x,y,t)、Ptemp(x,y+2,t)、Ptemp(x,y-2,t). In specific implementation, a time domain interpolation method can be adopted to calculate and obtain the predicted pixel value of each to-be-interpolated point, and the formula is as follows:
Figure BDA0001993675590000111
wherein, Ptemp(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y-2,t)、Ptemp(x,y+2,t)Respectively representing the predicted pixel values of 2 other points to be interpolated, x representing the coordinate in the horizontal direction, y representing the coordinate in the vertical direction, t representing the coordinate in the time domain field, F(x,y+j,t+1)J-2, 0, 2 each represents a pixel value of a pixel point located in the odd field F3, F(x,y+j,t-1)And j ═ 2, 0, 2 each denote the pixel values of the pixel points located in the odd field F1.
In step S104, difference detection may be performed on the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point, and difference detection may be performed on the predicted pixel values of a plurality of other to-be-interpolated points adjacent to the current to-be-interpolated point and the pixel value of the first pixel point, so as to obtain a difference detection result.
In a specific implementation, a pixel difference absolute value between the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point may be calculated. Then, the absolute value of the pixel difference between the predicted pixel value of any one of the other points to be interpolated and the pixel value of the first pixel point may be calculated.
With continued reference to fig. 5, the number of the first pixel points is 2, and the coordinates are (x, y +1, t) and (x, y-1, t), respectively. The absolute value of the difference of each pixel can be calculated using the following formula:
Figure BDA0001993675590000112
wherein, diff0、diff1、diff2、diff3Respectively representing the absolute value of the pixel difference, F(x,y+j,t)And j is-1, and 1 represents the pixel value of the first pixel point.
In step S105, if the minimum value of the absolute values of the pixel differences is greater than the first preset threshold, the pixel value of the current point to be interpolated may be determined by spatial interpolation.
In a specific implementation, if the minimum value of the absolute values of the pixel differences is not greater than the first preset threshold, the boundary detection may be performed on the current point to be interpolated.
In a specific implementation, whether the current to-be-interpolated point is a boundary point may be determined as follows. Firstly, a plurality of third pixel points are determined in the field of the current point to be interpolated. The distance between each third pixel point and the current to-be-interpolated point is less than a second preset distance, and the second preset distance may be greater than the first preset distance.
And secondly, calculating the absolute value of the pixel difference between the pixel value of each third pixel point and the predicted pixel value of the current point to be interpolated. In a specific implementation, the predicted pixel value of the current point to be interpolated may be calculated based on a temporal interpolation method.
Specifically, the absolute value of the difference between any one of the third pixel points and the current to-be-interpolated point may be calculated by using the following formula:
diff＝|Ptemp(x,y,t)-F(x+i,y+j,t)|,i＝-2,-1,0,1,2；j＝-1,1；t>0；
wherein diff represents the absolute value of the pixel difference, x represents the coordinate in the horizontal direction, y represents the coordinate in the vertical direction, t represents the coordinate in the time domain field, Ptemp(x,y,t)A predicted pixel value, F, representing said current point to be interpolated(x+i,y+j,t)And representing the pixel value of any one of the third pixel points.
Thirdly, when the maximum value of the absolute values of all the calculated pixel differences is smaller than a second preset threshold, the current point to be interpolated may be determined as a non-boundary point, otherwise, the current point to be interpolated may be determined as the boundary point.
Further, if the current to-be-interpolated point is a boundary point, median filtering may be performed based on the predicted pixel value of the current to-be-interpolated point and the pixel value of the second pixel point to determine the pixel value of the current to-be-interpolated point. Otherwise, if the current point to be interpolated is not the boundary point, the pixel value of the current point to be interpolated can be determined by adopting a spatial domain interpolation method. The second pixel point and the current point to be interpolated are located in the same field, and the distance between any pixel point in the second pixel point and the current point to be interpolated is smaller than a first preset distance.
In a specific implementation, the second pixel point may be selected from the third pixels, and a plurality of known pixel points closest to the current point to be interpolated are selected.
In a specific implementation, the number of the second pixel points may be multiple, for example, 2. At this time, the pixel value of the current to-be-interpolated point may be determined by performing median filtering on the predicted pixel value of the current to-be-interpolated point and the pixel value of the second pixel point.
Specifically, the following formula can be used for median filtering:
P(x,y,t)＝mid(Ptemp(x,y,t),F0,F1)；
wherein, P(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y,t)Representing the predicted pixel value of the current point to be interpolated, x representing the coordinate in the horizontal direction, y representing the coordinate in the vertical direction, t representing the coordinate in the time domain field, F0And F1A pixel value representing said second pixel point, mid (P)temp(x,y,t),F0,F1) Represents a pair F0、F1And Ptemp(x,y,t)And performing median filtering.
Fig. 6 is a schematic diagram of a pixel point for boundary detection according to an embodiment of the present invention. Referring to fig. 6, solid dots represent the plurality of third pixel points. Assuming that the predicted pixel value of the current point to be interpolated is Ptemp(x,y,t)The coordinates and pixel values of each third pixel are known. In fig. 6, the number of the third pixels is 10, and it is assumed that the pixel value of each third pixel is F(x-2,y-1,t)、F(x-1,y-1,t)(not shown), F(x,y-1,t)、F(x+1,y-1,t)(not shown), F(x+2,y-1,t)、F(x-2,y+1,t)、F(x-1,y+1,t)(not shown), F(x,y+1,t)、F(x+1,y+1,t)(not shown), F(x+2,y+1,t)。
Fig. 7 is a schematic diagram of a point to be interpolated being located at a boundary according to an embodiment of the present invention. The vertical line dots represent the to-be-interpolated points, the black solid dots and the hollow dots represent pixel points positioned at two sides of the boundary, and the black solid lines represent the boundary. Referring to fig. 6, if the current point to be interpolated is marked as a vertical dot and is located at the image boundary, the pixel values of the second pixel points are respectively F(x,y-1,t)、F(x,y+1,t)Then, the result of performing median filtering based on the predicted pixel value of the current point to be interpolated and the pixel value of the second pixel point is P(x,y,t)＝mid(F(x,y-1,t),F(x,y+1,t),Ptemp(x,y,t))。
Further, for each field of the input image data, after the pixel values of each to-be-interpolated point in each field are obtained through calculation, the pixel values of each pixel point may be combined together to obtain field data of each field. De-interlacing image data can be obtained based on the field data of each field.
The following describes the data interpolation of the missing points in the even field data as a specific embodiment, with the odd field data as the known information.
Fig. 8 is a flowchart illustrating a further image data interpolation method according to an embodiment of the present invention. Referring to fig. 8, first, operation S801 is performed, i.e., data is input, which may include data information of an even field to be interpolated and its adjacent two odd fields.
Second, operation S802 may be performed to determine whether there is a large difference in pixel values. Specifically, a time domain interpolation method is adopted to obtain a predicted pixel value of the current point to be interpolated, and determine whether there is a difference between the predicted pixel value and an adjacent known pixel point where the predicted pixel value is located? If the difference is large, frame loss or violent motion is possible, time domain information is unreliable, and the adoption of a time domain interpolation method can cause large difference of odd-even data of the image and even screen blooming. To determine the pixel value of the current point to be interpolated,
further, when the difference is large, operation S803 is performed, and spatial interpolation is performed, that is, the pixel value of the current point to be interpolated is calculated by using a spatial interpolation method. Otherwise, after the situation with a large difference is eliminated, indicating that the time domain information is reliable information, operation S804 may be performed to determine whether the to-be-interpolated point is a boundary point, that is, determine whether the current to-be-interpolated point is located at an image boundary? If not, operation S803 may be performed to still perform data interpolation using spatial interpolation. Otherwise, if the image is located at the image boundary, operation S805 is performed to perform boundary point processing.
More specifically, the specific steps of determining whether there is a difference in pixel values are as follows:
based on the time-space domain correlation, the frame loss situation and the severe motion area which may occur in practical application can be detected. With continued reference to FIG. 5, x is the horizontal coordinate, y is the vertical coordinate, and t represents the time domain field coordinate. Taking the point to be interpolated with the coordinates (x, y, t) as the current point to be interpolated, and calculating by using mean filtering to obtain the initial predicted value P of the point to be interpolated with the coordinates (x, y, t), (x, y-2, t), (x, y +2, t)temp(x,y,t)、Ptemp(x,y-2,t)、Ptemp(x,y+2,t)。
Figure BDA0001993675590000141
Wherein, Ptemp(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y-2,t)、Ptemp(x,y+2,t)Respectively representing the predicted pixel values of the other points to be interpolated.
Then, the following formula can be adopted to respectively calculate the absolute value diff of the pixel difference between each point to be interpolated and the adjacent first pixel point0、diff1、diff2、diff3. Wherein, diff0、diff1、diff2、diff3Respectively representing the absolute value of the pixel difference, F(x,y+j,t)And j is-1, and 1 represents the pixel value of the first pixel point.
Figure BDA0001993675590000151
Further, determining the first preset threshold TH1The difference detection result may then be determined using the following formula:
Figure BDA0001993675590000152
wherein flag1 indicates the difference detection result, min (diff)0,diff1,diff2,diff3) Represents a pair diff0、diff1、diff2、diff3Taking the minimum value. When the pixel difference absolute value diff0、diff1、diff2、diff3Are all greater than the first preset threshold value TH1Time domain information is considered unreliable, and at this time, flag1True (true), a spatial interpolation algorithm may be directly employed. Otherwise, flag1False (false). Wherein the first preset threshold TH1The value of (b) can be determined experimentally.
When flag1And if false, the time domain information is reliable. In this case, the predicted pixel value P obtained by the time domain interpolation method may be usedtemp(x,y,t)And performing boundary detection as a reference value, and judging whether the current point to be interpolated is positioned at the image boundary.
In a specific implementation, the following formula can be used to calculate the predicted pixel value and each known pixel point (e.g., F) of the current to-be-interpolated point(x+i,y+j,t)I ═ 2,.., 0,. 2; pixel difference absolute value between pixel values of j ═ 1, 1): diff ═ Ptemp(x,y,t)-F(x+i,y+j,t)|,i＝-2,...,0,...2；j＝-1,1。
Through analysis, if the point to be interpolated is located at the image boundary, the predicted pixel value P of the point to be interpolated must existtemp(x,y,t)And the pixel value F of the known point(x+i,y+j,t)The absolute value diff of the difference between them is greater than a certain threshold (e.g. the second preset threshold). If the point to be interpolated is not at the image boundary, the point to be interpolated P existstemp(x,y,t)And known point F(x+i,y+j,t)The absolute value diff of the difference between them is less than the second preset threshold. At this time, a spatial interpolation method may be adopted to calculate the pixel value of the point to be interpolated.
For example, the second preset threshold is TH2At this time, the following formula may be adopted to determine the boundary detection result:
Figure BDA0001993675590000161
wherein, flag2Indicates the boundary detection result, flag2If the value is true, the point to be interpolated is positioned at the image boundary, otherwise, the point to be interpolated is not positioned at the image boundary.
As known to those skilled in the art, for a point to be interpolated at a boundary, its pixel value is usually the same as the pixel value F of a known pixel point in the same region(x+i,y+j,t)Similarly, the absolute value diff of the corresponding pixel difference is small.
In order to improve the precision, the spatial domain and the time domain correlation are comprehensively considered, and 2 known pixel points with the minimum pixel difference absolute value can be selected from the known pixel points in the field of the current point to be interpolated. Suppose that the pixel values of the 2 known pixels are respectively F0、F1The predicted pixel value of the current point to be interpolated obtained by the time domain interpolation method is Ptemp(x,y,t)In pair F0、F1、Ptemp(x,y,t)And after median filtering, obtaining the pixel value of the current point to be interpolated.
Under the condition, the calculation formula of the current to-be-interpolated point is represented as:
Figure BDA0001993675590000162
wherein, Pspatial(x,y,t)The pixel values calculated by the spatial interpolation method are shown.
In order to save processing time, the input field data of each field can be divided into a plurality of stripes (slices) to be processed synchronously. Specifically, input image data is first divided into a plurality of bands, and then deinterlace is performed for each condition. Because the reference information of the upper and lower boundaries of each band may have a boundary crossing problem when multi-band processing is adopted (for example, if the upper boundary only refers to the lower information and the lower boundary only refers to the upper information, a more obvious "fault" phenomenon occurs in the actual processing process), the upper and lower boundaries of each band can be de-interleaved again after the processing of all bands is completed.
Therefore, by the technical scheme provided by the embodiment of the invention, the frame loss condition possibly existing in the data transmission process in practical application can be detected, the algorithm fault tolerance is improved, the space-time domain correlation is fully utilized, the detection and processing of the image boundary are realized, and the detail information can be effectively reserved. In addition, the image can be processed in a multi-strip mode in parallel, the processing speed can be effectively improved, and the method is suitable for real-time processing application.
Fig. 9 is a schematic structural diagram of an image data interpolation apparatus according to an embodiment of the present invention. The image data interpolation device 2 may be implemented by a terminal, for example, a television, a camera, or other devices, and executes the method solutions shown in fig. 4 to 8.
Specifically, the image data interpolation apparatus 2 may include: a scanning module 201, adapted to perform interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, where each field includes a plurality of to-be-interpolated points and a plurality of pixel points; the calculating module 202 is adapted to calculate, by using a time domain interpolation method, a predicted pixel value of a current point to be interpolated for the current point to be interpolated in each field; a first determining module 203, adapted to determine a pixel value of a first pixel point adjacent to the current point to be interpolated, where a horizontal coordinate of the first pixel point is the same as a horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field; a difference detection module 204, adapted to perform difference detection on at least the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point to obtain a difference detection result; the second determining module 205 is adapted to determine the pixel value of the current to-be-interpolated point by using a spatial interpolation method when the difference detection result exceeds a first preset threshold.
In a specific implementation, the image data interpolation apparatus 2 may further include: a boundary detection module 206, adapted to perform boundary detection on the current point to be interpolated when the difference detection result does not exceed the first preset threshold; a filtering module 207, wherein if the current to-be-interpolated point is a boundary point, the filtering module 207 is adapted to perform median filtering based on a predicted pixel value of the current to-be-interpolated point and a pixel value of a second pixel point to determine the pixel value of the current to-be-interpolated point; otherwise, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method, wherein the second pixel point and the current point to be interpolated are positioned in the same field, and the distance between any pixel point in the second pixel point and the current point to be interpolated is smaller than a first preset distance.
The total number of the second pixel points is 2, and the filtering module 207 includes: a filtering submodule 2071, adapted to perform median filtering by using the following formula to determine the pixel value of the current to-be-interpolated point:
P(x,y,t)＝mid(Ptemp(x,y,t),F0,F1)；
wherein, P(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y,t)To representThe predicted pixel value of the current point to be interpolated, x represents the coordinate in the horizontal direction, y represents the coordinate in the vertical direction, t represents the coordinate in the time domain field, F0And F1A pixel value representing said second pixel point, mid (P)temp(x,y,t),F0,F1) Represents a pair F0、F1And Ptemp(x,y,t)And performing median filtering.
In a specific implementation, the boundary detection module 206 may include: a first determination submodule 2061, a first calculation submodule 2062, and a second determination submodule 2063.
Specifically, the first determining submodule 2061 is adapted to determine a plurality of third pixel points, each third pixel point is located in the same field as the current to-be-interpolated point, a distance between each third pixel point and the current to-be-interpolated point is smaller than a second preset distance, and the second preset distance is greater than the first preset distance; the first calculating submodule 2062 is adapted to calculate the absolute value of the pixel difference between each third pixel point and the current point to be interpolated; the second determining submodule 2063 is adapted to determine the current point to be interpolated as a non-boundary point when the maximum value of the calculated absolute values of the pixel differences is smaller than a second preset threshold, and otherwise, determine the current point to be interpolated as the boundary point.
In a specific implementation, the first calculating submodule 2062 is adapted to calculate each pixel difference absolute value between any pixel point of the third pixel points and the current to-be-interpolated point by using the following formula:
diff＝|Ptemp(x,y,t)-F(x+i,y+j,t)|,i＝-2,-1,0,1,2；j＝-1,1；t>0；
wherein diff represents the absolute value of the pixel difference, x represents the coordinate in the horizontal direction, y represents the coordinate in the vertical direction, t represents the coordinate in the time domain field, Ptemp(x,y,t)A predicted pixel value, F, representing said current point to be interpolated(x+i,y+j,t)And representing the pixel value of any one of the third pixel points.
In a specific implementation, the image data interpolation apparatus 2 may further include: a third determining module 208, adapted to determine, for each field of the input image data, pixel values of other to-be-interpolated points in each field except the current to-be-interpolated point and pixel values of respective pixel points, so as to obtain field data of each field; an output module 209 is adapted to derive output image data based on the field data of each field.
In a specific implementation, the calculation module 202 may include: the dividing submodule 2021 is adapted to divide each field into a plurality of strips, and perform data interpolation on to-be-interpolated points in the plurality of strips together, where each strip includes a part of to-be-interpolated points and a part of pixel points of the field; the second calculating sub-module 2022 is adapted to calculate, by using a time domain interpolation method, a predicted pixel value of the current point to be interpolated for the current point to be interpolated in each band.
In a specific implementation, the data interpolation apparatus 2 may further include: a fourth determining module 210, adapted to determine, after obtaining a predicted pixel value of the current point to be interpolated by using a time-domain interpolation method, a plurality of other points to be interpolated adjacent to the current point to be interpolated, where the plurality of other points to be interpolated and the current point to be interpolated are located in the same field, a horizontal coordinate of the plurality of other points to be interpolated is the same as a horizontal coordinate of the current point to be interpolated, and a distance between any one of the plurality of other points to be interpolated and the current point to be interpolated is smaller than a third preset distance; the fifth determining module 211 is adapted to calculate and obtain the predicted pixel values of the other points to be interpolated by using a time domain interpolation method.
In a specific implementation, the difference detection module 204 may include: the detection submodule 2041 is adapted to perform difference detection on the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point, and perform difference detection on the predicted pixel values of the other to-be-interpolated points and the pixel value of the first pixel point.
In a specific implementation, the number of the other points to be interpolated is 2, the fields arranged according to the time sequence include multiple odd fields and even fields, the current point to be interpolated and the other points to be interpolated are located in the even fields, and the predicted pixel values of the current point to be interpolated and the other points to be interpolated can be determined by using the following formulas:
Figure BDA0001993675590000191
wherein, Ptemp(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y-2,t)、Ptemp(x,y+2,t)Respectively representing the predicted pixel values of the other points to be interpolated, x representing a horizontal coordinate, y representing a vertical coordinate, t representing a time domain field coordinate, F(x,y+j,t+1)J ═ 2, 0, 2 each denote a pixel value of a pixel point located in the odd field (t +1), F(x,y+j,t-1)And j ═ 2, 0, 2 each denote a pixel value of a pixel point located in the odd field (t-1).
In a specific implementation, the detection submodule 2041 is adapted to calculate a pixel difference absolute value between a predicted pixel value of the current to-be-interpolated point and a pixel value of the first pixel point; and the method is also suitable for calculating the pixel difference absolute value between the predicted pixel value of any one of the other points to be interpolated and the pixel value of the first pixel point.
In a specific implementation, the number of the first pixel points is 2, and the detection submodule 2041 is adapted to calculate the absolute value of the pixel difference between the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point, and the absolute value of the pixel difference between the predicted pixel values of the other to-be-interpolated points and the first pixel point by using the following formula, where diff is0、diff1、diff2、diff3Respectively representing the absolute value of the pixel difference, F(x,y+j,t)J ═ 1,1 denotes the pixel value of the first pixel:
Figure BDA0001993675590000201
for more details of the operation principle and the operation mode of the image data interpolation device 2 shown in fig. 9, reference may be made to the description in fig. 4 to 8, and details thereof are not repeated here.
Further, the embodiment of the present invention further discloses a storage medium, on which computer instructions are stored, and when the computer instructions are executed, the technical solutions of the methods in the embodiments shown in fig. 4 to fig. 8 are executed. Preferably, the storage medium may include a computer-readable storage medium. The storage medium may include ROM, RAM, magnetic or optical disks, etc.
Further, an embodiment of the present invention further discloses a terminal, which includes a memory and a processor, where the memory stores a computer instruction capable of running on the processor, and the processor executes the technical solution of the method in the embodiment shown in fig. 4 to 8 when running the computer instruction. Preferably, the terminal may be a television, a camera, or the like.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (15)

1. A method of interpolating image data, comprising:
carrying out interlaced scanning on input image data to obtain a plurality of fields arranged according to a time sequence, wherein each field comprises a plurality of points to be interpolated and a plurality of pixel points;
calculating a prediction pixel value of a current point to be interpolated in each field by adopting a time domain interpolation method;
determining a pixel value of a first pixel point adjacent to the current point to be interpolated, wherein the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are located in the same field;
at least carrying out difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point to obtain a difference detection result;
and when the difference detection result exceeds a first preset threshold value, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method.
2. The image data interpolation method according to claim 1, further comprising:
when the difference detection result does not exceed the first preset threshold, carrying out boundary detection on the current point to be interpolated;
if the current point to be interpolated is a boundary point, performing median filtering based on a predicted pixel value of the current point to be interpolated and a pixel value of a second pixel point to determine the pixel value of the current point to be interpolated; otherwise, determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method, wherein,
the second pixel point and the current point to be interpolated are located in the same field, and the distance between any pixel point in the second pixel point and the current point to be interpolated is smaller than a first preset distance.
3. The method of claim 2, wherein the total number of the second pixels is 2, and the performing median filtering based on the predicted pixel value of the current point to be interpolated and the pixel value of the second pixel to determine the pixel value of the current point to be interpolated comprises:
performing median filtering by adopting the following formula to determine the pixel value of the current point to be interpolated:
P(x,y,t)＝mid(Ptemp(x,y,t),F0,F1)；
wherein, P(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y,t)Representing the predicted pixel value of the current point to be interpolated, x representing the coordinate in the horizontal direction, y representing the coordinate in the vertical direction, t representing the coordinate in the time domain field, F0And F1A pixel value representing said second pixel point, mid (P)temp(x,y,t),F0,F1) Represents a pair F0、F1And Ptemp(x,y,t)And performing median filtering.
4. The image data interpolation method according to claim 2, wherein the detecting the boundary of the current point to be interpolated includes:
determining a plurality of third pixel points, wherein each third pixel point and the current to-be-interpolated point are located in the same field, the distance between each third pixel point and the current to-be-interpolated point is smaller than a second preset distance, and the second preset distance is larger than the first preset distance;
calculating the absolute value of the pixel difference between each third pixel point and the current point to be interpolated;
and when the maximum value of all the calculated absolute values of the pixel difference values is smaller than a second preset threshold value, determining the current point to be interpolated as a non-boundary point, otherwise, determining the current point to be interpolated as the boundary point.
5. The image data interpolation method according to claim 4, wherein the absolute value of each pixel difference between any one of the plurality of third pixel points and the current point to be interpolated is calculated by using the following formula:
diff＝|Ptemp(x,y,t)-F(x+i,y+j,t)|,i＝-2,-1,0,1,2；j＝-1,1；t>0；
wherein diff represents the absolute value of the pixel difference, x represents the coordinate in the horizontal direction, y represents the coordinate in the vertical direction, t represents the coordinate in the time domain field, Ptemp(x,y,t)A predicted pixel value, F, representing said current point to be interpolated(x+i,y+j,t)And representing the pixel value of any one of the third pixel points.
6. The image data interpolation method according to any one of claims 1 to 5, characterized by further comprising:
for each field of the input image data, determining pixel values of other points to be interpolated in each field except the current point to be interpolated and pixel values of each pixel point to obtain field data of each field;
based on the field data of each field, output image data is obtained.
7. The image data interpolation method according to any one of claims 1 to 5, wherein the calculating a predicted pixel value of the current point to be interpolated by using a temporal interpolation method for the current point to be interpolated in each field comprises:
for each field, dividing the field into a plurality of strips, and performing data interpolation on points to be interpolated in the plurality of strips, wherein each strip comprises partial points to be interpolated and partial pixel points of the field;
and calculating the current to-be-interpolated point in each strip by adopting a time domain interpolation method to obtain the predicted pixel value of the current to-be-interpolated point.
8. The image data interpolation method according to any one of claims 1 to 5, wherein after the predicted pixel value of the current point to be interpolated is calculated by using a temporal interpolation method, the data interpolation method further comprises:
determining a plurality of other points to be interpolated adjacent to the current point to be interpolated, wherein the plurality of other points to be interpolated and the current point to be interpolated are positioned in the same field, the horizontal direction coordinates of the plurality of other points to be interpolated are the same as the horizontal direction coordinates of the current point to be interpolated, and the distance between any one of the plurality of other points to be interpolated and the current point to be interpolated is less than a third preset distance;
and calculating by adopting a time domain interpolation method to obtain respective predicted pixel values of the other points to be interpolated.
9. The method according to claim 8, wherein said detecting at least the difference between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point comprises:
and performing difference detection on the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point, and performing difference detection on the predicted pixel values of the other points to be interpolated and the pixel value of the first pixel point.
10. The image data interpolation method according to claim 9, wherein the number of the other points to be interpolated is 2, the plurality of fields arranged in the time sequence include a plurality of odd fields and a plurality of even fields, the current point to be interpolated and the other points to be interpolated are located in the even fields, and the predicted pixel values of the current point to be interpolated and the other points to be interpolated are determined by using the following formulas:
Figure FDA0001993675580000031
wherein, Ptemp(x,y,t)A pixel value, P, representing said current point to be interpolatedtemp(x,y-2,t)、Ptemp(x,y+2,t)Respectively representing the predicted pixel values of the other points to be interpolated, x representing a horizontal coordinate, y representing a vertical coordinate, t representing a time domain field coordinate, F(x,y+j,t+1)J ═ 2, 0, 2 each denote a pixel value of a pixel point located in the odd field (t +1), F(x,y+j,t-1)And j ═ 2, 0, 2 each denote a pixel value of a pixel point located in the odd field (t-1).
11. The method of claim 10, wherein the detecting the difference between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point, and the detecting the difference between the predicted pixel values of the other points to be interpolated and the pixel value of the first pixel point comprises:
calculating a pixel difference absolute value between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point;
and calculating the absolute value of the pixel difference between the predicted pixel value of any one point in the other points to be interpolated and the pixel value of the first pixel point.
12. The method according to claim 11, wherein the number of the first pixel points is 2, and the absolute value of the pixel difference between the predicted pixel value of the current point to be interpolated and the pixel value of the first pixel point, and the absolute value of the pixel difference between the predicted pixel values of the other points to be interpolated and the first pixel point are calculated by using the following formulas:
Figure FDA0001993675580000041
wherein, diff0、diff1、diff2、diff3All represent the absolute value of the pixel difference, F(x,y+j,t)And j is-1, and 1 represents the pixel value of the first pixel point.
13. An image data interpolation apparatus, comprising:
the scanning module is suitable for carrying out interlaced scanning on input image data to obtain a plurality of fields which are arranged according to a time sequence, and each field comprises a plurality of points to be interpolated and a plurality of pixel points;
the calculation module is suitable for calculating the current to-be-interpolated point of each field by adopting a time domain interpolation method to obtain a predicted pixel value of the current to-be-interpolated point;
the first determining module is suitable for determining a pixel value of a first pixel point adjacent to the current point to be interpolated, the horizontal coordinate of the first pixel point is the same as the horizontal coordinate of the current point to be interpolated, and the first pixel point and the current point to be interpolated are positioned in the same field;
the difference detection module is suitable for at least carrying out difference detection on the predicted pixel value of the current to-be-interpolated point and the pixel value of the first pixel point to obtain a difference detection result;
and the second determining module is suitable for determining the pixel value of the current point to be interpolated by adopting a spatial domain interpolation method when the difference detection result exceeds a first preset threshold value.
14. A storage medium having stored thereon computer instructions, characterized in that the computer instructions are operative to perform the steps of the method of any one of claims 1 to 12.
15. A terminal comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the method of any one of claims 1 to 12.
CN201910188662.9A 2019-03-13 2019-03-13 Image data interpolation method and device, storage medium and terminal Active CN111294545B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910188662.9A CN111294545B (en) 2019-03-13 2019-03-13 Image data interpolation method and device, storage medium and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910188662.9A CN111294545B (en) 2019-03-13 2019-03-13 Image data interpolation method and device, storage medium and terminal

Publications (2)

Publication Number Publication Date
CN111294545A true CN111294545A (en) 2020-06-16
CN111294545B CN111294545B (en) 2021-03-23

Family

ID=71024172

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910188662.9A Active CN111294545B (en) 2019-03-13 2019-03-13 Image data interpolation method and device, storage medium and terminal

Country Status (1)

Country Link
CN (1) CN111294545B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112927142A (en) * 2021-04-02 2021-06-08 中国人民解放军国防科技大学 High-speed high-resolution video generation method and device based on time domain interpolation
CN113852830A (en) * 2021-09-23 2021-12-28 杭州国芯科技股份有限公司 Median filtering video de-interlacing method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6545719B1 (en) * 2000-03-31 2003-04-08 Matsushita Electric Industrial Co., Ltd. Apparatus and method for concealing interpolation artifacts in a video interlaced to progressive scan converter
CN101018286A (en) * 2007-02-09 2007-08-15 天津大学 De-interlacing method with the motive detection and self-adaptation weight filtering
CN101309385A (en) * 2008-07-09 2008-11-19 北京航空航天大学 Alternate line eliminating process method based on motion detection
CN101437137A (en) * 2008-12-19 2009-05-20 四川虹微技术有限公司 Field interpolation method
CN101640783A (en) * 2008-07-30 2010-02-03 展讯通信（上海）有限公司 De-interlacing method and de-interlacing device for interpolating pixel points
CN102186046A (en) * 2011-05-31 2011-09-14 杭州士兰微电子股份有限公司 Video image de-interlacing scanning method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6545719B1 (en) * 2000-03-31 2003-04-08 Matsushita Electric Industrial Co., Ltd. Apparatus and method for concealing interpolation artifacts in a video interlaced to progressive scan converter
CN101018286A (en) * 2007-02-09 2007-08-15 天津大学 De-interlacing method with the motive detection and self-adaptation weight filtering
CN101309385A (en) * 2008-07-09 2008-11-19 北京航空航天大学 Alternate line eliminating process method based on motion detection
CN101640783A (en) * 2008-07-30 2010-02-03 展讯通信（上海）有限公司 De-interlacing method and de-interlacing device for interpolating pixel points
CN101437137A (en) * 2008-12-19 2009-05-20 四川虹微技术有限公司 Field interpolation method
CN102186046A (en) * 2011-05-31 2011-09-14 杭州士兰微电子股份有限公司 Video image de-interlacing scanning method and device

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112927142A (en) * 2021-04-02 2021-06-08 中国人民解放军国防科技大学 High-speed high-resolution video generation method and device based on time domain interpolation
CN113852830A (en) * 2021-09-23 2021-12-28 杭州国芯科技股份有限公司 Median filtering video de-interlacing method
CN113852830B (en) * 2021-09-23 2023-12-29 杭州国芯科技股份有限公司 Median filtering video de-interlacing method

Also Published As

Publication number Publication date
CN111294545B (en) 2021-03-23

Similar Documents

Publication Publication Date Title
KR101536794B1 (en) Image interpolation with halo reduction
JP4162621B2 (en) Frame interpolation method and apparatus for frame rate conversion
US8144778B2 (en) Motion compensated frame rate conversion system and method
US5784115A (en) System and method for motion compensated de-interlacing of video frames
US7796191B1 (en) Edge-preserving vertical interpolation
US8305489B2 (en) Video conversion apparatus and method, and program
JP2009533928A (en) Method for determining estimated motion vector in image signal, computer program, computer program product, and display device
JP2005318621A (en) Ticker process in video sequence
US7787048B1 (en) Motion-adaptive video de-interlacer
JP2009532984A (en) Motion compensated frame rate conversion with protection against compensation artifacts
KR20070020994A (en) Apparatus for converting image signal and method thereof
JP3842756B2 (en) Method and system for edge adaptive interpolation for interlace-to-progressive conversion
CN103369208A (en) Self-adaptive de-interlacing method and device
CN111294545B (en) Image data interpolation method and device, storage medium and terminal
KR100422575B1 (en) An Efficient Spatial and Temporal Interpolation system for De-interlacing and its method
KR100968642B1 (en) Method and interpolation device for calculating a motion vector, display device comprising the interpolation device, and computer program
JP4355577B2 (en) Up-conversion by noise suppression diagonal direction correction
KR101158847B1 (en) Deinterlacing apparatus and method using edge map
JP5448983B2 (en) Resolution conversion apparatus and method, scanning line interpolation apparatus and method, and video display apparatus and method
KR100931110B1 (en) Deinterlacing apparatus and method using fuzzy rule-based edge recovery algorithm
KR20030082249A (en) Motion adaptive spatial-temporal deinterlacing method
US10264212B1 (en) Low-complexity deinterlacing with motion detection and overlay compensation
KR100624304B1 (en) Apparatus and method for de-interlacing adaptively field image by using motion
KR100949137B1 (en) Apparatus for video interpolation, method thereof and computer recordable medium storing the method
KR100616164B1 (en) Apparatus and method for de-interlacing adaptively field image by using median filter

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant