TWI238369B - Adaptive image deinterlace system based on intelligent network - Google Patents

Abstract

The present invention provides an adaptive image deinterlace system based on intelligent network, which comprises a first intelligent network, for generating a first signal, a second signal, and a third signal based on three image frames; a first interpolating filter, for conducting a first type of interpolation on these image frames based on the first signal; a second interpolation filter, for conducting a second type of interpolation on these image frames based on the second signal; and, a third interpolation filter, for conducting a third type of interpolation on these image frames based on the third signal.

Description

Case No. 91111386_ V. Description of the invention (1) [Preface 1238369 said that the amendment is a smart network (D e-inter 1 ace) system, which is a Lidan] soil adaptive image de-interlacing (Estimation), direction detection and T-shirt, image motion estimation (Motion, various interpolation actions. Table method to the image time frame [background] general TV and DVD movie pictures, rather than computer screen continuous broadcast ° Interlace )

Line) to form an image time frame, and even early scan lines (the Scan conventional TV system uses these two images to form a frame of another image. On the TV screen, the householder is displayed with a “: signal screen number display. The relationship between persistence and perception is not very clear = I is a little bit flickering, but because of the advancement of digital systems and planar display technology, the planar electrical panel (such as liquid crystal display, LCD) technology from single-image : Product :: continued: Leather launched. When displaying the image, the TV signal must be: a ==== member = board must be connected to the-frame, in order to complete the g seven mouths into a single image ", You ca n’t produce a whole TV image. ^ ^ ^ ^ I ί # (WeaVe interpolat.on) Scan line. ^ Or even) Scan line inserts the image of the even-numbered (or singular) monitor into the two images. The time frame is synthesized to conform to the center of the frame when an image is displayed on the plane. I f: This method deletes an image and only repeats the action of repeat scanning (Line Repetiti〇n) at the end. Page 4 1238369 Λ

_f E 9111138R Amendment V. Invention Description (2) But the resolution will be sacrificed by half. For the moving day, the method is only suitable for dealing with static day ~~ Suitable for the moving day, right

Interpolation). Intermediate value interpolation (Median detection, cannot respond to image movement: = Extreme method does not adjust the direction of the post-dynamic daylight surface effect is still not non-af to modify, so its insertion surface. Non-hanging, only suitable When the slow-moving painting image is played, the movement of the characters is mostly dynamic. How | Water and wind are all static, but for the best insertion (Interpc) lat daytime movement and aspects come. On) Action is the focus of this case. [Purpose of this case] The main purpose of this case is to provide a deinterlacing system based on the knowledge of the image, which can be used to interpolate the motion based on the image: cunning network. Speed and direction quality. In addition, the look-up table (Look Up Table) is used to display the necessary calculation time of the day to make the insertion speed faster. 'You can reduce [brief description] a kind of intelligent network-based to achieve the above purpose, this case proposes an adaptive image deinterlacing system, including: a first intelligent network, based on the Trinity signal, a second Signal and a third signal

Page 5 1238369 Case number 91111386

V. Explanation of the invention (3) A first interpolation filter is performed on the frame according to the first signal-the first form of interpolation (interp〇iati〇n) processing; ~ like ku interpolation filter n 'is based on The first: the letter box is subjected to a second form of interpolation processing; and the “like” 4 second interpolation filter is used to perform a third form of interpolation processing on the 0 frames based on the third signal. U pairs ... Qin π Deinterlaced adaptive image based on intelligent network as described above T: bl: The third-smart network includes -first-lookup table =: described by the intelligent network The road-based system, in which the three images of the wide-frame image solution and the wrong image signal. The interlace image of the clever TV is based on the intelligent network as described above. The total A * W should be like the parent figure in the interpretation of the father's mistakes. ^ Represents the frame of these images. The moving process of the images is based on the intelligent network as described. # Zhou Ying image is de-interlaced, and allows the younger brother ^^ to represent the movement of the image. The signal represents the middle of the movement degree of the image, and the largest,瀛 The first image has the smallest degree of movement. The first signal represents that the video is based on the intelligent network as described, where the first-interpolation filter is a woven deinterlacing (Interp〇uu〇n Fluer) ， Yes ， two two internal two cry wave filter (w = interpolation filter (Median Interpolation Fil: W, w mouth is within a median value, * cry no one gives-know λ Doga from er) 'third Interpolation Filter Suppressor 3-First heliocentric circuit and a diamond-shaped interpolation wave filter.

Page 6 1238369

Figure] __ Case No. 91111386 V. Description of the invention (4)

Month: As mentioned, it is an intelligent network system, where the second intelligent network $ (Look Up Table) 〇

Based on the adaptive image de-interlacing system, the interpolation filter system includes a second 杳 I as described in the intelligent network system, wherein the second intelligent network method performs a diamond-shaped Interpolation action. As described above, the intelligent network system is used, where the network is a smart network. 〇 As described, the intelligent network system is used as the neural network. As described, it is based on the intelligent system, in which the third look-up table is based on image resolution and resolution. The image de-interlacing interpolation filter detects the image interpolation. The image de-interlacing system is a type of neural network (Neural road-based adaptive image de-interlacing system. A Sigmoi d function is based on a third look-up table network. The adaptive image de-interlacing system is arranged in a Y-axis proportional manner. : In this case, the following diagrams and detailed descriptions can be used to further understand the intelligent network-based adaptation map F10 of the staggered i U implementation example: This case is a good example. Do mobile estimation (^ + · Estimation) of the first-intelligent network. Estimation Figure 3: This case is better on 1238369 --- Case No. 9111138R_Invention (5) Road. Figure 4: Diamond type interpolation diagram of the preferred embodiment of this case. Figure V. S i gm oid function curve of the preferred embodiment of this case. [Picture No.] / Frames Fl, F2, F3 in the image. 'Second signal S 2 and third signal s 3 Weave interpolation filter 11 Intermediate interpolation filter Device 1 2 The first intelligent network W1 The second intelligent network W2 Diamond-shaped interpolation filter 1 3 1 [Detailed description] Figure 1 is an adaptive image solution based on the intelligent network of the preferred embodiment of the present invention. The block diagram of the parental error system. When three consecutive images are taken, the frames F 1, F 2 ρ 3 are dynamically evaluated (M〇ti 〇η Estimat ion) through the first intelligent network ψ 1 to determine the image screen. The situation of movement. If the result of the evaluation is a static picture (the degree of movement is minimal), a first signal S1 is generated to drive the knit interpolation filter u (Weave,

Interpolation Filter). This filter is used to interpolate the frame in the image at the simplest. '# If the result of the evaluation is a dynamic picture, but the degree of movement is medium, a second signal S2 is generated and the intermediate interpolation filter l2 (Median product

Interpolation Filter) performs scanning line interpolation.

1238369

V. Description of the invention (6) If the result of the evaluation is a dynamic picture, but the degree of movement is the largest, a third signal S3 is generated to drive the second intelligent network W2 and the diamond-shaped interpolation filter 1 3 1 to perform interpolation. Insert motion. ~ Through the mechanism of Figure 1, it is possible to effectively select the daytime surface of various dynamic images-the best filter and wave filter are used for internal interpolation, so that the quality of the display can be improved 'so that the static or dynamic daytime surface can be effectively processed. • Figure 2 is the first d intelligent network for motion estimation (ftimati0n) in the preferred embodiment of the present invention. The component name of Η is the subtractor 21 Subtractor (subtractor), and the circle is the neural-like unit u ( neuron) 'It is a Sigmoid letter age, 7 books, and human stem. Futaibu Doujuan's self-transformation curve is shown in Figure 5. Reducing the calculation time ’, so when it ’s implemented, it ’s two. Table)’ is the first look-up table. Two f; UQQk Up Samples of the time frame (Pat ten), compare with meat = 丨 practice, feed each to find out the corresponding round-out corresponding to various dynamic results = § weekly internal coefficients can be the surface type, If the parameter α is between. ; Π6 ;. The selector 23 determines that the picture output parameters of the picture are captured. If the result is determined to be a dynamic picture of a technical secondary school, and the output of the second work is Q ^ between 0.3 and 1.0, it is determined to be high ^ ^ 蚩 and the parameter α If the agent can create a corresponding table, that is, the first ==: after the training is completed, the time frame can be found immediately: the first look-up table can be found immediately; the moving image is the second embodiment of this case. Test number two, road. The second intelligent network is also derived from the neural-like network 2: ..., the type-network unit 22), which is actually a plurality of neural-like m

In the establishment of the second look-up table temple, it is also used to feed the two: the second look-up table 〇ΐΚιυΙκΙπλίΙΠπ ^ ηΐιΕΓη? £ ιΓκ < μΙτ) 1Γ · ^ η1111 ...... 1238369 case number 91111386_ year month day Amendment V. Description of the invention (7) Out direction parameter / 5. The direction parameter / 3 can be normalized to 0 to 1:

IBI page ίο 1238369

Case No. 9111138R V. Description of the Invention (9) ___ The distortion of y in the middle of X is relatively large. Really, in the tabulation, it is changed to 7 and so on. In order to reduce the loss of y value, you can first convert the s i gmo i d function to: and output the corresponding x. We have x = ln (y) -ln (1-y) so that we can get the value of _ψ v_ X. When 'ran must exclude y = 0 and ^, i, and speech change from 0 to soil, the corresponding indication. The-column is the y value, and the second :::::: f lookup table is shown in Table 3.

Sigmoid can look up the table, you can come to 杳 ’,, '〃, if the tabulation is in progress, if it is an 8-bit 2 2 · value. Of course, the value in between.枓, corresponding to 0 to 255 respectively (^ "The case is mainly estimated by the first intelligent network, and the mobile evaluation can be made based on the frame of different degrees of movement. For the most moving image, it is further 2 = the first: ^, And = do the direction debt measurement, and then use the diamond-type interpolation method; Lu Zha, this way can make the image processing more effective 夂

^ 11 〇 ^ ^ ^ M ^ Slgm〇H t to come to see, in the actual operation can also reduce a lot of computing time-and can increase the speed of image processing. At the same time, the sigmoid is made by checking the y-distance, so when training neural networks, the accuracy can be mentioned. //, 爰, filed a patent application according to law ’The scope of the patent application is attached as follows: II.

Page 12 In summary, this case uses two types of intelligent network architectures to solve the problem. The wrong system can improve the picture quality and speed up the processing. At the same time, the accuracy can be improved, and the progress is unquestionable. The method of this case is unprecedented 1238369-case number 91111386_ year month day amendment__ V. Description of the invention (10) Supplementary notes: 1. Motion Compensation basically searches for an area to determine the correct movement Vector (Motion Vector). If you can correctly find the motion vector, you can get a good quality image, but using this method requires a large amount of external memory, such as DDR SDRAM, f requires line buffering. (Line Buffers), and line buffers need ffiSRAM to be realized, which will consume a lot of chip area. For example, if the search area is 8 X 1 6, there must be at least 8 Line Buffers, which will occupy a large area of the chip . 2. The source of the de-interiace image is Fields, not Frames, that is, only half of the odd field and even field are only half. Information. If you want to search towards 1, you must first fill in the missing half of the information by interpolation. Generally, you need to use the Recursive method to solve the problem. 3. The basic spirit of the case should be through offline learning, so that after completing the learning of various image patterns (pa 11 erns), the system does not need to do complicated calculations every time, but based on the learning experience, the input The pattern (Pat ter ns) makes the correct direction judgment, and then uses the diamond type structure to interpolate to complete the de-interlacing. This method is generally an average method. The advantage is that the final image is smoother and less jagged (Sawt 〇〇th), the disadvantage is that the image is not sharp (Sharp), so in actual implementation, it needs to be supplemented with edge enhancement (Edge Enhancement) post-processing (post-processing) technology in order to obtain the best image quality.

Page 13 1238369 ---- Case I wimsg__year month Nissan thousand_ Brief description of the diagram [Icon] This case can get a deeper understanding by the following diagram and detailed description: Figure 1 Block diagram of the adaptive network-based adaptive image deinterlacing system of the preferred embodiment. Figure 2: The first intelligent network for motion estimation (Motion on Estimation) in the preferred embodiment of the present case. Fig. 2 · A second intelligent network for direction detection in the preferred embodiment of this case. Figure 4: Diamond-shaped interpolation diagram of the preferred embodiment of this case. Figure 5. S i gm 0 i d function curve of the preferred embodiment of the present case. [Picture No.] Frames F 1, F 2, F 3 when the shirt is pictured—Signal S1, second signal S2, third signal S3 Knit interpolation filter 11 Intermediate interpolation filter 1 2 Intelligent network W1 Two intelligent networks W2, stone-shaped interpolation filter 1 3 1 Tea test point Pl, P2, P3, P4, P5, P6 Insertion point p is subtracted as, 21 neural units 2 2 selector 23

This Shenyan case number: 91111386 (the above blocks are filled out by the Bureau) Bulletin | ipc classification said Amendment 1238369 Chinese Invention patent theory, Mingshu 1st month of hard adaptation image deinterlacing system Invention name English

1. Gu Chaoqi 2. Yang Zhixin Inventor (total 2 persons) Name (English) it English —) Residence (Chinese 1.Chao ~ Chee Ku

Republic of China RQC 2. ROC No. 24, 9th Road, 2F) T Pingke Qianyuan & Gong # Dongjiu Road 24, 2F Applicant (1 person in total) / Residence (English) Name or Name (fX) Name or Yin Name (English) (H) i [iit Tu Wen) I Wei 1 All Electronics Co., Ltd. 1-Weltrend Semiconductor, Inc.-ROC, ROC Hsinchu Science Park Industrial Park No. 24 Industrial East 9th Road 2F (this address (Different from previous applicants)

Page Ronga: Correction

i η? 1238369 Emperor No. 91 Chuanzhi V. Description of the invention (8) Stone-0 ~ 0 · 2 represents 45 degree interpolation; 10,000 = 〇 · 2 ~ 〇 · 4 represents 60 degree interpolation; > / 5 = 0 · 4 ~ 〇 · 6 represents 90 ° interpolation; > point = 0 · 6 ~ 0 · 8 represents 12 ° interpolation; Lowan = 0 · 8 ~ 1 · 〇 represents 135 degree interpolation; Type 2 = Can be made into a look-up table after network training, that is, the second look-up table. Therefore, even if there is a frame input when there is an image, the moving direction of the image can be determined immediately. Figure 4 is a schematic diagram of diamond-shaped interpolation of the preferred embodiment of the present invention, in which

Pi'P2'ρ3'ρ4, ρ5, and ρ6 are reference pixels. Using the directional parameter cold in Figure 2, we can decide which points around the insertion point p should be referenced. The directions are divided into five regions in Figure IV. When the direction parameter is between 0 and 0.2, the direction is dl. When performing interpolation, p is interpolated according to p3 + p4. That is, use direction d 丨 and? Interpolate two adjacent points. If the direction parameter / 3 is from 0.2 to 0.4, it is the direction d2. In this direction, the points of the p-phase clock need to take four points p2, p3, p4, and p5 for interpolation. Similarly, the direction d 3 is taken as P2 and p5 for interpolation, the direction (14 is taken as 1) 1,1) 2,1) 5,1) 6, and the direction (15 is taken as Pl, p6. The shape of this interpolation diagram Like diamond, it is named diamond-type interpolation. In addition, the sigmoid function (see Figure 5) used in the neural-like network (the god-like unit 22) is also used to look up the table, that is, the third lookup table The .si gmo id function is as follows: γ-\ / (l + e ~ x) General table lookups (such as Table 1) are based on x to find out the value of y. When tabulating, 'X is equidistant and y Because the function finds the corresponding value. However, the curve of the sigmoid function (as shown in Figure 5), when the value of 辔 is small, but is large near x = 〇 (middle). If you come to Portugal you Zhizhi, on page 11

Claims (1)

She Hu91111386 6. Application for patents include: Year correction # Adapted to the image based on the intelligent network, the winter-wisdom type network is based on the three images, one-two-second signal and one third signal; y ^ 第— Interpolation filter, which is based on the first frame ... Interpolation ^ The second interpolation filter is based on the second frame_ —The interpolation process from the first form; and the third and third interpolation filters are based on the third letter 2 • The interpolation process from the second form is applicable. It is described by wisdom one: ^ π de-interlacing system, wherein the first intelligent type is QL0k Up Table). It is described in item 1 of the patent scope that it is a smart factory and two solutions to the parent error system, in which the three images are time-lapsed (lntMlace) image signals. 4 The wisdom f should be used as described in item 1 of the patent scope of the application, such as a de-interlacing system, where these signals represent the degree of movement of an image. 5 The intelligently adapted image de-interlacing system described in item 4 of the scope of patent application, wherein the third signal represents the greatest degree of generation, the second signal represents the movement of the image, and the first signal represents the smallest degree of movement of the image. 6 · As described in item i of the scope of the patent application, it is necessary to use the smart letter to de-interleave the time frame to calculate the production number, etc .; The Kuang network is a table that shows the degree of the image. It includes the movement of the image frame based on the intersection of the first time and the first time. Based on this type of network 1238369 ---- t #. 91111386_year, month, and day of revision__ 6. The scope of the patent application is adapted to the image deinterlacing system, where the first interpolation filter is a woven interpolation; Weave Interpolation Filter, The second interpolation filter is 'is a median interpolation filter (Median Interpolation F i 11 er)'. The third interpolation filter includes a second intelligent network and a diamond-shaped interpolation filter. Type-based network-based network interpolation filter Type-based network-based network interpolation filter (shape interpolation) Type-based network is based on a neural network-like system ^ ^ ~ e · 6lgm 〇id type 2 network as the base 7 · As described in item 6 of the scope of the patent application, the image deinterlacing system is adapted intelligently, wherein the second intelligent type includes a second look-up table. 8 · Intelligently adapt the image deinterlacing system as described in item 6 of the scope of the patent application, wherein the second intelligent system detects the image movement method and performs a diamond-shape interpolation action 9 as applied The adaptive image deinterlacing system described in item 6 of the patent scope, among which the intelligent comet "Network Network". ~ Tannet 1 0 · The adaptive image deinterlacing system is known as described in item 9 of the patent scope Among them, the _ # function is based on a third look-up table. !! net 1 1 · As described in item 10 of the scope of patent application, the basic image deinterlacing system is adapted, in which the second example is used. Arranged. ~ Check Page 16