TW201140491A - Video warping graphic processor - Google Patents

Abstract

This invention relates to a video warping graphic processor. The processor includes a memory unit receiving a video including at least one frame; and a processing unit defining a target video cube having a predetermined warping ratio, wherein the target video cube having the predetermined warping ratio includes a specific area, and warping the at least one frame so that the processed at least one frame can conform to the aspect ratio of the target video cube.

Description

201140491 VI. Description of the Invention: The invention relates to an image processor of unequal scale film scaling, which is a processing method for unequal scale movie zooming. In particular, the processing method is a combination The processing method of cutting and unequal-scale film scaling and the application description of related devices and circuits using the same. [Prior Art]

Due to the rapid development of technology, it is an important issue to change the resolution of pictures and videos under different display devices. Different display formats are available under different display devices, such as mobile phones, PDas, and wide. Screen TV, laptop, etc. For example, mobile phones and pDAs of this type of mobile device have higher resolutions due to the smaller screen size and the difference in the ratio of the played picture to the video. At present, the monthly b is expected to linearly scale or crop the left and right sides of the picture and the movie according to the size of the target screen to achieve a predetermined aspect ratio. However, simple linear scaling can cause distortion of the image content, and cropping will remove some of the important edges of the job. Therefore, the county has a scholar's zooming technique, which uses non-equal-distortion distortion to achieve the size and format of the ^ and ^. The shape ratio of the more important inner valley on the human visual (10) is less important. In recent years, content perception maps have been achieved by using non-equal distortions or segmentatKm-based compositions. Unfortunately, the former has the image of the missing image and the image of the segment. Silly people have a piece of heart. However, re-scaling the film is basically different from static ^ and can not completely solve the problem only rely on the existing, _ time _. In the case of the _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ (2) When the important object in the film covers the whole picture, in this case, the zooming technology based on the image content is limited, and the re-slipping is impossible to achieve without removing or distorting the important content. . Therefore, for mobile devices such as mobile phones and PDAs, there will be a smaller screen and the scale of the pictures and videos played will not be

It is also important to be able to propose a film processing method and its broadcast and placement using this method. SUMMARY OF THE INVENTION Like the processing of evil, in 1 is to provide a unequal scale of the film zoom of the thorn, only one unequal scale film scaling processing method. In this section, L will tell you how to make an important image object. In a predetermined scale, the target movie cube must be cut, and the object is not cut, and in the simple time, the image information is still Can be (four). Verifiably, the film is used in 7a cookware at the request of a smaller screen and ratio. The following is a description of this purpose. Example η i 2 2 ΐ 范例 的 的 的 的 狗 是 是 是 是 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗 狗The computer can be set in a video content ί in the identification and/or decision-making period to ask the first-name "Guangte area", in a film re-calibration process 2 reveals a specific scope; The shirt figure is a special ί, the side of the material ·· ^ is used for at least _ 1_) and / or line (^ (4). This decision can be based on the 201140491 in (i) and the information appearing in a frame Relevant content, and/or the content disappears in a next particular number of frames, the next particular number of frames being associated with a particular region 'and/or (ii) containing the relevant region Action = information directed to the foreground object. The example program can further include: base & the information to determine the particular region of the movie @box, the particular ^ is protected from being cropped. For example, the region It can be determined based on visual traffic, and the sample program includes testing the average flow vector associated with each information-related pixel to determine whether information appears in any particular frame of the previous k frames, and continues Any of the frames of the j frames remain visible, where k and $ can be f. The information that failed the test can be marked. The entropy associated with the flow of information related to a particular row is significantly Shift can be determined. Use quantization The flow vector and/or based on the flow probability, the entropy can be determined. The line and the step include some of the most detailed information related to the uniform distribution of the specific large possible entropy H read flow exceeding the predetermined threshold. In an embodiment, the example program may be further included in the movie cube. The unequal-scale program may be executed at least temporarily. === to promote the phase __平_放. (4) Limit, vertex mesh vertices position, wherein the grid-shaped grid position can be linearly combined with points. Variable or multiple _& ΐΐ *Multiple specific regions can be predetermined as a key frame'敎 may be automatically generated and/or human 2011 2011 201140491 In addition, the illustrated unequal scaling subroutine may use a grid comprising a group of squares, and the exemplifying process may further comprise determining at least one with delay = specific The specific square of the flow vector, wherein the specific squares may include a size that is adjacent to at least one further time axis, and the size of the square of the particular square has an outline. Can be made - the zoom process The illustrated unequal scaling subroutine may use a pixel level grid and/or sliding window. In addition, the illustrated process is further included in a user accessible format and/or readable by the manufacturer. The format_storage device is configured to display the display and/or the storage device. For example, the foregoing exemplary process crane access age and system embodiment are applicable, and an embodiment is applicable here. In the course of a movie, a process of unequal scaling of a particular area of a video content is promoted. For example, the exemplary embodiment of the present invention may include, by way of example, a message of at least an image frame. The processing may include at least "specific line and/or touch information. This decision may be made to the frame that appears in the frame and/or is set under a certain number of areas. The disappearance of the domain is significantly made by moving the information of the foreground object. The shadow side is unequal proportional to the viewing rate = including a unequal scale on the film material and the text is protected from being cut off during the cutting process. The present invention provides an unequal-scale film memory unit for receiving at least one of the drawings. The ~w and - fused units are defined to contain the specific target film cubes of the specific 201140491 scale; and the frames are placed such that the processed _box conforms to the target movie cube example. According to still another aspect of the present invention, the present invention provides a _ zoomed image processor, including a memory unit, for receiving a picture frame - a movie and a processing unit, performing (8) a boundary area - a predetermined target movie scaling ratio 3D advance, case scaling, so that after processing 'equal graph:) == the size of the coordinates, so that the film has a new format continued one / f 'The present invention provides an image processing of unequal scale film zoom, including - The memory unit, in order to receive the frame - the movie and a processing unit, performs an unequal scaling so that the processed ί emoticon ratio 'makes the movie have a new format to zoom ίΐίϊ:: A unequal proportion of film shadows (4): New 袼 2 = 2: proportionally retracted into the rectangular body' 俾 该 【 【 【 【 【 【 【 车 车 车 车 车 车 车 车 车 ί ί ί ί ί ί ί ί ί ί ί ί ί ί ί ί ί The PDA or the picture and the zoom ratio of the video will not meet the same requirements as 201140491, and the important image objects will be defined first. For the movie to be scaled, first define one of each frame 1. Calculate the visual flow of each frame. After the transformation, get the flow vector of each element in _ (fbwve 2, will all The flow vector leads to the fan-fiscal graph (such as eh (four) by number statistics' and the probability of distribution of the numbers is imported into the net formula (e coffee to obtain the entropy information of each row of at least one frame. Hey, look for the material to allow the cutting of the special combination and the conversion to the best

f 'conforms to the image cube (Yu s etS =, scale. For s' in each frame through the wire - specific area = === Saki. Her _ to the system can be) 3⁄4 content just appeared in the frame or Is the box, this; does not have the characteristics that continue to appear on the time axis. Figure (2) The feature area must contain a fixed area and the rightmost side is a specific line (:; 2 圏 ^ 明方 (4) - the method of the embodiment step 10: receiving a film containing at least one frame; 4 = Domain Search includes a moving foreground object associated with at least a frame. Step 12: Defining a predetermined scale target image of a particular area 201140491 slice cube 'where the predetermined scale is determined by humans. Step 13: The quantization The process includes importing visual traffic of at least one frame into a statistical chart for number statistics, and introducing the probability of distribution of the numbers into an entropy formula to obtain entropy information of at least one row of at least one frame, wherein the specific region is based on Entropy information is determined.

. For the movie to be scaled, it is necessary to define the image object that is important in the moving area object in a specific area of each frame. The method is as follows: 1: Calculate the visual flow (〇ptical fl〇w) of each frame, and after analysis and quantification, obtain the flow vector of each element in the frame (fl〇w :eCt〇r) ° 2: All flow vectors are imported into a fan chart for number statistics, and the probability of distribution of the numbers is imported into the entropy formula ?^〇Py) 'to find at least the entropy information for each row of the frame. 3: Using the per-line entropy of all the frames, find a specific area that is not allowed to be cropped (critical regionX% and unequal scaling for optimization operation = frame re-scaling) conforms to the target image cube (four) et ▲, size ratio In other words, in each frame, by defining a specific area, the month is retained. Take the practice of narrowing the film as an example. Check 7, box, and define the horizontal component of the specific figure to know the film. The content is in the next amount, ί and test whether it is in or not ^^^ The flow rate of each row of pixels is seen in the frame (4) W, if and ===; 201140491 This line will be marked in a specific area. Determine the specific line to each The entropy of a line of visual flow is used as a basis, and a particular line must contain a significantly moving object whose movement must be independently different from the movement of the photographic lens. To calculate the entropy, first, quantize the visual flow vector using the pie chart Here, c stands for every 2 pixels.) The longer vector is quantized to the area with more divisions (the tiny flow vector is derived from the noise, and there is no need to distinguish more quantization spaces). Let /(3⁄4 denote a flow vector 乂 via quantized integer values such as

Eq.(i):

/(fi) = 2fc+ with fc= [〇.5£(fi)j; 1(3⁄4 and θ(^ respectively represent the length and direction of /, the principle of this formula is explained ^. This fan shape is made up of many pitches - The concentric circle is composed, the radius of the kth = 2 (k + l), and the k ring is divided into 2k equal parts, and the angle of each equal knife 2 is 2π/2"; as shown in Fig. 2 *, all compartments are consecutively numbered from the innermost to 幵D. Place the -flow vector in the original of the pie chart: use Eq.(l) to find the interval number in which the vector is located. In detail, it represents The flow vector is located in the first ring of the pie chart, which is the first interval in a ring of (6)/(&#). y first introduces the entropy function (entn) py). The rotten Fayden is the most left-most right-most boundary of the 11-domain; the entropy function SS={El, E2, E3,.., where AH P={Pl, P2, P3”.., Pn}, Lai Letter H(C) = - Σ P(fi) · 1〇g2 P(fi)·

i ^ C network function money an important feature, its - the value of the function must be greater than zero, the basin ',,, assume the total number of events in the N towel, the job function you 丨 (10) ^ ~ still = ... = where the equal sign When it is established, the entropy function of the system s is the largest at this time. The main reason for 201140491 is here, when the probability of each event is equal, the more average: n to the maximum 'in terms of _flow vector, when the probability distribution 3 j parent flow vector is not the same, and there are important objects to go again Once the probability distribution is concentrated in a range, the volume i representing the flow is an unimportant background area. Therefore, the quantized straight flow distribution probability (the height is mainly per-line m ΐ ) When the city is available, set the #流行 function of a certain line to exceed ο.ι Ξ 特定 specific area; and i is the maximum entropy function value when each event is used, f. (8) and Fig. 3(b) show the boundary of the rightmost boundary of the G. The specific area boundary in the specific area of the department, but does not mean that it is placed in the limit of the operation and time red transition, and the scene:罟: 可以 can be seen on the future time axis. Therefore 'is necessary for detection, because the flow entropy used by the system is a stalk that is difficult to scale in an unequal scale, so that the processed ίί target film is mixed. Size ratio, as shown in Figure 4 (8) and eight uxiH Wherein, in step 14, the at least one optimization frame i is combined with the target data space content in time ratio and the optimization formula is rooted, and the video re-scaling architecture proposed by The unequal scale scaling based on the variational optimization of the two-way continuation, and the optimization is included in the cutting operation. 1. By using the grid grid structure to separate the film cube region, the function calculation network The vertex position of the grid: the minimum energy function value is awaited under several conditions to obtain the new vertex position, and then the re-positioning of each 10 201140491 insert produces a scaled movie. The objective function is composed of 4 f pieces' Achieve the visually important content of time and space, and the coherence of object content on the timeline. Define the mark symbol. In the film, each picture is framed by 仏o, H Zhongguang{ν=ίι,ν〇η } is a collection of vector positions, E and ^ are not squares and * quad faces, and all the frames are connected to each other. The new vertex vt'i, γ is transformed by scaling. · All, optimize the calculation process. To simplify the complexity, 1 converts the symbol t W into , if it is earlier, the vertex of the frame is v, the table *. The reduced movie is large = x, ry, rz), and ry is the scaled resolution of the movie. ^ The total number of frames. ίϋϊ The input movie cube _ is reduced to a predetermined scale. The optimization formula contains the similar energy used to maintain the spatial content, for time consistency. The Temporal Coherence energy of the content on the time axis, and the secondary smoothing energy (sec〇nd_〇rder sm〇〇thin (4)) that smoothes each frame after cropping, and by making An iterative minimization fimction solves a least squares problem for the energy to obtain a set of optimized results. In step 14, at least the frame is unequal scale based on the grid structure (the unequal scale is used to slide the frame on the long and wide axes, and the outer cut is less important. The area, to reduce the accumulation of distortion), to maintain the shape of the object in the particular area, and to use the geometry unit scale for the unequal scaling. In order to achieve time-consistent, the visual flow of at least one geometric unit size of the frame is obtained to obtain a linear deformation of up to 4 and a single 70 size, and to the size of the parent-geometry unit before and after the zoom. Linear deformation can be as consistent as possible. The following is a description of the preferred embodiment. The processing method of unequal proportion film scaling stipulates that after scaling and deformation, the vertices coordinates of the four corners of the original film must be located in the new resolution film. This singer 201140491 can appear in the film cube two, after the non-special domain has been deformed, there is Let V丨 and vU tf, as such, be cut off. The vertices of the dot, · must be cautious | ^ The upper left and lower right corners of the SAR must satisfy the lower ship such as the domain tilted in the target zoom cube, must be ί^ > 0. xj/ ^ yr^0·- Vr<r2 Fora]] 〇<t<7.s; Maintaining an equal-scale movie zoom has a domain U in the timeline. The lungs are not used. The cropping area is used for 7 holdings. The visually important inside each frame. The object, «πηΓΙ^ί, please refer to the data sheet 3〇), the similar energy dimension is two; ^) 3 and Vi4 represent the four vertices of the square q respectively; and the two ί ί is composed of four health words (eg, u (four); the expected q and q, to achieve the best similar solution · · Eq. (4) 4 Γ _

U V

[s W’W = argmin p s ~r 1 v _

—tAr s JVii' L This is a linear least squares problem that can be written as follows, where Qiu. (5) ^%ι Iihi 1 0 - V x[ ~\ Vh Xix 0 1 v1 Aq = :r:i4 Vi4 1 0 ;b9/ = < -Vu •^44 0 1 _ -vL . The matrix Aq is completely dependent on the grid architecture, and bq is an unknown matrix. [ACW, V]q, q, will be able to From the square q to get 12 201140491

Dc{q,q') = [AgiA^Aq)~l-/)b^ can be written as follows Eq.(6) A (A 〆)=ΣΣ W ), »9'

Is the weight value of the visually important square in the tth frame, and the spatial importance map of each frame 'combined color gradient measurement' image visual feature map, and accurate face detection, The important information maps are normalized to a range of [0.1, 1.0] to prevent over-scaling of unimportant regions from causing significant distortion; the unequal scaling method is similar to (KRAHENBijHL, et al. 2009) and (WANQ et al. 2009). In order to avoid strong distortion of the grid line. Eq.(7) = Σ{ί,ί}^Μ-4Ύ+ ^{iJ>€Eh〇^ ~νϊ)2)·, and Eh represent the vertical and horizontal edges in the grid structure, respectively. In order to achieve time-axis consistency of film scaling, this patent proposes a function to maintain object motion information and flash the movie to a minimum. With visual traffic, you can compile ^^: / cornering to the next frame, represented by Pit+i. ^(lineartransformation), π, this real complement does not need to calculate the position of the ground. The purpose of this patent is to reduce the sex of the film when the film is zoomed. Therefore, the ship's volume ’, μ ““必='f fly as follows Eq. (8)

This energy function already includes camera movements. There is no need to deal with these two conditions separately. Leftovers, body movements, A, the unknowns, like the net is determining I think these are the most relevant, the most common combination of Φ Φ 201140491 Quantity is not the same: 〒: 格q (thick) and ρ (thin) In the visual flow stream, the lion read her, "Fuxian face. Use uj+1 d, wd is the average weight of the square point νΓ, defined as follows Eq. (l〇). ~ Use V] Dong Xin (4)) =E丨丨功") t+" 0,k)QE(qj) fc 川, ,Ε(¥) is a collection of squares q 〗. 丄 ΐ ΐ ΐ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Outside the frame of the film, for these squares, use the following Eq(11) to simplify the restriction of the adjacent squares by time. (==(νΓ - <) - (f _ f , 丨丨 2 (M:)eE(4) k ;,1

Eq|^Q = Qf\%, the sum of similar energy on all time axes is as follows: the same amount of profit, miscellaneous _, scalar time axis, phase = body #杂' & represents non-symmetric error in the graph There will be 7 temples in the box to determine the problem of the 'system proposed', which can not only maintain the time axis of the corresponding square, but also progress to the degree of difference. Clearly and ^ axis in Eq. (8), when the movement of the film is very smooth, Lee ^ 2 € take; ^ Γ β and its corresponding square ^ replace the original S ^, Ρ ί, in, set λ = 5. Because in the film, the shape of the object is often not 'in particular, when there is an object or the camera is moving. ii. The example can be allowed to be slightly different when scaling. - To 201140491, the specific energy is used in this implementation. In the example, first select a fixed point and choose. The position is changed between adjacent frames to maintain the top left point v. 'Limited slip function rders_ variable = A = 丨K - (Vf + f )|丨2 ° t 1 The quantity function 'this weight balance energy number ^ type..., his function ke, which makes the ride have _ instead The single top minimum is 3= get an optimal solution' to make the deformation mesh structure distortion reach ^ - Dc + D( + ^Dt + SDe, in the system 'r=i〇, 占=15, as the == The restriction inequality is to protect a specific area from being cropped: and ί The edge flip limit is a conditional inequality, which is non-negative to prevent the grid from being maintained after the grid is placed in the system. 边界 ί = 2 f ensures the frame Re-shrink the boundary. Use the top and bottom lines of each frame to make 1 - sub-change when you are in the palm of your hand. This depends on whether: the opposite is true: love: raw. The system of this patent, according to puatoiset

TtT^cess gradient sg (4) and multi-parallel processing are core miscellaneous. Compared to -u:ri 201140491, it will require a lot of memory space, but it will also greatly reduce the time required for the operation. Once the deformed mesh architecture has calculated the knot, the scaled movie is clipped from the defined target zoom cube. In addition, the image of each frame is scaled to each square, the system uses linear interpolation, or EWA surface splatting using et al 2_] 〇 according to the specific disclosure of the present disclosure The sample program was tested on a desktop PC with a DU0 2.33 GHz central processing unit and a Nvidia 3 f』5 graphics card. For example, using the method described by Rasheed and Shah (please iin17), and then changing the film according to the scene, tailoring the different scenes to re-scale is independent and unaffected' because the field 5 is irrelevant, there is no need to consider Similar hodgepodge has improved, and memory consumption, because the calculated if lattice structure will be more advanced. In this case, the 4 new zoom system is optimized for the operation. 'Depending on the content of the movie, the average needs to use two ^ three ί ž 格 (m Qin 策略 strategy 'in a rough degree. In -^'if' Improve system efficiency when distorted: achieve an average of 6 frames per second, and can be used to demonstrate the results of the sample to prove that it can handle various types of results automatically generated by specific examples based on this method, algorithmic spread : Its body: graphics segmentation (through the program according to the specific example of the disclosure and linear scaling, and by, for example, 201140491, for example, Wang et al. (see Resources 26), motion-based film scaling (MAR) The program is compared with the SVR program edited by, for example, et al. (refer to the time-edited movie scaling (SVR) program. Since both MAR and SVR programs are currently the most advanced film scaling technology, this result is The MAR and SVR programs were compared. Previous studies [w〇lf over spit 2〇?η, [Rubmstem et al· 2〇〇8], [zhang et al. 2〇〇8] did not scale when targeting the movie. Objects that consider motion are consistent on the time axis Sex, therefore, it is inevitable that it is not comparable to action-based methods, 2(8)9]

Based on a wide range of user touches, this overview is derived. Interestingly, the image rescaling method P〇ng et al. 2〇〇9], [Rubinstein et & 2〇 combined with cutting and other techniques] to obtain the most g-like results of the county picture, but These methods require a long time #_, and are used for static, imagery, and do not extend to the film, considering the problem of objects on the timeline. As a comparison - the literature, the most important comparison with [Wangetal 2 Xinjiang; because this study t, it is explicitly processed _ one is its _ sift method, each frame feature point alignment, if m uniform For a monotonous background, =AR will fail, as shown in Figure 7(a)_7(6), where the 7th 3I (b) is a linear zoom (Hnearscaiing) frame. Then, Figure 7 (4) is = question, its ^ can not be in ^: Change: = below: This ^^^^ ^ frame '6th _ and 6th _ is linear scaling (lineaf sm1^ == For S] 201140491, all action type movies can be processed without distortion, and the points are aligned, so compared with MAR, the degree changes and the camera lens moves. Take the knife to solve the grid structure of KrShenbtihletal. 2009 [KrShenbtihletal. 2009] It is silly - now the film is re-scaled. (For example, refer to the big picture to the time when the nutrient is recorded so fast = single =, the upper adjacent image is the same figure (3): the higher important information can be obtained. Then, ^ 彡 纲 桃 桃 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Method, this = shadow = is much better than the result automatically generated) (in) hand = found, there is no phenomenon of sharp shaking of the lens. Inventor 2 2 f: also has the advantage of cutting and unequal scaling, special Under such harsh conditions, the important object of the cut == ί see ί changes the camera path. It is worth mentioning that the method of this case will achieve better results, because S is also added to analyze the operation. However, in order to Change:: The content of the big situation has a ship, __ every square sound is the same. This case experimented with a variety of different mesh analysis network, the operation time and the size of the memory required, structure, ^ Produced '20X20 ' _, Μ grid rack will take time: 吒, ": door ^ 3f tongue although using a fine grid, remember the hidden two, but also make the re-scaling film more smooth 201140491 j in the test ' It is found that the grid structure of the travel agent is fine. The loss of Lin-Lin Naϊίί ^ is based on the investigation of 96 different backgrounds and ages, and the basis of the method is estimated. This # uses ρ_ 1963] The proposed comparison method, which will The results of the film are in contrast to the results of New enbahl et al 2(8) [Wang et al. 2_]; the inventor will provide the results of an original film and either method, allowing the user to choose the re-scaling they like. Version. At the beginning of the test, no special technical guidance was provided to the user to ensure the accuracy of the experiment. In the experiment, six different films were used in this case, using the disclosure algorithm 2009], [W Fat g et al. 2009] Fully automatic width reduction of 5% of the film. Each movie will do two: owe two or two methods, so each user will be asked to do 3 χό = 18 tests. The six films selected in this case include different types of scenes and object motion patterns: such as live shots, CG movies, close-ups, wide-angle shots, single or multiple foreground objects, fast or slow moving lens faces. And with or without parallax effect; in the six films, we used five movie themes and one CG animated short film, and try to keep each clip movie without too many frames' because each movie will add 3 more comparisons. And we can't expect each user to spend more than 20-30 minutes to participate in the experiment. And the questions are presented in random order to avoid bias. In this experiment, the total number m8 (l8x96) of the answer' was obtained and each method was rated a total of 1152 times (2x6x96). Preferred Example Program MVR SVR Total Demonstration Program 488 508 996 MVR 88 - 309 397 SVR 68 267 335 201140491 Table 1 Results. She obviously prefers 86% of the results of this case (9^/1^·α本县县果在比比纽, obtained the vote rate

粢萁冰命iwr *人. Compared with SVR, there is more than 88.2% of the drought rate. In addition, there are more than 84 7% of 152 compared with MAR. 152. Only 34.5; ^= The majority of the participants tend to have better results. The measured coefficient of en two is (4) 356 'represents statistical significance for households < 〇. (U ° Kendall's consistency system, which represents whether there is a contradiction between the two or two cycle comparisons, such as H2 - 3M; In this user result survey, 78% of the respondents, who are in the position of their job, are not satisfied with the miscellaneous, which means that they also: contradictory phenomenon occurs, and the average consistency coefficient of this user survey is | = 〇·94, the standard deviation is 〇j, and there are only 3 user consistency scores 〇·5, indicating that the results of this survey are extremely accurate. In this disclosure, all previous videos are completely rescaled. It is not possible to conduct user surveys, so the focus of this case will be on the two recent technologies [KrShenbiihl et al. 2009], [Wang et al. 2009]. In the study [Krahenbiihl et al. 2009] The SVR method is clearly superior to linear scaling; and [Wang et al. 2009] shows that the results are better than [Wolf et al.2007] and [Rubinstein et al. 2008], so this case is not repeated and then linear. Scaling, [Wolf et al. 2007], [Rubinstein et al. 2008] for comparison According to the user survey, we can further help us to study in depth and understand what re-contracting films most people want to see, but also need more people to assist in the investigation and design more complex tests. For example, this case can also be designed for all use. In 201140491, the original film was not in contact with the various methods. In the m survey experiment, this case is also specifically for the technology used in this disclosure. The technique based on the time of occurrence of the object on the time axis. However, (4) the result of this experiment is slippery. Under the _ table, thirty silk shots are known, and the flattening accepts such m cases. The big side allows the squatters to know what they said before, and keep the shadow of important objects in time 盥 space 1, this is Contradictory goals; if - the movement of the object covers all the faces, in other words, wants to be consistent - all the backgrounds in which the object has moved in space: ϊ put in the case 'unequal ratio Rendering line = putting the cut off part of the visually appearing or future look as shown in the first picture (= first person _ shown 'photo lens over time - straight around the picture ί ί two on the foreground object "Excuse" has covered the method of spatially and has no difference with linear scaling. "疋frit can still retain the shape of important objects of the film: Medium 1 (4) is the original frame, Figure 8 (b) For the linear scaling (l=arscaling) frame '8th _ is the picture processed by Qin et al., and the 8th (d) is the basic example. Compared with the pure use of cutting, the method of this case is a glimpse To prevent important information from being removed by ΐΐίΐ specific areas of the object; this, cut...5 unequal scaling, can greatly reduce the use of virtual lenses; Proportional scaling, effectively utilizing the spatial position 'will compress the unimportant background deformation. The cut and the unequal ratios are (4) 爿, and the variational optimization function strikes a balance between the two. The method of this case is to increase the area of 201140491 which can be deformed and removed in time and space, but there are many visually significant features and foreground objects in the area of the f彡#朗, once rescaled, in time and space. A phenomenon that can cause distortion distortion, such as Figure 9 (8) _ 9th, just shown, where Figure 9 (a) 々 is the original frame, Figure 9 (9) is the linear scaling gaearscaHng), 'The first Figure 9 (6) is a picture processed by Wang et al., and Figure 9 (4) is a real example. In this extreme case, τ must be analyzed from an artistic perspective to determine which of the key frames in the film can be permanently present. In addition, when shooting; the extremes of the sizing method, such as tilting 1' 3Ϊί automatic cutting standards may cause some significant objects to be permanently cut by the knife.... However, the system architecture of this case is very flexible and steerable. ' So as long as the full definition of this type of mobile two-portage = grid; the number ί can not represent this square _ brain when 'the second capacity When allowed, the Han Su level _ grid can be completed "free: device body 23,: position 25 22, - axis storage as shown in Figure 10" computer storage media 2 set 20 Λ select wiring and a sense Detector, etc. This is where the display device 26 performs image signal processing.

SI SI 22 201140491 The screen is controlled to receive and output the processing device 26 and/or the storage device 25 message. In addition, the viewer accesses data in a format/user readable format.颂 7 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或 或Object information (step 3 is at least one specific area of the map (step 33). ",,,,,,,,,,,,,,,,

~ In addition to all the documents proposed and the reference to the (four) heart = set, the implementation of the instruction can be - hard - the synonym name can be summarized in this embodiment.

In the film reconstruction, the system mentioned in the embodiment can save the content of the movie from being moved by the object in this embodiment. In the movie, the object is heavy. The scaling of the image is still preserved. According to the preface, the time dimension of the two pieces is solved. And according to the use of view,! Applicable to the dimension of space and time in the process of reconstruction. - according to the specific exemplary embodiment disclosed in the present invention, since the concept of the overall film is '%%,>{segment age (four) and optimization* program towel is very important, the cost of the meter is relatively higher than that of only The results of the exemplary embodiment of the processing results, the foregoing only describes the principles of the disclosure. Various modifications and changes have been made for the technology in the field, and more (four) systems, devices, and methods are disclosed. However, the original case is still included; and it is a random storage of t== Body = and other processing devices are image processor descriptions that should be known as macro, mamfr_, and then the first stitch scale movie. DETAILED DESCRIPTION Image ϊίΐ GGU The unequal scale film scaling of the present invention includes a memory unit 51G and a processing unit 520. In the case of ΐ, the image processing 11 500 is applied to the slab of the present invention, and the processing unit 520 defines a frame including a zoom target image. In accordance with the disclosure of the target = upper, = then = real: and r within a μ, can be respectively: receive a shadow containing at least one frame, 疋匕 3 one of the specific areas of a predetermined target film scale of the three-dimensional 24 201140491 image The coordinates, and the such frames of the class match the target 岑=仃 unequal scaling, so that the processed slice has a new format for display? - the size of the dimensional image coordinates, such that the image is applied to each of the plurality of - the number of frames - the application of the predetermined scaling, so that each frame is consistent with the three mi ^ ^ has a new format to The steps for displaying the display can also be:

The plurality of frames, the wire-eyes = * and the 03 mesh "image heart six-year s (four) ί target rectangular body, the two-dimensional scale of the rectangular body is one unit time for its third dimension, the image is not a new case & Inside the rectangle, the film has a new format for display. References 1. AVIDAN, S„ AND SHAMIR, A. 2007. Seam carving for contentav ACM Trans· Graph· 26,3,10. image resizing . 2.

BARNES, C., SHECHTMAN, Es FINKELSTEIN, A.? AND GOLDMAN, DB 2009. PatchMatch. A randomized correspondence procedure for structural image editing. ACM Trans. Graph. 28,3. 3· BUATOIS, Ls CAUMON, G., AND LT VY, B. 2009. Concurrent number cruncher; a GPU implementation of a general sparse linear solver. Int. J. Parallel Emerg. Distrib. Syst· 24,3S 205-223. 4. CHEN,L_Q·,ΧΙΕ,X ., FAN, X., MA, W. Y·, ZHANG, J., AND ZHOU, HQ 2003. A visual attention model for adapting images on small displays. ACM Multimedia Systems Journal 9, 4, 353-364. CHO, TS5 BUTMAN, M., AVIDAN, S.5 AND FREEMAN, WT 2008. The patch transform and its applications to image editing. In CVPR 08. 25 201140491 6. DAVE), HA 1963. The Method of Paired Comparisons. Charles Griffin & Company. 7. DESELAERS, T.5 DREUW, Ps AND NEYS H. 2008. Pan, zoom, scan: Time-coherent, trained automatic video cropping. In CVPR. 8. DONG, W., ZHOU, Ns PAUL, J.-C.5 AND ZHANG, X. 2009. Optimized image resi Zing using seam carving and scaling. ACM Trans. Graph. 28, 5, 1-10. 9. FAN, X., XBE? X., ZHOU, H.-Qs AND MA5 W.-Y. 2003. Looking into video Frames on small displays. In Multimedia Ό3,247-250.

10. GAL, R., SORKINE, 0.5 AND COHEN-OR, D. 2006. Featureaware texturing. In EGSR, 06, 297-303. 11. ΓΓΤΙ, L., KOCH3 C., AND NIEBUR, E. 1998. A IEEE Trans. Pattern Anal. Mach. Intell. 20, 115 1254-1259. 12. KARNI, Zs FREEDMAN, D.5 AND GOTSMAN, C. 2009. Energy based image deformation. Comput Graph. Forum 28, 5, 1257-1268. 13. KRAHENBtJHL, P.5 LANG, M., HORNUNG, A., AND GROSS, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28s 5. 14. LIU, Fs AND GLEICHER, M. 2006. Video retargeting: automating pan and scan. In Multimedia Ό 6, 241-250. 15. LIU, H., ΧΙΕ, X., MAS W, Y., AND ZHANG , H.-J. 2003 · Automatic browsing of large pictures on mobile devices. In Proceedings of ACM International Conference on Multimedia, 148-155. 16. PRITCH, Y.? KAV-VENAKI, E.5 AND PELEG, S. 2009 Shift-map image editing, In ICCVO9. 17. RASHEED, Z., AND SHAH, M. 2003. Scene detection in Hollywood movies and TV sho Ws. In CVPR ?03s vol. 2, Π-343-8. 26 201140491 18. RUBINSTEIN, Μ., SHAMIR^ A., AND AVID AN, S· 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3. 19. RUBINSTEIN, Ms SHAMIR, A., AND AVIDAN, S. 2009. Multioperator media retargeting· ACM Trans. Graph· 28, 3,23. 20. SANTELLA, As AGRAWALA, M, DECARLO, D., SALESIN, D.; AND COHEN, M. 2006. Gaze-based interaction for semiautomatic photo cropping. In Proceedings of CHI, 771-780.

21. SHAMIR, As AND SORKINE, O. 2009. Visual media retargeting. In ACM SIGGRAPH AsiaC ourses. 22. SMAKOV, D., CASPI, Y., SHECHHTMAN, E-, AND IRANI, M. 2008· Summarizing visual data using Bidirectional similarity. 23. In CVPR, 08. SUH, B., LING, H_, BEDERSON, B. B., AND JACOBS, DW 2003. Automatic thumbnail cropping and its effectiveness. In Proceedings of UIST5 95-104. 24. VIOLA , P.? AND JONES, MJ 2004. Robust real-time face detection. Int. J. Comput. Vision 57, 2, 137-154. 25. WANG, Y.-S.5 TAI, C.-L·, SORKINE,0·,AND LEE,T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5, 118. 26. WANGS Y.-S.? FU? H, SORKINE , 0.? LEE? T.-Y.5 AND SEIDEL, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Trans. Graph. 28, 5. 27_ WERLBERGER^ M” TROBIN, W.5 POCK,T” WEDEL,A” CREMERS,D·,AND BISCHOF, H. 2009. Anisotropic Huber-Ll optical flow. In Proceedings of the British Machine Vision Conference (BMVC). 28. WOL F, L.5 GUTTMANN, M., AND COHEN-OR, D. 2007. Nonhomogeneous content-driven video-retargeting. In ICCV 507. 27 201140491 29. ZHANG, Y.-F., HU, S.-M. , AND MARTIN, RR 2008. Shrinkability maps for content-aware video resizing. In PG Ό8. 30. ZHANG, G.-Xs CHENG, M.-M., HU, S.-M, AND MARTIN, RR 2009. A Shape-preserving approach to image resizing. Computer Graphics Forum 28, 7, 1897-1906.

[Main component symbol description] 20 processing device 21 processor 22 input/output device 23 computer storage medium 24 execution command 25 storage device 26 display device 10 to 14 steps 31 to 33 step 5 〇 0 image processing of unequal scale film scaling 510 Memory Unit 520 Processing Units 540-560 Steps [Simplified Schematic] FIG. 1 is a flow chart used in the present invention. Figure 2 is a pie chart. Fig. 3 a) and Fig. 3 (b) are schematic diagrams showing the leftmost and right boundary of a specific region by the above method. In steps 4, (8) and 4 (b), the frames are unequally scaled so that the processed _s meets the size ratio of the target movie cube. Figures 5 (8) and 5 (b) The corresponding grid diagram is shown. Fig. 6 (8) - Fig. 6 (8), wherein Fig. 6 (8) and Fig. 6 (4) are frames of the original 28 201140491 'Fig. 6 (b) and Fig. 6 (f) are linear scaling (linear scaling) The frames 'Fig. 6 (c) and Fig. 6 (g) are pictures processed by Wang et al., and Figs. 6 (4) and 6 (h) are the examples. Figure 7 (a) - Figure 7 (d), where Figure 7 (a) is the original frame, Figure 7 (b) is the linear zoom (iinear scaling) frame 'Figure 7 (8) The picture processed by Kr^henbUhl et al., and Fig. 7(d) is the embodiment. Figure 8 (a) - Figure 8 (4) shows that the photographic lens has been around the picture for a long time. The girl's movement track of the foreground object in the picture has covered all the scenes in the space, Figure 8 ( a) is the original frame, Figure 8 is a linear scaling frame, Figure 8 (6) is a picture processed by Wang et al., and Figure 8 (4) is the embodiment. 9 (d), in which the ninth picture (8) is the original picture frame, the first picture (b) j linear scaling (linear scaling) frame, the ninth picture (6) is the picture that the w person handles, and Figure 9 (d) is the present embodiment. Figure 10 is a block diagram of one of the systems of the present invention. For the embodiment mentioned _ - flow off. The block selects the image processor of the unequal scale film zoom of the present invention. 12 - Another flow chart proposed by the embodiment of the present invention. 29

Claims (1)

201140491 VII. Patent application scope: An image processor for unequal scale film scaling, comprising: - a memory unit to receive a movie containing at least one frame; and a processing unit to define a specific area A predetermined scaling movie cube; and unequal scaling of the frames 'so that the processed frames conform to the size ratio of the target movie cube. The image processor of claim 1, wherein the step (a) includes a step of finding a specific region including one of the scene objects associated with the at least one frame. 3. The image processor of claim 2, further comprising a step of finding an optical flow of at least one frame and performing quantization to obtain at least one quantized result accordingly Flow vector (fl〇w vect〇r). The image processor of claim 3, wherein the quanting process comprises: importing at least one frame of visual traffic (〇ptical fl〇w) into a chart=line number statistics, and The probability of the number distribution is imported to determine the entropy information of at least one of the at least one frame. ^ Image processing as described in claim 4 of the patent application [where the specific region is determined based on entropy information. The image processor of claim 1, wherein the predetermined shrinkage ratio is determined by an artificial one. 7. The image processor of claim 1, further comprising at least one frame for unequal scaling, such that the processed 4 frames conform to the size ratio of the target movie cube. 8. The image processor of claim 7, wherein the optimization formula is designed based on spatial content and temporal consistency. /, 'Human 9. As in the image processor described in claim 7, the equivalent formula in i contains similar energy for maintaining spatial content (two f = gy), for time-induced After the similar energy Uempoml Cotoenee er^gyaf^M on the time axis, the secondary smoothing energy (see_-〇rder anoothing _gy) of the flat 201140491 is applied to each frame. 10. If the image is located in the image of the scope of the patent, the ^^(iterative minimization) problem is obtained, and a set of optimized results is obtained. In this step, the n__base_filament structure is used to perform the shape of the object in the specific area without the zoom symbol. =^ Please use the image described in the 2nd item, where the size is used. The grid architecture is unequal scaled. The image processor of claim 11, wherein at least the geometric unit size is obtained for at least the geometric unit size of the at least one of the geometrical dimensions. Deformation can be scaled at least before and after - geometric unit size linear 1 pattern image processing 11, the towel is unequal than sliding on the long and wide axes, cutting the lesser than the external & field, hunting to reduce Accumulation of distortion 15. An image processor of unequal scale film scaling, comprising: a memory unit to receive a movie containing at least one frame; and a processing unit, which is executed: 3D Chen (4) paste The ratio of the box 2^ waits for the frame to enter 彳τ An equal-scale, such that the size of the processed three-dimensional image coordinates of the map is such that the film has a 16.16 image processor that is unequal-scale film scaling, comprising: two ^It units to receive inclusion At least - a frame - a film; and a processing unit, which performs: (a) accepting a plurality of frames; 201140491 'One of the specific areas is predetermined to be scaled to each of the plurality of frames. Scaling 'makes the processed per-display.,,' example, to make the movie have a new format for displaying the image processor of the 2-player movie, including: meta' to receive at least one Frame-film; and a processing early element, is performed (7^ is subject to a plurality of frames containing a target image; a two-dimensional scale of the rectangular body is sufficient to accommodate the image' and The unit time is its third dimension, the unit time ', & the time interval of each adjacent two frames in the plurality of frames; and (c) the target image is unequally retracted into the rectangle, so that The film has a new For display. 32 [S]