TW201141204A - Method of warping video - Google Patents

Abstract

A method of warping video is provided. The method includes the processes of (a) receiving a video including at least one frame; (b) defining a target video cube having a predetermined warping ratio, wherein the target video cube having the predetermined warping ratio includes a specific area; and (c) warping the at least one frame so that the processed at least one frame can conform to the aspect ratio of the target video cube.

Description

201141204 VI. Description of the Invention: [Technical Field] The present invention relates to a device and a circuit for processing a method for unequal-scale film scaling, and more particularly to a method for processing a combination of cropping and unequal 2 film scaling And devices and circuits using the processing method. [Prior Art] Due to the rapid development of technology now, it is important to change the resolution of pictures and films.

= There are various playback formats in the injury, such as mobile phone, = door = device, due to the smaller screen and the displayed picture, according to the size of a large screen, the size of the screen; Or crop the contents of the left and right sides to 2 = hook ground line but simple linear scaling will cause image content = important object information produced at the edge of the image = true near: = the distortion of the scale required for the picture and the film The shape of the valley is proportioned, and the distortion distortion is hidden in less important ί. The content perceives the picture song or the cropping composition is achieved, the defect 'the latter will work ^ the still picture retraction. The existing problems are _: _ method, * can completely solve the problem (a) object plum __ coffee core can, must be clearly 201141204 f; in the conventional technology, each frame in the film re-scaling operation, only f Smoothing the timeline, this method can't handle movies with complex motions' and the ripples and distortions inside. ^ (B) When the important object in the film covers the entire picture, any zooming technology based on the image content will be limited. In this case, it is impossible to rescale without removing or distorting the important content. "For the sake of this, the inventors have invented the processing method of the unequal scale film scaling and the apparatus and circuit using the processing method" in view of the lack of the prior art to improve the lack of the prior art.

SUMMARY OF THE INVENTION One purpose is to provide an unequal proportion of film zoom. In the present invention, it is described how to determine an important image material when the target film cube is avoided from being cropped, and the same (4) bribe can be used (4). Verifiable, the description of the purpose of the different scales of the pictures and videos played by Yuetian. The film processing method of the sun and the moon is not distorted. The following is an example of a shortcoming of this example. The purpose can be to explore the above-mentioned fandom to overcome by a 5, the prior art (such as 'here described here) often related to the lesser period - specific area, In the film recalibration process, the specific example is covered and not cut. For example, according to the video data related to the present frame, for example, the receiving and at least the film processing is configured by hardware, and the example program can also include: deciding to use at least 201141204 for a specific column and/or line ( Row) information. This decision can be made by the base =.= and (d) the information in the - frame, and / or the content disappears within a certain number of frames of iii 'the next specific number of maps And/or (9) contains information about the particular area associated with the foreground object. The example program can further include: based on the comparison of the regions of the movie frame, the particular region is protected from cropping. For example, the region can be determined based on visual traffic, and the sample program includes testing the average traffic vector associated with each information-related pixel to see if any information appears in any particular frame of the previous k frames. And in any of the frames of the continuation of the frame, it remains visible, where k and the number. The * message that has not passed the job can be marked. Significant shifts are determined based on the entropy associated with the flow of information related to a particular row. Entropy can be determined using quantized traffic vectors and/or based on fluency. The step-by-step method includes selecting a specific singularity associated with information of any one of a particular enthalpy or column exceeding a predetermined threshold to select a sneak sigh The sequence can be included in the sub-program of 4 cases, in which a specific area is converted in a target shadow = square. An at least temporary-to-equal proportional function can be used. - Fixed vertices can be restricted to facilitate smooth playback of adjacent frames. The vertices may further include identifying the location of the grid coffee, wherein the grid f can be determined based on the objective function of the least squares difference technique and the ί function. - or a plurality of specific areas may be pre- or a key frame, wherein the predetermined sigh is automatically generated and / or person ^ 201141204. In addition, the illustrated unequal scaling subroutine may be used - i, including a group a square, and the exemplifying process may further comprise a particular square that determines at least one of the flow vectors outside of the particular frame, wherein the square may include a size that is at least - more advanced than at least On the ^ axis = the size of the square of the particular square is the same. Also, you can use a zoom to limit it. The illustrated unequal scaling subroutine can use a water point and/or a sliding window. Moreover, the exemplary process is further included in a storage device display and/or storage device in a format accessible by the user and/or in a user readable format. Moreover, it is not intended to facilitate the aforementioned exemplary process computer access media and embodiments, and an embodiment is applicable. There is also an example of processing image data to facilitate the process of unequal scaling of a particular region of at least one of the tiles during the film reproduction process. For example, the exemplary embodiment of the program may include receiving a video material, including a message about at least an image frame, and a hardware processing configuration, and the program may include at least one specific Line and / Lang determine the information. This decision may be made by (1) being associated with information that appears in a frame and/or is set in a particular number of specific areas (ii) missing information (ii) contains significant moving foreground objects in the associated area. to make. The exemplifying process determines a specific image frame area to be unequally displayed according to the message, and is further included in the film material - unequal scaling, the specific area can be changed within the range of the target film cube濩 to avoid being cut off during the cutting process.申 and other objects, features and advantages, once with the illustrations and tUti reference, 'the following detailed exemplifying embodiment is a hypothetical concept. 'This method provides a method of unequal film scaling'. The method includes private reception. A film containing at least one frame; 201141204 (b) defining a predetermined scale target containing the - specific area; and (e) converting the frame to a frame such that the size of the target movie cube matches the size of the target movie cube. After the processing, according to the above concept, 'the invention provides another kind of processing, less - the frame of the film - the film; (9) defines the unequal scale of the package r chart, so that the processing figure s σ the picture shows the film The size of the 2D image coordinates. The invention further provides a film processing method, the method is a plurality of frames; (b is applicable to each of the plurality of frames; and (4) is scaled for each frame; Each frame after the frame conforms to the pre-proportion. The invention further provides a film processing method, the method os. (a) further comprises a plurality of frames of a target image; (the standard = body ' The two-dimensional scale of the rectangular body is sufficient for the _ target image and the time is its third dimension. The unit time is the time interval of the plurality of frames; and the target image is not equal [implementation] Playing a movie containing at least one frame on a mobile phone with a smaller screen, a PDA, or playing a picture will have different requirements on the zoom ratio of the movie, first defining important image objects. ^ For the movie to be scaled, first define The important image objects contained in one of the frames of the frame, such as moving the foreground object, are protected from being cropped. The method is as follows: 1 «The visual flow of the computing frame 1 (〇pticai fj〇w), After the points After the amount of 201141204, 1 to the flow vector of each pixel in the frame (fl〇w (4). (4) Household 1 has a flow vector to import a fan type statistics, (4) (4) to carry out the special permission of the cut 圔ϋί3 In the parent-by-definition-specific area, the content of each item/area must not be deleted. The savvy traffic is used to calculate the target movie cube containing all the specific areas, and the content outside the ΐϊϊί domain can be Specifically, if you want to make a segment, let the specific column be preserved. Take the imitation of the film as an example. Widening (1) When the content just appears in the frame or is about to disappear in the next picture. Wang, this content does not have the characteristics that continue to appear on the timeline. =) The specific area must contain significant moving foreground objects, and the left and right sides of the area are specific lines. (4) ieal e()lumnb' See section 1 is a diagram of a first embodiment of the present invention. The flow of the method of the present invention is as follows: Step 10 of the method of the method: receiving a movie containing at least one frame; one of the $=1 fields containing at least one The frame is associated Moving foreground object Η *ίΐ12: Defining a target movie cube containing a predetermined scale of a specific area, where the scaling is determined by humans. Step 13: The quantization process includes at least the following visual plan number for numbering Statistics, and the number of the distribution machine to obtain at least one entropy information of at least one frame, the middle domain is determined according to the entropy information. , "X Temple 疋 "201141204 District Chengjin The zoomed movie is defined first in each frame-specific σσ^, 匕3, such as moving the foreground object, important image objects, why not be cut out, the amount t is ^ to calculate the visual flow of each frame, after After vem, get the flow vector line of each pixel in the frame ^^=All flow vector leads-statistics to enter, 旒, and import the number of these numbers into the entropy' to get at least - Entropy information for each line of the map. 3 Shenzhen found a specific __cai that does not allow cutting

The image is removed without seeing the miscellaneous lines; in other words, the g H column can be retained. Take the practice of narrowing the film as an example: Time =:= 2 zone 3__ foreground object. And define the specific..., the leftmost and the rightmost are the critical column , frame, the amount of t of the amount of t can be known, the content of the film leaves at the next ΐ22; therefore, the average flow of pixels per line is different. In the second frame, it will come from the previous frame and will keep ki and will remain available. #ί = ki=k2=3G) 'If these conditions are not met, then e will be "e* in the viewing area _. In each line of S7py), and the object that must touch the meaning of the line, f is strictly independent of the movement of the photographic lens. In order to calculate the entropy, the visual flow vector is quantified by the pie chart ((4) here c generation, mother - 仃 all pixels), the longer vector will be quantized by the area with more thorns 201141204 (the tiny flow vector represents the noise, there is no need to distinguish more quantization spaces). Let / (/D denote a flow vector / The quantized integer value is as follows Eq.(l): 7(fi) = 2 ' + with fc = L〇, 5£(fi)J,

And θ (/3 respectively represent the length and direction of /, respectively, the principle of this formula is as follows: this fan shape is composed of many concentric circles of the same pitch, the radius of the kth ring is 2 (k + l), and The k-ring is divided into 2k equal parts, and each aliquot takes an angle of 2π/2 scoops as shown in Fig. 2, and all the cells are consecutively numbered starting from the innermost one. A flow vector is placed here. The origin of the pie chart 'Eq. (1) is used to find the interval number in which the vector is located. In detail, :(6)" represents the flow rate vector located in the first ring of the pie chart, which is called fi) / (2? r/2fc) The first interval in the j ring. Function (entnW) ° This patent uses the information theory to find the leftmost and rightmost boundary of a specific region; entropy & meaning, 'assuming there is a system s there are multiple events, such as plum, then p-dt The iec 墒 function has several important features. The second is that the N is that the value of the function must be greater than the total number of events in the old (four)-ί, then the entropy function holds <1〇ί , with the net handle main ^ The entropy function of Υ is the largest:; the function of the second function is to be in the 'representing each flow, when the probability = change; then, when the probability = two; there = object again: all - to the ' That represents the unimportant back! Ti: silk 201141204 entropy of each line 'so you can get the day #, = ^ system, fl 'set when the flow entropy function of a certain line exceeds 〇.wmax locomotives are included in a specific area; * ^为#Every event:! ί 'Maximum entropy function value. Fig. 3 (4) and 3rd _ are: The schematic diagram of the leftmost and rightmost boundaries of the 3=^_ region. In this department, the limit of the specific area of the reservation is restricted, but it does not mean that the content must be completely cut; the cropped area of the object must be operated and the _-axis is limited, and when the object is coming Can be seen. Coffee 'is necessary for detection ==3⁄4, because the flow entropy used by the system is - the material is scaled unequally to the frame' so that the size of the target film cube after processing, as shown in Figure 4 (8) and ^ Figure (b) does not. Wherein, in step 14, at least one optimization frame is further included to perform unequal scaling 'so that the size ratio of the processed graphics cubes is 'and the optimization formula is based on the two internal and temporal consistency And design. Dong proposed that the film re-scaling architecture is based on the unequal scale scaling of the variational optimization chrome 2 and the optimization of the cropping operation. By separating the fiscal grid from the film cube area, the 1st function calculates the position of the grid: under several conditions = pass to the minimum energy function, ie ride_location, then for each ϊίϊτ1) Insert repositioning produces a scaled movie. The objective function consists of & conditions' to achieve the visually important content of the reliance and space, and the coherence of the object content on the timeline. Define the tag symbol. In the movie, the grid structure of each frame t is Μι= ίν1 'Q} 'where ν=丨ν^ν:,..,ν=ιη} is the vector position collection, Ε and 201141204 Q represent the edges and squares respectively. Quad face, and all frames are connected to each other. The new vertex after scaling deformation <=(义,}; These variables use the optimization calculation process. To simplify the complexity, the symbol t is reduced to Vi; when the single frame vertex 'is represented by V%. The movie size (rx, ry, rz) '匕 & is the scaled movie resolution. & the total number of frames. Basically, compresses an input movie cube to a predetermined scale target movie cube.

The optimization formula includes a similar energy jc^fonnal energy for maintaining spatial content, a similarity for temporal consistency on the time axis, a quantity (Temporal Coherence energy), and a frame for each frame after cutting. Smoothed secondary smoothing energy (sec〇nd-order smoothing energy). And by using an iterative function (iterative minimizatiQn to solve a least squares problem, to obtain a set of optimized results. ιϋ, in step 14 'at least - Chen based on the exposure architecture and unequal scaling (this unequal The scaling is done using the less important _ field outside the frame, to reduce the distortion of the == lattice structure for unequal scaling, and to achieve the visual traffic of the two = two geometry size to get at least: a few 7; =; before and after the change The following is a better description of the present embodiment. The reddish financial position is fixed after the _ deformation, the dan of the four corners of the vertices must be mann 3 non-specific areas == two U, so that Cut out. Net i top / in the 1 frame 'close to the upper left and lower right corners of the SAR, the apex, must be _ vertices, indeed the fixed area - set 201141204 range interval. In order to force a specific area must be in the target Within the scaling cube, the following conditions must be met as follows Eq.(3): ^>〇, a^'<rx,

Vv < ry, for all 0<t<rz, in addition, the unequal-scale film scaling proposed in this patent has a consistency of the object on the time axis, and does not require a knives region on the time axis. Design different limits. In order to maintain the visually important objects in each frame of the film,

Use the similar energy proposed by Zhang et al. (please refer to item 3). Let Vn, and Vw represent the four vertices of square q, respectively, = parameterization, the parameterization is divided by four numbers, g, ^% v); q and q' of ^, to achieve the best similar solution: As follows Eq.(4) [*)r, u, u]g = argmin Σ

Τ S

Vi,

u V

[= , where Eq (5) is a linear least squares problem, which can be written as '~Vix 1 0 Vh -r.^ 〇1 • » . "4 1 Vix • * * * . . * · * ♦ xi4 -Vu 1 0 -oi 3 bqt == ;,;i4 L ”

Will take strict, grid architecture, and ... a matrix of unknowns. ',]q,q,' will be able to get from square q to c(9!?,) = U9KX)-U;[-/)b7, can be written as follows Eq.(6) D〇= Σ Σ ΚΏα iq', q·'), the weight value' and each frame wq is the spatial importance map of the visually important square 12 201141204 in the t-th frame, combined with the color gradient measurement, the image Visual feature map, as well as accurate face detection, the range of normal information is [0.1, 1.0], preventing undisturbed areas from over-scaling and causing significant distortion; unequal scaling method Approximate to (KRAHENBtFHL, et al. 2009) and (WANQ et al. 2009). In order to avoid strong distortion of the grid line. As follows Eq. (7)

Dt = Σι { - χί/)2+ ~νΙ)2)·

Ev and Eh represent the vertical and horizontal edges in the grid architecture, respectively. In order to achieve time-axis uniform film scaling, this patent proposes an energy function to maintain object motion information and minimize the flicker, trembling, fluctuations, etc. of the film. With visual traffic, you can get the situation when each square evolves from the next frame to the next. 1 t find the best-fit linear transformation Tit (linear, ie =(3⁄4) %Pi+, this example does not need to calculate the inner 都有 包含 因 因 , , , , , , , , , , Rather than its essence. The purpose of this patent is to maintain the above-mentioned line conversion when the film is zoomed. 'Therefore, the following energy functions are formulated. As follows, Ea f starts. This volume function already includes camera lens movement and independent individual motion, and j separately processes These two conditions. The rest are to be stationed, unknowns, like mesh vertices. Using the vertices of u/+1 +1, the vertices are linearly combined with the nearest grid point %+1. As shown in Figure 5. (a) = Analysis shows that, in this example, _ thin squares left view = linear combination of coarse mesh vertices, represented by u symbol. For example, gq g uses u^+1=J2^vd+l: 13 201141204 The average weight of the definition can now be (4)^, re-every (must = EI-cho («) - (d1/)||2, ' Ε (#) is a collection of squares q /. The next 'square Q, has moved to the outside of the film frame, for the phase "the grid can be scaled after the time, she is on the page ί<, Ι) =, £, ί'Ι (ν5,~νί,)-(ν5+, , -^1,)Ι12 'The sum of similar energy on all time axes is as follows: D-EE^)+EE w *(4) t(4) Same amount = use locality limit to achieve the second axis of the retention time axis ifl '14 The non-conformity error will continue to solve 11 problems between the frames, and the method is proposed to maintain the consistency of the corresponding square of the time axis, and to reduce the error degree in the _ step. When the film is very smooth, use β and its corresponding 乂ί PU to call the original qit and its corresponding square ρ, +λ, instead of the original Pi 'in the experiment' set λ = 5. Because in the film, the object The shape is 'small', often 'not when there is an object or the camera moves'. Therefore, the actual _ can be expected to be inconsistent when riding. So far, the energy proposed in this patent only discusses how to adjust the scaling. In this embodiment, a fixed point is selected first, and the top left point V is selected, and the position is restricted to maintain smoothness between adjacent frames. Therefore, borrowed by a second-order smoothing term. As follows Eq (13)

De=nJ2 \\2ν*〇 ~ (νΙ~ν + ν*+1,)||2! 201141204 ο Minimal. Ϊ́ϊϋζ ΐϊϋζ want to extend the best solution, so that the deformation grid structure distortion reaches D = Dc + D^ + yDt + ms, in this system, r = 〗 〖,. The second boundary restriction inequality is to protect the special system and the Φ = it; The straight boundary limit uses the linear equation to determine the boundary of the $ boundary, which is straightforward when it is operated. At the top of the frame = the optimization process, it is necessary to obtain the minimum value of the objective function in some linear constraints and linear conditions. ,tit ...in this embodiment, each time a new constraint equation is generated, it will be used as the basis for the next lamp iteration. Please note here that the matrix will change every time the conditional limit changes, depending on whether or not the contradiction of the transition is violated. The system of this patent 'according to [Buatoisetal.2009] this research, using GPU (GmPhicS Process Unit) common gradient solver (_ gradient solver) and multiple parallel processing as the core architecture, compared to the general single-time implement, it will A lot of memory space is needed, but it also takes a lot of time. Once the deformed mesh structure has been calculated, the scaled movie is clipped from the defined zoom cube. In addition, the image of each frame is scaled to each square, the system uses linear interpolation, or the Gaussian filter set using the solid imaging method proposed by [Krahenbiihl et al 2009] (20118204 meter) (EWA surface splatting) . According to the specific example program of the present disclosure, the central processing unit and the drain card M column of the desktop type s2.33 ghz, for example, use the method described by - and return (please y17), and then change the movie according to the scene, ^Change, no correlation, no tree phase = upper ^ 2 2 ίίΐ improved performance and ^1 body consumption, because the calculated squared growth. In order to scale the movie, the average per square is the pixel content, and the average is two to three generations. This case uses the multi-grid fine Qin (four) strategy, in the rough _ under the ί solution 'in the grid (four) variable material to improve the system efficiency month 匕 in the resolution of 6 stickers especially a8, the total number of 288 = reached an average of 6 per second processing Frames, and can be called ': Shadowmoon. The above mentioned figures show the results of the examples to demonstrate that various types of movies can be processed. According to the automatically generated result of the specific example of the present disclosure, the algorithm uses the parameter setting mentioned above. In some special cases, the case 5 allows the user to manually determine the important object in the film; Box Ul sees the flow of the image to segment the object (Qing (4), and automatically extends to other frames through the visual flow. The program and linear scaling according to the specific example of this disclosure, and such as 'Wang * people (please refer to 26 The action-based movie zoom (MAR) program is compared with the movie zoom (SVR) program edited by the user, for example, 6 times w. It is the most advanced film scaling technology, so it compares the U with the MAR and SVR programs. Previous studies _f vomit 201141204 2007], [Rubinstein et al. 2008], [Zhang et al. 2008] When scaling, there is no consideration of the consistency of the action object on the time axis. Therefore, it is inevitable that it is not comparable to the action-based method. Yangba enbaMetal 2〇〇9] is based on extensive user surveys. Conclusion. Interestingly, the image rescaling method pong et al. 2009], [Rubinstdn et al 2〇〇9] combines cropping and other techniques to obtain the most similar results to the original image in an optimized way, but these methods It takes a long time to calculate, and it only applies to the static image 'and does not shuttle to it, considering the problem of objectivity on the time axis.

/ As a comparative literature, it is mainly compared with [WangetaL2〇〇9]; because in this study, it explicitly deals with the problem of consistency on the time axis. However, it is necessary to carry out the SIFT method to perform the alignment of the feature points of each frame. If the film has only a uniform monotonous background, it will cause the MAR to fail, as shown in Fig. 7 (8)_Fig. 7(d). The seventh is the original ''7th _) silky linearization (frame), the seventh figure (c) is the picture processed by KrShenMihl et al., and the seventh figure is the embodiment. The door ί has a parallax when the feature points are aligned (4) (4), questions, and methods cannot be consistent in the case of objects with different depths of field. ' MAR This method will degenerate into a riding scale, ^ ^ Figure 6 (8) - Figure 6 (h), where picture 6 (8) and p frame, frame of the 6th and 6th tree, 6th (6) and 6th (g) are [Wang The g) and the sixth (h) are the examples. In contrast, instinctively processing all action type movies without distortion, and does not need to: recognize, point, and therefore compare with the certificate, the case === degree change and the camera lens movement of the film. The thinking/fruit [Krahenbtihl et al. 2009] grid architecture is rescaled in pixels and in time. (See, for example, reference 13). For, and 201141204 shows the information of Zhao Zhao; Jfi; == according to visual traffic convex to avoid kneading distortion and fluctuations. (10) The dish will not change when the month b: ft changes significantly. In this experiment, the result of this case is $ ϋ 体 or the camera path is changed violently. It's worth mentioning that g, iiiti uses a smaller, even pixel-level, grid frame and better results, because the visually significant object ϊ ii ^ L accurately analyzes the operation. However, in order to improve the image, it is touched to the 0 η fine grid [because each square in the grid is produced. The shares are all the same. In this case, we experimented with various grids to analyze the grid architecture and then calculate the operation time and the required memory size. M. The system of this case lies in 4〇x4〇, 2〇x2〇, 10x10, 5x5 grid frames. Lai uses a fine grid to save the space. But it also makes the re-scaling film smoother. 丄 It is found in the experiment that the 20x20 pixel grid architecture is based on the balance of quality. According to the previous disclosure, one of the examples of hemp is used to perform the investigation of the method of estimating the case by using 96 different backgrounds and ages. This case uses the comparison proposed by [David 1963] / the film results of this disclosure procedure with [KrShenbtihl et al. 2009] and 201141204

The results of [Wang et al. 2009] are compared to each other; the inventor will provide an original film content and the results of either method, allowing the user to choose the re-scaled version they like. At the beginning of the test, no special technical guidance was provided to the user to ensure the accuracy of the experiment. In the experiment, the film used six kinds of films that did not have a residual algorithm, Yang Ling Technology 2〇09], [Wang et al. 20〇9] fully automatic width reduction of 5〇%. The parent video will be compared twice in a two-two way, so each user will be asked to do 3><6 = 18 tests. The six films selected in this case, including different types of scenes and object motion patterns: such as live shots, CG movies, close-ups, wide-angle shots, single or multiple foreground objects, fast or slow moving shots And with or without parallax effect; in the six films, we used five movie themes and one CG animated short film, and try to keep each clip movie without too many frames' because each movie will add 3 more comparisons. And we can't expect each user to spend more than 20-30 minutes to participate in the experiment. And the questions are presented in random order to avoid bias. In this experiment, the total number of 1728 (18X 96) answers was taken and each method was evaluated a total of 52 times (2χ 6χ96). Preferred Example Program MVR SVR Total Demonstration Program - 488 508 996 MVR 88 - 309 397 SVR 68 267 - 335 Table 1 Table 1 shows the results of the paired comparison of 96 user study participants. A total of 1728 comparisons were performed. In this example, in the table middle column element % table 201141204 shows how many times the method z is better than the method y. And it can be observed from the chart thirty-three that the 'user=check result is obviously preferred by the algorithm of the case, the 'r» is the total, and the result of the invention is obtained after comparison. The rate is 86.5% (996/1152) times. With SVR, she has more than 88.2% of the votes, and in addition to MAR, there are more than 84% of the votes. Conversely, SVR has only 29.1% (335/1152) and MAR only 34.5% (397/1152), which means that most participants tend to prefer the results of this paper. The measured Kendall's consistency coefficient is „ = ()356 ' represents statistical significance for p<〇.〇m Kendall's consistency system, which represents whether there is a contradiction between the two or two cycle comparisons, such as; According to the results of the survey, 78% of users, individual users' preference statistical inconsistency is ambiguous, meaning that they have no contradictory phenomenon; and in this user survey, the average consistency coefficient is J = o.94, the standard The difference is 0.1, and only 3 user consistency scores are f = 〇.5, indicating that the survey results are extremely accurate. 'In this disclosure, a complete user survey of all previous film rescaling techniques is It is unlikely, so the case will focus more on the recent two technologies [Kr^henbtthl et al. 2〇〇9], [Wang et ai. 2009]. In the study [Krahenbiihl et al. 2009], The SVR method is clearly superior to linear scaling; and [Wang et al. 2009] shows that the ratio is better than [w〇lf when al hire η and [Rubinstein et al. 2008] are better. Scaling, [Wolf et aUO〇7], [Rubinstein et al. 2008] Comparisons based on user surveys can further help us to further study and understand what re-contracted films most people want to see, but more people need to assist in the investigation and design more complex experiments. For example, this case can also be designed to make all Use ^ to evaluate various methods without touching the original film. In the user survey experiment, 'this case is also specifically for the technology used in this disclosure: whether the cutting based on the time of occurrence of the object on the time axis will cause the user Watching the film's annoyance. But through the results of this experiment, the user only focuses on the center of the film, whether or not the original video is played, 20 [ S3 201141204 often ignores the information on both sides of the film, and under the sliding, by The chart thirty-three digital ίί dials the flat to accept this kind of cutting situation. The users of the big eve can maintain the type in the film as described before the present invention, which is a contradictory goal; time and space The movement of the object covers all the pictures, in other words, the two important things, the object must be moved in the empty money. 1 hour = the same scaling on the upper, and 愔yp π# ', ', must maintain the effect of the phase. In the situation: the case

Using the crop to remove part of the visually appearing J point J 8 (:) - Figure 8 (4) 'The photographic lens has been on the back of the foreground. In the ί ί:, 'Tao eta1·2009' method and the linear scaling difference, : 疋 = == 7 'remains the important shape of the film, the 8th _ is the original _, the 8th _ is linear scaling ( l=arScaling) frame, Figure 8 (〇 is the second picture of the person, and figure 8 (d) is the embodiment. With pure use of the cutting, leave a specific area of the object To prevent important information from being removed; unequal scaling can greatly reduce the use of virtual lenses; and if the cropping is not available or insignificant, unequal scaling, effectively utilizing the spatial position, will not Important background deformation compression. Crop and unequal ratio ^(歹i scaling use ratio ' is balanced by the variational optimization function. The method in this case is to increase the time and space that can be deformed and removed. 'But in the content of the film space, there are still many visually significant features and foreground objects. Once rescaled, distortion and distortion may occur in time and space, as shown in Fig. 9 (4) - Fig. 9 (d), where Fig. 9 (8) is the original frame, and Fig. 9 (9) The frame for linear scaling (Fig. 9 (c) is the picture processed by Wang et al., and the picture 9 (9) 21 201141204, this is true, =. In this extreme case, it must be from the art The angle is divided into several points, which determines which of the key frames in the film can be permanently present outside the 2f. When the photographic lens is rendered in extreme shooting, such as tilting the shot, the cut may cause some of the lining objects to be permanently However, the system architecture of this case is very flexible and can be restricted by the action guide (4), so as long as this type is fully defined, the problem of tilting the lens must be relied upon. Secondly, the method of this case must be noise and instigation. The age of f may be due to the specific area of the cut, affecting the scaled edge of the edge using the grid structure, when one of the squares contains the i-face object, the conversion data of Yun = 不能 can not represent this grid problem. Pixel level袼Architecture can completely avoid this device 〇 〇 明 明 明 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该 该Receiving and transceiving: This is used - and 26 or / or the storage device 25 is used: 此外. In addition, the explicit access format / user-readable data, display / or library - use The process is as follows: First, the package 22 201141204 contains at least one frame of the movie (step 31). Next, the search includes foreground object information associated with at least one frame (step 32). ), New, and ^ are protected from the specific area (step 33). The foregoing describes various embodiments and principles of the present invention, and is familiar with the technology. From the actual disclosure, the invention can be learned from the simplifications of the present invention and the implementation descriptions from the above (four) buttons, devices, and methods. The concept of the invention will be understood. In addition, it is proposed that the riding literature and reference can be incorporated into the implementation of the present invention. As far as you can understand, the f-brain storage medium contains a memory device that uses three hard disks, such as a memory, a memory, a compact disc, a memory stick, and the like. The processing device/computing device 2 executes the instruction 'maen) 'mainframe and its combination. It is understood by the seven persons that the synonymous nouns and the synonymous terms of the terms mentioned in the embodiment can be included in the summary of the embodiment. According to the above, it can be seen that in the embodiment of the present invention, the apparatus, the storage medium, the method and the flow of the embodiment can cut the inside of the movie. Moving objects play an important role in this detail, and objects moving in the middle of the building can still maintain the scaling of the image. According to the time dimension of the previous 2:=== piece, the problem is solved. According to the present invention, it is applicable to space in the process of reconstruction. According to the specific exemplary embodiment disclosed herein, the analysis and optimization of the segment from the lens to the field f is the process of the JK calculation. Box greening time system. The eyes of the technology-based people have been exposed: ί=理;算的;Sacred according to the case revealed content = the problem. 0 mouth quality, rationality, and results - the results of the implementation of the program can be a nominal reality The content of the above description of the principles of this disclosure. Narrative

{- ·> ^1 J 23 201141204 Various modifications and changes made by the example can be easily accomplished by the teachings of this case. The "New" device and method are disclosed in the present invention, but still contain the principles of the present invention and are incorporated herein by reference. Described here _ Jing domain κ: two == read memory "disc,

Three embodiments are further illustrated by y, wherein the steps of the second embodiment are as follows: (a) receiving a movie containing at least one frame; - a dimension containing - a specific area - a predetermined scaling target movie 2D image coordinates And the box is turned off and the zoom is made so that the processed maps are only the size of the 3D image coordinates of the video. The steps of the third embodiment are as follows: (8) accepting a plurality of frames; 〇>) defining a predetermined scaling ratio of one of the plurality of 敎-敎 regions; and arranging each of the 0 frames to unequal_scale Each frame after processing conforms to the predetermined scaling. The steps of the fourth embodiment are as follows: (a) accepting a plurality of frames including a target image; Η ί 疋 一 a target rectangular body 'the two-dimensional scale of the rectangular body is sufficient to accommodate the fit image' and for one unit time For its third dimension, the unit time is when the adjacent side frame is separated by each adjacent frame; and (c) the target image is unequally retracted into the rectangular body. The above-described embodiments are merely examples for convenience of description, and do not limit the invention of the present invention. Therefore, those skilled in the art can make modifications and changes to the above embodiments without departing from the spirit and scope of the invention. References 1. AVIDAN, S., AND SHAMIR, A. 2007. Seam carving for contentaware image resizing ACM Trans. Graph. 26, 3, 10.

2. BARNES, C., SHECHTMAN, E., FINKELSTEIN, A., AND GOLDMAN, DB 2009. PatchMatch: A randomized correspondence procedure for structural image editing. ACM Trans. Graph. 28, 3. 3. BUATOIS, L., CAUMON, G., AND UE VY, B. 2009. Concurrent number cruncher: a GPU implementation of a general sparse linear solver· Int· J. Parallel Emerg. Distrib.

Syst. 24, 3, 205-223. 4. CHEN, L· Q·, ΧΙΕ, X” FAN, X., MA, WY, ZHANG, HJ, AND ZHOU, Η· Q. 2003. A visual attention model for adapting images On small displays. ACM Multimedia Systems Journal 9, 4, 353-364. 5. CHO, TS, BUTMAN, M., AVIDAN, S·, AND FREEMAN, W. T_ 2008· The patch transform and its applications to image editing. In CVPR Ό 8. 6. DAVID, HA 1963. The Method of Paired Comparisons. Charles Griffin & Company. 7. DESELAERS, T., DREUW, P.5 AND NEY, H. 2008. Pan, zoom, scan: Time- Cohenent, trained automatic video cropping. In CVPR. 8. DONG, W.5 ZHOU, N., PAUL, J.-C.9 AND ZHANG, X. 2009. Optimized image resizing using seam carving and scaling. ACM Trans. Graph 28, 5, 1-10. 9. FAN, X·, ΧΙΕ, X·, ZHOU, H.-Q” AND MA, W, Y. 2003. Looking into video frames on small displays· In Multimedia '03, 247-250.

[S] 25 201141204 10. GAL, R., SORKINE, 0., AND COHEN-OR, D. 2006. Featureaware tejduring. In EGSR '06, 297-303. 11. ITTI, L.# KOCH, C., AND NIEBUR, E. 1998. A model of saliency based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 11, 1254-1259. 12. KARNI, Z., FREEDMAN, D., AND GOTSMAN , C. 2009. Energy based image deformation. Comput. Graph. Forum 28, 5, 1257-1268. 13. BCRAHENBiJHL, P., LANG, M., HOKNUNG, A., AND GROSS, M. 2009. A system for Retargeting of streaming video. ACM Trans. Graph. 28, 5.

14. LIU, F., AND GLEICHER, M. 2006. Video retargeting: automating pan and scan. In Multimedia Ό 6, 241-250. 15. LIU, Η., ΧΙΕ, X., MA, W.-Y., AND ZHANG, H.-J. 2003. Automatic browsing of large pictures on mobile devices. In Proceedings of ACM International Conference on Multimedia, 148-155. 16. PRITCH, Y., KAV-VENAKI, E., AND PELEG, S 2009. Shift-map image editing. In ICCVO9. 17. RASHEED, Z., AND SHAH, M. 2003. Scene detection in Hollywood movies and TV shows. In CVPR Ό3, vol. 2, Π-343-8. 18 RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S. 2008. improved seam carving for video retargeting. ACM Trans. Graph. 27, 3. 19. RUBINSTEIN, M., SHAMIR, A., AND AVIDAN, S 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3, 23. 20. SANTELLA, A·, AGRAWALA, M” DECARLO, D., SALESIN, D., AND COHEN, M. 2006. Gaze-based interaction In semicematic photo cropping. In Proceedings of CHI, 771-780. 26 201141204 21. SHAMIR, A., AND SORKINE, O. 2009. Visual media retargeting. In ACM SIGG RAPH Asia Courses. 22· SMAKOV, D” CASPI, Y·, SHECHTMAN, E” AND IRANI, M. 2008· Summarizing visual data using bidirectional similarity. 23. In CVPR '08. SUH, B·, LING, H., BEDERSON, Β·B·, AND JACOBS, D. W. 2003· Automatic thumbnail cropping and its effectiveness. In Proceedings of UIST, 95-104. 24. VIOLA, P., AND JONES, MJ 2004. Robust real-time face Detection. Int. J. Comput. Vision 57, 2, 137-154.

25. WANG, Y.-S., TAI, C.-L., SORKINE, O., AND LEE, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5,118. 26. WANG, Y.-S., FU, H.5 SORKINE, 0., LEE, T.-Y., AND SEIDEL, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Trans. Graph. 28, 5. 27. WERLBERGER, M., TROBIN, W.5 POCK, T., WEDEL, A., CREMERS, D., AND BISCHOF, H. 2009. Anisotropic Huber-Ll optical flow. In Proceedings of the British Machine Vision Conference (BMVC). 28. WOLF, L., GUmVlANN, M.5 AND COHEN-OR, D. 2007. Nonhomogeneous content-driven video-retargeting. In ICCV 507. 29. ZHANG, Y. -F., HU, S.-M., AND MARTIN, RR 2008. Shrinkability maps for content-aware video resizing. In PG Ό8. 30. ZHANG, G.-X., CHENG, M.-M., HU , S.-M, AND MARTIN, RR 2009. A shape-preserving approach to image resizing. Computer Graphics Forum 28, 7, 1897-1906. [Main component symbol description] 27 201141204 20 processing device 21 processor 22 input / round Out device 23 computer storage medium 24 executes instruction 25 storage Device 26 display device 10~14 steps 31~33 steps

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a view of the present invention. Fig. 2 is a sector diagram. Cheng Tu. Fig. 3 (8) and Fig. 3 (b) are schematic diagrams of the right boundary. The method detects the leftmost and most 4th (8) and 4th (b) of the specific area, and the oblique #& causes the processed scale to switch to unequal scaling, Figure 5 (8) and 5 The ratio of the size of the indicated σ/to the eye. Fig. 6 (8) - Fig. 6 shows the frame of the intent: Fig. 6 (9) and Fig. 6 (E) are the original Fig. 6 (4) and Fig. 6 (h) are the embodiments. The picture processed by Hall I, and shown in Figure 7 (8) - Figure 7 (4), where the 7th 7th (9) is a linear zoom (linear: ί I if the second picture shows), the photographic lens with time - straight surround In the picture, the girl, the singer _ the moving lion on the foreground object, has covered all the background areas of the space 1, where the 8th picture (8) is the original picture frame, the 8th _ is = linear scaling (linearscaling) frame, the 8th Figure (8) is a picture of S] 28 201141204 processed by Wang et al., and Figure 8 (d) is the embodiment. Figure 9 (8) - Figure 9 (8), wherein Figure 9 (8) is the original frame, Figure 9 (b) is a linear scaling frame, Figure 9 (c) is a picture processed by Wang et al., and Figure 9 (d) is the embodiment. Figure 10 is the present invention. A system block diagram. Figure 11 is a flow chart of an embodiment in accordance with the present invention.

29

Claims (1)

201141204: VII. Application for Patent Park: 1. A method for processing unequal proportions of video, the method comprising: (8) receiving a movie containing at least one frame; (b) defining a predetermined scaling target containing one of the specific regions The movie cubes; and (c) unequal scaling of the frames such that the processed frames conform to the size ratio of the target movie cube. The method of claim 1, wherein the step (a) further comprises a step of finding a specific area containing one of the moving foreground objects associated with the at least one frame. 3. The method described in claim 2 of the patent scope further includes a step for finding the visual flow (〇ptical fl〇w) of the frame of less than one frame and quantifying it to pay the corresponding length to Xiyili. The resulting vector of the result (£[qw vector). 4. The method of claim 3, wherein the quanting process comprises importing an optical flow of the frame into the chart for compilation, statistics, and probability of distribution of the numbers. An entropy (entr〇py) formula is introduced to obtain at least one entropy (inr〇py) information of at least one frame. 5. The method of claim 4, wherein the specific area of the towel is determined based on entropy information. #6. The method of claim 1, wherein the predetermined scaling is determined by an artificial one. 7. The method of claim 1, further comprising at least one optimization formula unequal scaling the frames such that the processed frames conform to a size ratio of the target movie cube. 8. The method of claim 7, wherein the optimization formula is designed based on spatial content and temporal consistency. 9. The method of claim 7, wherein the optimization formula comprises a similar energy for maintaining spatial content (c〇nf〇mial energy) for time-based content on a time axis Similar energy (Temp〇rd c〇her_ 30 201141204 garden ^ and pair (4) after each - second-order smoothing energy 〇月扪一人十碉肊罝 = the method described in claim 9 The number (touch _ mi - ^ is determined by using the iterative function small square problem, the get-group optimization of the if ^ amount of coffee milk solution is the most Π = please describe the i-term of the patent system and unequal proportion: which is used Geometry 13. As described in the scope of patent application, the sufficiency, using at least one frame of at least J: = between flow) to the county less - rib unit cymbal cymbal variant: see m, - bribe unit size The method according to the item of the invention, wherein the unequal_zoom=sliding smoothly on the long and wide axes, and cutting the outer region, thereby reducing the accumulation of distortion. f扪15. The method comprises: (a) receiving a movie containing at least one frame; (b) defining the package Including - a specific area - a predetermined scale target: a dimensional image coordinate; and (6) unequal scaling of the frames such that the processed frames conform to the size of the 3D image coordinates of the target movie. An image processing method, the method comprising: (8) accepting a plurality of frames; (b) defining a predetermined scaling ratio for one of the specific regions of each of the plurality of frames; and '(c) for each of the frames Performing unequal scaling so that each processed 31 201141204 frame conforms to the predetermined scaling ratio. 17. An image processing method, the method comprising: (a) accepting a plurality of frames including a target image; (b Defining a target rectangular body, the two-dimensional scale of the rectangular body is sufficient to accommodate the target image, and one unit time is the third dimensional scale of the unit time, the unit time is each adjacent two frames in the plurality of frames a time interval; and (c) causing the target image to be unequally retracted into the rectangular body. 32