WO2011079412A1 - Procédé de redimensionnement rapide d'image par opérateurs multiples - Google Patents

Procédé de redimensionnement rapide d'image par opérateurs multiples Download PDF

Info

Publication number
WO2011079412A1
WO2011079412A1 PCT/CN2009/001576 CN2009001576W WO2011079412A1 WO 2011079412 A1 WO2011079412 A1 WO 2011079412A1 CN 2009001576 W CN2009001576 W CN 2009001576W WO 2011079412 A1 WO2011079412 A1 WO 2011079412A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
operator
scaling
energy
pixel
Prior art date
Application number
PCT/CN2009/001576
Other languages
English (en)
Chinese (zh)
Inventor
董未名
张晓鹏
周宁
Original Assignee
中国科学院自动化研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院自动化研究所 filed Critical 中国科学院自动化研究所
Priority to PCT/CN2009/001576 priority Critical patent/WO2011079412A1/fr
Publication of WO2011079412A1 publication Critical patent/WO2011079412A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4023—Scaling of whole images or parts thereof, e.g. expanding or contracting based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels

Definitions

  • the present invention relates to the field of cross-disciplinary technology combining image analysis, image processing and computer graphics, and more particularly to an image scaling method for fast multi-operators. Background technique
  • the World Wide Web is the main channel for visual information dissemination.
  • the ratio of the vertical and horizontal dimensions of image display devices on the World Wide Web varies.
  • the display effect of the screen needs to meet the limits of these display ratios.
  • the main problem at present is that display devices and display software are difficult to adapt to changes in display scale.
  • Scaling Scaling
  • SL Scaling
  • SC Seam Carving
  • seam welding is a very clever image scaling method that compresses or amplifies the remaining minor areas while keeping the image body as constant as possible.
  • Seam welding is usually based on images The details of the content are judged by complexity, and various techniques such as finding the edge of the image content are used to analyze the primary and secondary of the image content, and the least important pixel seam of the content is found according to the importance of the content, and then deleted ( Image is reduced) or copied (image enlargement).
  • the advantage of seam welding is that it can well maintain the shape of the image body content.
  • the disadvantage is that the overall visual effect of the image cannot be effectively maintained and the local structure of the object (such as the continuity and orientation of the boundary contour) may be significantly damaged.
  • Rubinstein et al. improved the joint welding method (RUBINSTEIN M., SHAMIR A., AVID AN S.: Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3 (2008), 16 ⁇ ), using front end energy (forward energy) Instead of backward energy to protect the edge contours of objects in an image, this method still makes it difficult to maintain the overall visual effect of the image.
  • Multi-Op Dong et al. DONG W., ZHOU N., PAUL J.-C, ZHANG X.: Optimized image resizing using seam carving and scaling.
  • Image scaling method using image Euclidean Distance (IMED) and Dominant Color Descriptor (DCD) to construct an image distance function to splicing seams It is optimized in conjunction with uniform interpolation for image scaling.
  • SCSL image Euclidean Distance
  • SCSL Dominant Color Descriptor
  • the current multi-operator image resizing method maintains the special advantages of various methods in scaling results by combining different methods.
  • Each "operator" in this document corresponds to a specific image scaling method.
  • the multi-operator scaling method successfully uses the image similarity measure to guide the image scaling process, and searches for the best operator sequence in the scaling process, which is attractive.
  • the final scaling effect is primarily determined by the number of uses of each operator.
  • the bi-directional patch matching involved in the multi-operator scaling method is computationally inefficient, making scaling time-consuming and difficult to use. Summary of the invention
  • a fast multi-operator image scaling method includes the steps of:
  • the present invention can be calculated using fixed parameters for most instances and enables fully automated fast image scaling of any size.
  • the present invention can be applied to a large number of fields, such as web image multi-resolution design, advertisement design, movie post-production, and the like.
  • Figure 1 is a flow chart of the algorithm of the present invention
  • Figure 1 DCD effect description.
  • Figure 3 Results of two-dimensional retargeting.
  • the technical problem solved by the present invention constructs a fast multi-operator graph scaling method, and SC, SL and CR are used in combination to perform core content-related scaling on a given image.
  • the method of the present invention is based on the image energy and the dominant color, respectively defining a cost function (const function) for the operator used, and using the cost function to evaluate the local structure and overall vision of each operator on the current image in the scaling process. The damage caused by the effect.
  • the search uses the operation sequence of each operator in combination to quickly scale the given image to any size.
  • the result of the present invention is an image obtained by scaling the source image according to a user-specified size.
  • the core of the present invention consists in constructing the cost function for each scaling operator and optimizing the operator used to select each step of the scaling operation. Without loss of generality, it is assumed that an image of a size is reduced to one pixel for each operator to reduce the width of the image by one pixel.
  • the specific method includes the following steps:
  • the loss of the main object information in the image caused by the operator and the destruction of the local structure of the object record the energy value corresponding to each operator. If the seam welding operator uses the front end energy, it needs to be uniformly interpolated and cut according to w/pair. The energy is corrected; the damage of each operator to the overall visual information of the image is quantified according to the dominant color information of the source image, and the color value corresponding to each operator is recorded; finally, the energy value and color value of each operator are combined. Calculate the cost function values using each operator together;
  • the operator used in this operation is optimized, and the Gibbs sampling method is used to generate the probability that the operator is selected;
  • Image scaling involves changing the size of a digital image, that is, the length and width of the image, as needed.
  • the content description describes the process of keeping the source image as much as possible in image scaling.
  • multi-operator image resizing is the combination of multiple operators.
  • operation sequence the order and number of times each operator acts affects the final result. The sequence they appear is called the operation sequence.
  • Operator field refers to the area of the image affected by each operator.
  • SC it is the pixel on the seam that needs to be deleted or inserted.
  • SL it is all in the image.
  • the pixel, for CR is the pixel on the border of the image.
  • a seam is defined as a set of image pixels that are one pixel wide, with the set of pixels traversing from the upper boundary to the lower boundary of the image, or from the left boundary to the right boundary.
  • SCSL method Core content-related image scaling method proposed by Dong et al. in 2009 (DONG W., ZHOU N., PAUL J.-C, ZHANG X: Optimized image resizing using seam carving and scaling. ACM Trans. Graph 28, 5 (2009), 125.). Use the SC and SL methods to scale the given image.
  • Image similarity measure Quantitatively evaluates the similarity between the scaled result image and the source image.
  • Patch matching The process of finding the smallest pixel block in the source image/result image for the pixel block in the resulting image/source image.
  • Bi-directional patch matching Find the pixel block with the smallest distance in the source image for the pixel block in the result image, and also find the pixel block with the smallest distance in the result image for the pixel block in the source image. process.
  • Scaling The method of finding the output pixel by finding the input image point corresponding to the output image during image scaling and then calculating the weighted average of a set of pixels near the point.
  • Cost function Cost ftmction
  • Saliency map A diagram used to describe the importance of each region of an image. Dominant color: Describes the main color in the image and the proportion of each color.
  • Number of dominant colors The number of primary colors in the image that the user needs to calculate.
  • Forward energy The energy added by the current image after the SC operation.
  • Backward energy The energy of the current image after the SC operation.
  • Image warping A method of changing the shape of an image by calculating a new position of each pixel in the image by a mapping function.
  • Ratio of operators The number of times each scaling operator is used in an operator sequence.
  • the energy of the pixel is used to indicate the local structure of the image.
  • global visual effects are also important.
  • color features are often used to represent global information about an image, independent of the imaging angle, translation, and rotation of the target object and region.
  • DCD determines a small number of dominant color values and their statistical properties: distribution and variance. DCD is defined as
  • the zoom operator zooms in or out on the image in the width or height direction.
  • CR is only used to reduce the size of the image. We remove the lower cost side in the width or height direction.
  • E ⁇ s) (1.0 - max ) . (e(/)) (si) + w max - ma (e ⁇ I)) (si) (4)
  • Scope operation field, operator scaled image
  • One pixel of the region, V s
  • SC s is seam and will be deleted or inserted.
  • SC s is the seam to be deleted or inserted;
  • SL s is all pixels in the image;
  • CR s is the pixel on the entire boundary of the image, if the current operation is in the width direction, then s is Left or right boundary, if the current operation is in the height direction, ⁇ is the upper or lower boundary.
  • the calculation of the largest operator (max) in equation (4) is necessary, because during the scaling process, the deformation of a few elements will cause the local structure of the image to be out of shape.
  • w max 0.5.
  • the operator's color value is calculated by considering the difference between the pixels in s and their corresponding dominant colors:
  • Equation (5) reflects the loss of DCD information after operation using an operator ⁇ 3 ⁇ 4. The present invention uses this method to assess the extent of global visual effects damage.
  • the DCD contains not only information of the color values, but also the percentage of pixels in the image that correspond to each dominant color.
  • the term of equation (5) is used to emphasize the importance of a larger proportion of dominant colors, because a smaller value corresponds to more loss of global visual information, because in this case the color of the pixel is closer to its corresponding The dominant color. This is also a reason we usually use a smaller weight co ded because it has a large impact on the calculation of the (s) value of the item. DCD helps to balance visually important objects well.
  • the forward energy considers the amount of energy inserted into the image, not just the energy that is removed, so that better results can be obtained for many examples, especially in terms of protecting the boundaries of objects.
  • the stochastic mechanism comes from a statistical analysis of the operator cost. This method smoothes the deviation in the operator cost calculation.
  • Our technique of image scaling using the operator cost function greatly speeds up the computational speed of multi-image scaling, and compared to the method of using bi-directional image block matching, it speeds up the operation without compromising the final visual effect.
  • Our algorithm uses 3 operators to reduce the size of an image of 500 X 333 to half the size in 10-40 seconds. Using the same number of operators, the Multi-Op calculation takes 120-1200 seconds, and the SCSL method takes 40-180 seconds. All calculated configurations of the present invention are: Dual core CPU 2.53 GHz, 2 GB memory. This performance can basically meet the interactive requirements of a personal computer.
  • Figure 2(a) shows the source image
  • 2(b) and 2(c) are the energy map and dominant color map of the source image, respectively
  • Figure 2(d) shows the scaling result of SC, where the stem of the left lotus leaf is obvious.
  • Fig. 2(e) shows the scaling result of SL, and the lotus and lotus leaves have obvious deformation;
  • Fig. 2(f) shows the scaling result of CR, the lotus leaf is completely missing, and the lotus is also partially missing;
  • FIG. 3 is the source image, and the results of the method of the present invention (Fig. 3(d)) are used over the use of the Multi-Op method (Fig. 3(b)) and the SCSL method (Fig. 3(c)). More SL operations, better protection of the edge continuity of the object. The results show that the scaling quality of our fast scaling method yields results comparable to those of the Multi-Op and SCSL methods without the need for complex similarity calculations.
  • the method of the present invention can achieve 30-200 times faster acceleration while maintaining the same scaling quality.
  • the inventive method has potential for application in video scaling. This will necessitate consideration of continuity between adjacent frames, especially when there is a significant difference in content.
  • the method of the present invention performs a large number of image tests, most of which are difficult to accomplish with a single scaling operator.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

L'invention porte sur un procédé de redimensionnement rapide d'image par opérateurs multiples. Ce procédé comprend les étapes suivantes consistant à : calculer le descripteur de couleur dominante de l'image d'origine et utiliser l'image d'origine en tant qu'image réelle, calculer quantitativement la valeur de la fonction de coût réalisant l'opération de redimensionnement de l'image réelle au moyen de chaque opérateur, sélectionner de manière optimale les opérateurs nécessaires à cette opération selon la valeur de fonction de coût de chaque opérateur, retirer un pixel à la largeur de l'image réelle au moyen des opérateurs sélectionnés et utiliser la nouvelle image en tant qu'image réelle, enregistrer les opérateurs utilisés dans cette opération et les mémoriser dans la séquence d'opération si les tailles de l'image atteignent les tailles cibles, l'opération étant ensuite terminée. Le procédé peut calculer au moyen de paramètres fixes et peut réaliser automatiquement le redimensionnement rapide d'une image de taille quelconque.
PCT/CN2009/001576 2009-12-29 2009-12-29 Procédé de redimensionnement rapide d'image par opérateurs multiples WO2011079412A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2009/001576 WO2011079412A1 (fr) 2009-12-29 2009-12-29 Procédé de redimensionnement rapide d'image par opérateurs multiples

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2009/001576 WO2011079412A1 (fr) 2009-12-29 2009-12-29 Procédé de redimensionnement rapide d'image par opérateurs multiples

Publications (1)

Publication Number Publication Date
WO2011079412A1 true WO2011079412A1 (fr) 2011-07-07

Family

ID=44226092

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2009/001576 WO2011079412A1 (fr) 2009-12-29 2009-12-29 Procédé de redimensionnement rapide d'image par opérateurs multiples

Country Status (1)

Country Link
WO (1) WO2011079412A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6449398B1 (en) * 1999-06-17 2002-09-10 Hewlett-Packard Company Method for correction adjustment of resized image aspect ratio mismatch
CN101477692A (zh) * 2009-02-13 2009-07-08 阿里巴巴集团控股有限公司 图像特征提取方法及装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6449398B1 (en) * 1999-06-17 2002-09-10 Hewlett-Packard Company Method for correction adjustment of resized image aspect ratio mismatch
CN101477692A (zh) * 2009-02-13 2009-07-08 阿里巴巴集团控股有限公司 图像特征提取方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DONG ET AL.: "Fast Multi-operator Image Resizing.", LIAMA TECHNICAL REPORT., 3 December 2009 (2009-12-03) *

Similar Documents

Publication Publication Date Title
US8717390B2 (en) Art-directable retargeting for streaming video
US8861836B2 (en) Methods and systems for 2D to 3D conversion from a portrait image
US8983236B1 (en) Methods and systems for up-scaling a standard definition (SD) video to high definition (HD) quality
US9111364B2 (en) Method of content aware image resizing
US8374462B2 (en) Content-aware image and video resizing by anchor point sampling and mapping
KR101605983B1 (ko) 얼굴 검출을 이용한 이미지 재구성
US20150131924A1 (en) Creation of Rectangular Images from Input Images
US8649558B2 (en) Video processing with region-based warping
Yan et al. Depth mapping for stereoscopic videos
US8373802B1 (en) Art-directable retargeting for streaming video
US9672866B2 (en) Automated looping video creation
EP2555156B1 (fr) Mosaïcage d'images
Choi et al. Sparse seam-carving for structure preserving image retargeting
Kiess et al. A survey on content-aware image and video retargeting
Cui et al. Distortion-aware image retargeting based on continuous seam carving model
Li et al. Video retargeting with multi-scale trajectory optimization
WO2011079412A1 (fr) Procédé de redimensionnement rapide d'image par opérateurs multiples
Zhang et al. Coherent video generation for multiple hand-held cameras with dynamic foreground
US9330434B1 (en) Art-directable retargeting for streaming video
Chang et al. R2p: Recomposition and retargeting of photographic images
Chamaret et al. Video retargeting for stereoscopic content under 3D viewing constraints
Tang et al. Adaptive cropping with interframe relative displacement constraint for video retargeting
Yuan et al. Video retargeting: A visual-friendly dynamic programming approach
Shankar et al. A novel semantics and feature preserving perspective for content aware image retargeting
Kav-Venaki et al. Feedback retargeting

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09852697

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25-10-2012)

122 Ep: pct application non-entry in european phase

Ref document number: 09852697

Country of ref document: EP

Kind code of ref document: A1