WO2014169561A1 - 图像修复方法和装置 - Google Patents

图像修复方法和装置 Download PDF

Info

Publication number
WO2014169561A1
WO2014169561A1 PCT/CN2013/083447 CN2013083447W WO2014169561A1 WO 2014169561 A1 WO2014169561 A1 WO 2014169561A1 CN 2013083447 W CN2013083447 W CN 2013083447W WO 2014169561 A1 WO2014169561 A1 WO 2014169561A1
Authority
WO
WIPO (PCT)
Prior art keywords
curve
image
subset
curves
image blocks
Prior art date
Application number
PCT/CN2013/083447
Other languages
English (en)
French (fr)
Inventor
黄惠
尹康学
龚明伦
陈宝权
汪云海
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to US14/765,447 priority Critical patent/US9547885B2/en
Publication of WO2014169561A1 publication Critical patent/WO2014169561A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/153—Transformations for image registration, e.g. adjusting or mapping for alignment of images using elastic snapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows

Definitions

  • the present invention relates to image processing technology, and in particular to an image repair method and apparatus. Background technique
  • Image restoration refers to the repair and reconstruction of a damaged image, so that the observer does not see that the image obtained by the restoration reconstruction has been repaired.
  • the traditional image restoration methods mainly include two types, one is diffusion-based repair technology, which uses propagation mechanism to propagate information from the undamaged area to the area to be repaired; the other is based on texture synthesis, in the damaged area. Search for pixel blocks that are closer to the boundary pixels of the damaged area, and then copy the found pixel blocks under certain structural constraints and erase the gaps between the pixel blocks.
  • the conventional image restoration methods assume that there is a misalignment between the image blocks in the image to be repaired, that is, there is an overlap region between the image blocks.
  • the actual image restoration process there is a case where there is no overlapping area between image blocks, and therefore, the above two images cannot be applied to the repair of any damaged image.
  • an image repairing method is also proposed.
  • the image blocks that are not overlapped are first interpolated, so that the image blocks overlap in the extrapolated area, and then the image blocks after the extrapolation are registered, thereby realizing no Image restoration of overlapping areas.
  • An image repairing method includes the following steps:
  • the image block is registered by minimizing the energy of the connecting curve in the surrounding field
  • Image filling is performed on the registered image block to obtain a restored image.
  • the step of processing the initially registered image blocks to obtain a connection curve between image blocks is:
  • the step of constructing a surround field of the image to be repaired by the connection curve is:
  • the whole of the curve of the existing direction is used as the boundary condition of the Dirichlet boundary to solve the harmonic vector field; the whole of the second longest curve is extracted as a whole by the pair of non-directional curves, and the harmonic vector field of the whole curve according to the existing direction is determined.
  • the direction of the second longest curve as a whole, until the direction of the non-directional curve is U-direction;
  • the direction portion of the surrounding field is obtained by solving the harmonic vector field of all the curves as a whole, and the size of the surrounding field is calculated according to the number of the associated protruding curves and the projection of the protruding curve.
  • the step of registering the image block by minimizing the energy of the connection curve in the surrounding field is:
  • the transformation corresponding to the image block is obtained.
  • the transformation position and the angle of the connection curve are obtained, and the image block is registered according to the transformation position and the angle of the connection curve.
  • the step of performing image filling on the registered image block to obtain the restored image is:
  • the image is reconstructed from the image block after the structure is filled to obtain the restored image.
  • the step of structurally filling the registered image block by the connection curve is:
  • An image repairing apparatus includes:
  • the processing module includes:
  • An image block obtaining unit configured to acquire an initially registered image block, where the image block is from an image to be repaired
  • a protruding curve acquiring unit configured to obtain a protruding curve of the image block inward along the edge of the initially registered image block
  • the surround field construction module includes:
  • a first extracting unit configured to extract a whole of the first longest curve from the whole curve of the image to be repaired, and give a direction to the whole of the first longest curve
  • a vector field solving unit for solving the harmonic vector field with the entire curve of the existing direction as the Dirichlet boundary condition
  • a circulation unit configured to extract a whole of the second longest curve as a whole by a pair of non-directional curves, and determine a whole direction of the second longest curve according to the harmonic vector field of the curve of the existing direction, until the directionless curve The whole is given direction;
  • the surrounding field forming unit is configured to obtain a direction portion of the surrounding field by solving a harmonic vector field of all the curves as a whole, and calculate a size portion of the surrounding field according to the number of the associated protruding curves and the projection of the protruding curve.
  • the registration module is further configured to obtain a transformation corresponding to the image block by minimizing the energy of the connection curve in the surrounding field, and obtain a transformation position of the connection curve according to the transformation corresponding to the image block. And angle, the image block is registered according to the transformed position and angle of the connection curve.
  • the filling module includes:
  • the structure filling unit is configured to perform structural filling on the registered image block by using a connection curve; and the texture synthesis unit is configured to perform texture synthesis on the image block after the structure is filled to obtain the restored image.
  • the structure filling unit includes:
  • a discretization unit for discretizing the connection curve to obtain an ordered point set
  • a first subset extracting unit configured to extract a first subset from the ordered point set, where the first subset includes all points located on a gap between image blocks;
  • a second subset extracting unit configured to extract, from the ordered point set, a second subset having a smallest color difference from the first subset, wherein the length of the second subset is equal to the length of the first subset ;
  • a merging unit configured to morph and copy the pixel strips of the second subset into the first subset, and fuse the pixels of the first subset with surrounding pixels.
  • the image repairing method and device described above process the initially registered image blocks to obtain a connecting line between the image blocks, and construct a surrounding field of the image to be repaired by the connecting line structure, by minimizing the connecting curve at
  • the image is registered by the energy in the surrounding field, and the image is filled with the imaged image to obtain the restored image. Since the registration of the image block does not require overlapping regions between the image blocks, and minimization is achieved.
  • the connection curve achieves accurate registration based on the numerical optimization of the energy surrounding the field, so it can be applied to the repair of any damaged image and improve the accuracy of the repair.
  • 1 is a flow chart of an image repairing method in an embodiment
  • FIG. 2 is a flow chart showing a method for obtaining a connection curve between image blocks by processing the initially registered image blocks in FIG. 1;
  • Figure 3a is a schematic illustration of an initially registered image block in one embodiment
  • Figure 3b is a schematic diagram of the highlighted curve extracted in Figure 3a;
  • Figure 3c is a schematic view of the connection curve in Figure 3b;
  • Figure 3d is a schematic diagram of a surrounding field constructed by Figure 3c;
  • Figure 3e is a schematic diagram of the registered image block
  • Figure 3f is a schematic diagram of the image block after filling
  • FIG. 4 is a flow chart of a method for obtaining a salient curve in an image block along the edge of the initially registered image block in FIG. 2;
  • Figure 5a is a schematic diagram of a prominent curve associated with each other in an embodiment
  • Figure 5b is a schematic diagram of the distance calculation performed in Figure 5a;
  • Figure 5c is a schematic diagram of the association set formed in Figure 5b;
  • Figure 5d is a schematic diagram of the interrelated highlight curves obtained by screening through the association set of Figure 5c;
  • FIG. 6 is a flowchart of a method for finding a correspondence relationship of starting points in a salient curve by a robust algorithm in one embodiment
  • FIG. 7 is a flow chart of a method for constructing a surrounding field of an image to be repaired by a connecting curve in FIG. 1;
  • FIG. 8a is a schematic diagram of an image block between registrations in an embodiment;
  • Figure 8b is a schematic diagram of the image block after registration corresponding to Figure 8a;
  • FIG. 9 is a flow chart of the image obtained by image filling the image block after registration in FIG. 1 .
  • FIG. 10 is a flow chart of a method for structurally filling a registered image block by a connection curve in FIG. 9;
  • Figure 11 is a schematic view of the structure filling in Figure 10.
  • FIG. 12 is a schematic structural diagram of an image repairing apparatus in an embodiment
  • Figure 13 is a schematic structural view of the processing module of Figure 12;
  • Figure 14 is a schematic structural view of the protruding curve acquiring unit of Figure 13;
  • Figure 15 is a schematic structural view of the connection curve acquisition module of Figure 13;
  • Figure 16 is a schematic structural view of the surround field structure module of Figure 12;
  • FIG. 17 is a schematic structural view of a filling module of FIG. 12;
  • FIG. 18 is a schematic structural view of the structure filling unit of FIG. 17. detailed description
  • an image repairing method includes the following steps: Step S110: Processing an initially registered image block to obtain a connection curve between image blocks.
  • the initially registered image blocks are image fragments from the image to be repaired, and these initially registered image blocks form the image to be repaired.
  • the initially registered image block is roughly arranged by the user for the image block.
  • the connection curve obtained by processing the initially registered image blocks is used in the gap existing between the image blocks for the image block in which the associated relationship is in a broken state.
  • Step S130 constructing a surround field of the image to be repaired by connecting curves.
  • the surround field includes a size portion and a direction portion, which can be used to mark the magnitude of the energy of the curve in the image to be repaired and its direction.
  • Step S150 registering the image block by minimizing the energy of the connection curve in the surrounding field.
  • a certain optimization algorithm is used to minimize the energy of the connecting curve in the surrounding field, and further, based on the numerical optimization, the energy of the connecting curve in the surrounding field is at least converged to the local optimal result.
  • the registration of the image block is achieved by the minimized connection curve, so that the registration of the image block does not deviate, and the accuracy of the registration in the image to be repaired is improved.
  • the optimization algorithms applied to the registration image block may be the steepest descent method, the Newton method, and the BFGS algorithm. Here, no - enumeration.
  • the steepest descent method also known as the gradient method
  • the gradient method is to search for the optimal value in the negative direction of the gradient when solving the unconstrained problem of minimizing the energy of the connected curve in the surrounding field.
  • the Newton method is used.
  • the gradient of the objective function and the quadratic function of the Hesse matrix are used to find the extremum. After each step, a new quadratic function is again calculated for optimization.
  • Step S170 performing image filling on the registered image block to obtain a restored image.
  • the gaps existing between the registered image blocks are filled so that a plurality of image blocks form a complete image.
  • the above image repairing method can repair the two-dimensional broken image, and can also repair the three-dimensional broken image, which will not be described here.
  • the foregoing step S110 includes:
  • Step S111 Acquire an initially registered image block, the image block being from an image to be repaired.
  • a plurality of image blocks in the broken image are subjected to rough initial registration by a user-triggered operation to obtain an initially registered image block, as shown in Fig. 3a.
  • Step S113 obtaining a prominent curve in the image block along the edge of the initially registered image block.
  • the line segment in the image block is extracted by a certain algorithm or an interactive means, that is, the extraction of the highlighted curve, for example, the smart scissors method input by the user interaction or the method of calculating the highlighted curve in the image block.
  • the smart scissors method is to acquire a series of seed points of the user's point set on the image, and automatically connect the seed points by finding the minimum energy path between adjacent seed points to finally obtain a corresponding curve, and the curve is Highlight the curve as shown in Figure 3b.
  • a salient curve will be calculated by algorithms such as edge-preserving filtering, partial derivation, boundary extraction, and processing of boundaries.
  • the foregoing step S113 includes:
  • Step S1131 Perform edge-preserving filtering on the initially registered image block, and obtain partial gradient to obtain a gradient map corresponding to each image block.
  • the algorithm used to implement the edge-preserving filtering may be weighted least squares filtering, anisotropic diffusion, robust smoothing, and bilateral filtering, etc., which are not enumerated here.
  • Step S 1133 marking a boundary of the image block, and obtaining a gradient peak point on the boundary of the mark according to the gradient map corresponding to the image block.
  • boundary extraction is performed for each image block, and the extracted boundary is marked. Specifically, it will be assumed that the image block that has been initially registered is under a white background. At this time, the extraction of the image block boundary can be realized by applying a certain boundary extraction algorithm.
  • the applied boundary extraction algorithm can be a relatively simple binary segmentation method or a dynamic contour method.
  • Step S 1135 the curve is detected from the gradient peak point on the boundary to obtain a corresponding curve, and the curve forms a curve set of the image to be repaired.
  • the peak points of the gradient are found along the boundary of the image block, and the peak points are taken as the starting point, and a plurality of steps are detected inside the image block along the direction with the largest gradient value to obtain a corresponding curve.
  • the curve is a plurality of curves to form a set of curves corresponding to the image to be repaired.
  • Step S 1137 extracting a salient curve from the curve set.
  • the average curvature above, and the above four items are divided by the corresponding maximum value to ensure that the value of the value range is [0, 1]
  • the curve in the curve set is filtered by the calculated score to obtain a salient curve. Specifically, after calculating the score corresponding to each curve in the curve set, it is determined whether the calculated score is greater than a threshold, and if not, discarding the corresponding curve, and if so, retaining the corresponding curve, and the final retained curve is To highlight the curve.
  • the threshold can be set to 1.0.
  • Step S115 obtaining the mutually protruding protruding curves according to the protruding curves, and obtaining a connecting curve between the mutually adjacent protruding curves.
  • the protruding curves associated with each other are protruding curves having a corresponding relationship between the starting points on different image blocks, and the curve for connecting the two adjacent protruding curves is a connecting curve, and the connecting curve is also The curve connecting the two image blocks is on the gap between the image blocks, as shown in Figure 3c.
  • the surrounding field of the image to be repaired constructed by the above step S130 is as shown in FIG. 3d, and the image of the connecting curve in the surrounding field is minimized by the above step S150 to register the image block as shown in FIG. 3e.
  • the complete image as shown in FIG. 3f can be obtained by corresponding filling, and the broken image can be truly restored without overlapping regions, and the accuracy of image restoration is improved.
  • step S115 includes:
  • the corresponding relationship between the starting points in the salient curve is found by the robust algorithm.
  • the starting point of the corresponding relationship is located on different protruding curves, and the protruding curves where the starting points of the corresponding relationship are located are located on different image blocks.
  • the protruding curve in which the starting point of the corresponding relationship exists is a protruding curve that is associated with each other, and the curve between the mutually adjacent protruding curves is a connecting curve.
  • the connecting curve is an Emilet curve, and the highlighted curve as framed in Figure 5a is an associated protruding curve.
  • the plurality of salient curves of the image to be repaired are paired to obtain a starting point where the corresponding relationship exists, and the salient curve where the starting point of the corresponding relationship exists is an interrelated protruding curve, which is connected by a smooth Emile curve.
  • Two interrelated sharp curves, the Emirt curve is the connecting curve, and the connecting curve connects the two image blocks through the gap between the image blocks.
  • the step of finding the correspondence between the starting points in the salient curve by the robust algorithm includes:
  • Step S 1151 constructing a scalar field of the image to be repaired according to the protruding curve.
  • the scalar field of the image to be repaired can be constructed by the following formula:
  • n is the number of prominent curves
  • r is any point of space
  • ⁇ is the starting point of curve j
  • ⁇ is the projection of r to the outwardly close arc at the starting point of curve j
  • ⁇ is the control parameter
  • the default is 0.05
  • Step S1153 for any starting point of an arbitrary protruding curve, connecting any starting point and other starting points to each other through a connecting curve, obtaining an integral of the scalar field on the connecting curve as a distance, and forming an associative set with the starting point corresponding to the minimum distance as an element.
  • the correspondence between the starting points in the arbitrary protruding curves is obtained according to the difference between the surrounding pixels corresponding to the elements and the pixels around any starting point.
  • the protruding curve includes starting points respectively located at both ends of the line segment. For any starting point, it is connected with other starting points around the line, and the line segments connecting the two starting points are connected curves, and at this time, between each other The two starting points with the smallest distance are the starting points where the correspondence exists.
  • the starting point and the other starting point are first connected by a smooth Emilet curve, and the other starting points are different from the starting point, and The protruding curve and the protruding curve where the starting point is located are at the starting point of different image blocks, and then the integral of the scalar field on the Emmett curve is obtained as the distance between the two starting points, and not simply Apply the distance between two points.
  • any starting point is connected to the scalar field and another starting point to which it is the smallest distance to calculate the distance.
  • the starting point corresponding to the extracted minimum distance is used as an element to form an associative set.
  • the formed related set contains a plurality of elements, and therefore, further extraction of the elements is required.
  • the surrounding pixels corresponding to each element in the association set compare the difference between the surrounding pixels and the pixels around the starting point, thereby retaining the element with the smallest difference, and deleting the remaining elements from the associated set.
  • the element, that is, the starting point is corresponding to the starting point.
  • the two frame-selected protruding curves connected by the connecting curve are the associated protruding curves.
  • the foregoing step S130 includes:
  • Step S131 extracting the whole of the first longest curve from the whole curve of the image to be repaired, and giving a direction to the whole of the first longest curve.
  • the curve of the image to be repaired is formed by a joint curve of the correlation and a connection curve connecting the mutually adjacent protruding curves.
  • the longest curve is extracted from the entirety of the plurality of curves, and the extracted curve as a whole is the first longest curve as a whole, and a random direction is given to the first longest curve as a whole.
  • Step S133 Solving the harmonic vector field by using the entire curve of the existing direction as the Dirichlet boundary condition.
  • the extracted first longest curve is solved as a blending vector field for the Dirichlet boundary condition.
  • the second longest curve is extracted as a whole by a pair of non-directional curves, and the overall direction of the second longest curve is determined according to the entire harmonic vector field of the curve of the existing direction until the direction of the non-directional curve is given to the whole.
  • the direction of the first longest curve extracted is given and the size of the harmonic vector field is solved correspondingly.
  • the remaining curves are all non-directional, and therefore, the remaining curves will be one by one.
  • the overall direction is determined.
  • the longest curve is extracted from the whole of the non-directional curve, and the extracted curve as a whole is the second longest curve as a whole, and the second longest curve is determined according to the harmonic vector of the curve of the existing direction as a whole.
  • the direction, and the overall extraction of the curve, and the determination of the direction, until all the curves are given the direction, the longest curve extracted each time is the second longest curve as a whole.
  • the specific process of determining the overall direction of the second longest curve according to the overall harmonic vector field of the curve of the existing direction is: testing the second longest curve as a whole in the forward and reverse directions corresponding to the extending direction and the existing
  • the overall direction of the curve of the direction is the degree of agreement of the vector field, the direction with the highest degree of agreement is taken as the direction of the whole of the second longest curve, and the harmonic vector field of the curve of the existing direction is calculated, and so on, until all the curves are integrated. Both are given directions.
  • Step S137 obtaining a direction part of the surrounding field by solving the harmonic vector field of all the curves as a whole, and calculating the size part of the surrounding field according to the number of the associated protruding curves and the projection of the protruding curve.
  • k is the number of associated pairs of salient curves
  • m(j) represents the index of the curve associated with the connecting curve j
  • ⁇ is the projection of r to the outwardly intimate arc at the starting point of the connecting curve j
  • ⁇ 2 is Control parameters, the default is 0.01.
  • step S150 The transformation corresponding to the image block is obtained by minimizing the energy of the connection curve in the surrounding field, and the transformation position and angle of the connection curve are obtained according to the transformation corresponding to the image block, and the image block is registered according to the transformation position and the angle of the connection curve.
  • the registration of the image block is performed under the indication of the surrounding field, that is, the transformation T of the image block is obtained, so that the direction corresponding to all the connection curves is as consistent as possible with the direction of the surrounding field. Since the transformation of the image block will affect the position and angle of the connection curve, and thus the parameters of the connection curve, the connection curve for connecting the protruding curve is essentially a function related to the transformation T of the image block, ie H(T). ), at this point, you can get the relevant optimization formula:
  • f(r) ⁇ - t(r) T V(r) + l( ⁇ - d(r)) where / ⁇ is the ith element of H, and t(r) is / ⁇ at point r Tangential direction.
  • the default is 0.1
  • v(r) is the vector direction at point r
  • d(r) is the vector size at point r.
  • the BFGS optimization algorithm is used to solve the above optimization formula to obtain the transformation position and angle of the connection curve, and then the position of the image block is adjusted according to the transformation position and angle of the connection curve, thereby obtaining the transformation and transformation of the image block.
  • the subsequent image block is the registered image block.
  • step S170 includes:
  • Step S171 structurally filling the registered image block by using a connection curve.
  • connection line between the image blocks is used for structural filling to restore the image to be repaired according to the image structure, so that the image structure can be maintained as much as possible, and the image structure of the filled area is not caused as in the conventional image filling method. Blurred question.
  • step S171 includes:
  • step S1711 the connection curve is discretized to obtain an ordered point set.
  • Step S1713 Extracting the first subset from the ordered point set, the first subset including all the points located on the gap between the image blocks.
  • Step S1715 Extract a second subset having the smallest color difference from the first subset from the ordered point set, and the length of the second subset is equal to the length of the first subset.
  • the process of extracting the second subset from the ordered set of points is actually a process of searching the pixel block similar to the gap in which the first subset is located along the curve.
  • a pixel strip is extracted from the whole curve including the connection curve, and the first subset is used as a target point set, and the extracted pixel strip is deformed to obtain a deformed pixel strip, and the pixel strip is the first
  • the pixels around the subset that is, the color difference between the pixels in the image block in the first subset is the smallest, so as to seamlessly stitch the image blocks.
  • the transform performed on the extracted pixel strip, that is, the second subset may be a moving least squares transform, or may be replaced by other algorithms, such as a thin plate spline function method and a radial basis function method.
  • the thin plate spline function method is an interpolation method for finding a smooth curved surface with the smallest curvature through all the control points.
  • the surface of the thin iron plate is made by a given number of "splines”. Smooth, and the minimum degree of bending is also defined by the energy function.
  • the deformation of the second subset by the thin plate spline function method is to treat the entire image to be repaired as a xoy plane in a three-dimensional space, the displacement of the control point along the z direction, and the thin plate-like function interpolation is applied.
  • the displacements of the other points are calculated to calculate the displacement of all the pixels in all the second subsets to achieve image distortion.
  • the radial basis function method is mainly used to solve the interpolation problem. Since the radial basis function is only related to the distance to a certain point, it is called radial, and because the radial basis function is the basis used to approximate the function in the function space. , so called radial basis.
  • the image to be repaired is regarded as the xox of the three-dimensional space.
  • the displacement of the control point is along the Z direction.
  • the displacement of other points can be calculated by interpolation of the radial basis function, thereby calculating the displacement of all the pixels and realizing the image deformation.
  • Step S1717 deforming and copying the pixel strips of the second subset into the first subset, and merging the pixels of the first subset with the surrounding pixels.
  • the pixel strip corresponding to the second subset is deformed and seamlessly cloned into the gap where the first subset is located.
  • the Poisson fusion method is used to make the filling more Realistic, natural.
  • Step S173 performing texture synthesis on the image block after the structure is filled to obtain the restored image.
  • the image block in which the structure is filled is subjected to texture synthesis to implement filling of other parts in the image to be repaired.
  • the texture synthesis of the image to be repaired may be implemented by using a content-aware filling method to obtain the restored image.
  • the texture synthesis of the image block is realized by the content-aware filling method.
  • the most similar known blocks of all the image blocks containing it are calculated when the recovery value is calculated for the missing pixels, and the color values corresponding to the known blocks are summed to obtain the same. The best value for this one missing pixel.
  • an acceleration operation will also be introduced in the content-aware filling process to achieve a fast search of the closest known blocks by a combination of neighbor recommendation and random search.
  • an image repair apparatus includes a processing module 110, a surround field construction module 130, a registration module 150, and a fill module 170.
  • the processing module 110 is configured to process the initially registered image blocks to obtain a connection curve between the image blocks.
  • the initially registered image blocks are image fragments from the image to be repaired, and these initially registered image blocks form the image to be repaired.
  • the initially registered image block is roughly arranged by the user for the image block.
  • the connection curve obtained by processing the initially registered image blocks is used in the gap existing between the image blocks for the image block in which the associated relationship is in a broken state.
  • the surround field construction module 130 is configured to obtain a surround field of the image to be repaired by the connection curve.
  • the surround field includes a size portion and a direction portion, which can be used to mark the magnitude of the energy of the curve in the image to be repaired and its direction.
  • the registration module 150 is configured to match the image block by minimizing the energy of the connection curve in the surrounding field Quasi.
  • the registration module 150 minimizes the energy of the connection curve in the surrounding field by a certain optimization algorithm, and further ensures that the energy of the connection curve in the surrounding field at least converges to the local part based on the optimization of the value.
  • the optimal result is that the registration of the image block is achieved by the minimized connection curve, so that the registration of the image block does not deviate, and the accuracy of the registration in the image to be repaired is improved.
  • Registration Module 150 The optimization algorithms applied to register image blocks may be the steepest descent method, the Newton method, and the BFGS algorithm, where not enumerated.
  • the steepest descent method also known as the gradient method
  • the gradient method is to search for the optimal value in the negative direction of the gradient when solving the unconstrained problem of minimizing the energy of the connected curve in the surrounding field.
  • the Newton method is used.
  • the gradient of the objective function and the quadratic function of the Hesse matrix are used to find the extremum. After each step, a new quadratic function is again calculated for optimization.
  • the filling module 170 is configured to perform image filling on the registered image block to obtain a restored image. In this embodiment, the filling module 170 fills the gap existing between the registered image blocks, so that several image blocks form a complete image.
  • the above image repairing device can repair the two-dimensional broken image, and can also repair the three-dimensional broken image, which will not be described here.
  • the processing module 110 includes: an image block acquiring unit 111, a salient curve acquiring unit 113, and a connection curve acquiring module 115.
  • the image block obtaining unit 111 is configured to acquire an image block that is initially registered, and the image block is from an image to be repaired.
  • a plurality of image blocks in the broken image are subjected to rough initial registration by a user-triggered operation to obtain an initially registered image block.
  • the protruding curve acquiring unit 113 is configured to obtain the protruding curve of the image block inward along the edge of the initially registered image block.
  • the salient curve acquiring unit 113 performs line segment in the image block by a certain algorithm or an interactive means, that is, extraction of the salient curve, for example, a smart scissor method input by a user interaction or a method of calculating a salient curve in the image block.
  • the smart scissors method is to acquire a series of seed points collected by the user on the image, and automatically connect the seed points by finding the minimum energy path between adjacent seed points. In order to finally get the corresponding curve, the curve is a prominent curve.
  • the salient curve acquisition unit 113 calculates a salient curve by an algorithm such as edge-preserving filtering, partial derivation, boundary extraction, and processing on a boundary.
  • the above-described salient curve acquisition unit 113 includes a filter derivation unit 1131, a boundary processing unit 1133, a curve detection unit 1135, and a curve extraction unit 1137.
  • the filter derivation unit 1131 is configured to perform edge-preserving filtering on the initially-registered image block, and obtain a gradient map of each image block.
  • the algorithm for implementing the edge-preserving filtering by the filtering and deriving unit 1131 may be weighted least squares filtering, anisotropic diffusion, robust smoothing, and bilateral filtering, etc., which are not enumerated here.
  • the filtering and deriving unit 1131 performs edge-preserving filtering on each image block that has been initially registered by any of the above algorithms, and obtains a gradient map corresponding to each image block by performing image differentiation by image-preserving image blocks.
  • the boundary processing unit 1133 is configured to mark a boundary of the image block, and obtain a gradient peak point on the boundary of the mark according to the gradient map corresponding to the image block.
  • the boundary processing unit 1133 performs boundary extraction on each image block and marks the extracted boundary. Specifically, the boundary processing unit 1133 assumes that the image block that has been initially registered is under a white background. At this time, the extraction of the image block boundary can be realized by applying a certain boundary extraction algorithm.
  • the boundary extraction algorithm applied by the boundary processing unit 1133 may be a relatively simple binary segmentation method or a dynamic contour method.
  • the curve detecting unit 1135 is configured to perform curve detection from the gradient peak point on the boundary to obtain a corresponding curve, and the curve forms a curve set of the image to be repaired.
  • the curve detecting unit 1135 finds the peak point of the gradient along the boundary of the image block, and further uses the peak point as a starting point to detect a plurality of steps inside the image block along the direction with the largest gradient value to obtain a corresponding curve. At the time, the obtained curve is plural to form a curve set corresponding to the image to be repaired.
  • a curve extraction unit 1137 is configured to extract a salient curve from the curve set.
  • the curve extracting unit 1137 filters the curve in the curve set by the calculated score to obtain a salient curve.
  • the curve extracting unit 1137 determines whether the calculated score is greater than a threshold, and if not, discards the corresponding curve, and if so, retains the corresponding curve, and finally The retained curve is the highlighted curve.
  • the threshold can be set to 1.0.
  • the connecting curve obtaining module 115 is configured to obtain the protruding curves associated with each other according to the protruding curves, and obtain a connecting curve between the mutually adjacent protruding curves.
  • the protruding curves associated with each other are protruding curves having a corresponding relationship between the starting points on different image blocks, and the curve for connecting the two adjacent protruding curves is a connecting curve, and the connecting curve is also A curve connecting two image blocks, on the gap between the image blocks.
  • connection curve obtaining module 115 is further configured to find a correspondence relationship between the starting points in the protruding curve by using a robust algorithm, where the starting point of the corresponding relationship is located on different protruding curves, and the starting point of the corresponding relationship exists.
  • the highlighted curves are located on different image blocks.
  • the protruding curve in which the starting point of the corresponding relationship exists is a protruding curve that is associated with each other, and the curve between the mutually adjacent protruding curves is a connecting curve.
  • the above connecting curve is an Emilet curve.
  • connection curve obtaining module 115 pairs the plurality of salient curves of the image to be repaired to obtain a starting point where the corresponding relationship exists, and the protruding curve where the starting point of the corresponding relationship exists is an interrelated protruding curve, and then passes through a smooth
  • the Emile curve connects two interrelated sharp curves, which are the connecting curves, and the connecting curves connect the two image blocks through the gap between the image blocks.
  • connection curve acquisition module 115 includes a scalar field construction unit 1151 and a start point correspondence unit 1153.
  • a scalar field construction unit 1151 is configured to construct a scalar field of the image to be repaired according to the salient curve.
  • the scalar field construction unit 1151 can construct a scalar field of the image to be repaired by the following formula:
  • n is the number of salient curves
  • r is any point of space
  • ⁇ is the starting point of curve j
  • ⁇ is the projection of r to the outwardly close arc at the starting point of curve j
  • ⁇ is the control parameter
  • the default is 0.05 .
  • the starting point corresponding unit 1153 is configured to connect the arbitrary starting point with the other starting point through the connecting curve to obtain the starting point of the arbitrary protruding curve, and obtain the integral of the scalar field on the connecting curve as the distance, and the starting point corresponding to the minimum distance.
  • Forming an associative set as an element forming an associative set according to the difference between the surrounding pixels corresponding to the element and the pixels around any starting point, and obtaining the starting point of the arbitrary protruding curve according to the difference between the surrounding pixels corresponding to the element and the pixels around any starting point Correspondence relationship.
  • the protruding curve includes starting points respectively located at both ends of the line segment.
  • the starting point corresponding unit 1153 connects it with other starting points around, and the line segment connecting the two starting points is a connecting curve.
  • the two starting points with the smallest distance between each other are the starting points where the correspondence exists.
  • the starting point corresponding unit 1153 first connects the starting point with other starting points through a smooth Emilet curve, and the other starting points are different from the starting point. Highlight the curve, and the prominent curve and the protruding curve where the starting point is located are at the starting point of different image blocks, and then the integral of the scalar field on the Emmett curve is taken as the distance between the two starting points. Instead of simply applying the distance between the two points.
  • the starting point corresponding unit 1153 forms, as an element, a starting point corresponding to the extracted minimum distance as an element, and the formed related set contains a plurality of elements, and therefore, further extraction of the element is required.
  • the starting point corresponding unit 1153 compares the difference between the surrounding pixels and the pixels around the starting point, thereby retaining the element with the smallest difference, and deleting the remaining elements from the associated set.
  • the reserved element that is, the starting point has a corresponding relationship with the starting point
  • the two frame-selected protruding curves connected by the connecting curve are the associated protruding curves.
  • the surround configuration module 130 includes: a first extraction unit 131, The vector field solving unit 133, the loop unit 135, and the surround field forming unit 137.
  • the first extracting unit 131 is configured to extract the first longest curve integral from the whole curve of the image to be repaired, and give a direction to the whole of the first longest curve.
  • the curve of the image to be repaired is formed by a joint curve of the correlation and a connection curve connecting the mutually adjacent protruding curves.
  • the first extracting unit 131 extracts the longest curved whole from the plurality of curved wholes, and the extracted curved whole is the first longest curved whole, and gives a random direction to the entire first longest curved whole.
  • the vector field solving unit 133 is configured to solve the harmonic vector field by using the entire curve of the existing direction as the Dirichlet boundary condition.
  • the vector field solving unit 133 calculates the harmonic vector field as the Dirichlet boundary condition as a whole for the extracted first longest curve.
  • the circulation unit 135 is configured to extract the whole of the second longest curve as a whole by a pair of non-directional curves, and determine the overall direction of the second longest curve according to the harmonic vector field of the curve of the existing direction, until the entire non-directional curve is Give direction.
  • the direction of the entire first longest curve is given and the size of the harmonic vector field is solved accordingly.
  • the remaining curves are all non-directional, and therefore, the loop unit 135 will be paired one by one. The remaining curves determine the direction overall.
  • the loop unit 135 extracts the longest curve from the whole of the non-directional curve, and the extracted curve as a whole is the second longest curve as a whole, and determines the second most based on the entire adjusted vector field of the curve of the existing direction.
  • the overall direction of the long curve, and the overall extraction of the curve, and the determination of the direction, until all the curves are given the direction, the longest curve extracted each time is the second longest curve as a whole.
  • the loop unit 135 tests the degree of coincidence of the harmonic vector field of the whole curve of the second longest curve in the forward and reverse directions corresponding to the existing direction, and takes the direction with the highest degree of consistency as the second most The overall direction of the long curve, and calculate the harmonic vector field of the entire curve of the existing direction, and so on, until all the curves are given the direction as a whole.
  • the surrounding field forming unit 137 is configured to obtain a direction part of the surrounding place by solving the harmonic vector field of all the curves as a whole, and calculate according to the number of the protruding curves and the projection of the protruding curve. The size of the surrounding field.
  • the surrounding field forming unit 137 uses all the curves as the Dirichlet boundary condition to solve the harmonic vector field of all the curves as a whole to form a direction part of the surrounding field, and calculates the size of the surrounding field by the following formula. which is:
  • k is the number of associated pairs of salient curves
  • m(j) represents the index of the curve associated with the connecting curve j
  • ⁇ is the projection of r to the outwardly intimate arc at the starting point of the connecting curve j
  • ⁇ 2 is Control parameters, the default is 0.01.
  • the registration module 150 is further configured to obtain a transformation corresponding to the image block by minimizing the energy of the connection curve in the surrounding field, and obtain a transformation position and an angle of the connection curve according to the transformation corresponding to the image block.
  • the image block is registered according to the transformation position and angle of the connection curve.
  • the registration module 150 performs registration of the image blocks under the direction of the surrounding field, that is, the transformation of the image blocks is performed, so that the direction corresponding to all the connection curves is as uniform as the direction of the surrounding field. Since the transformation of the image block will affect the position and angle of the connection curve, which in turn determines the parameters of the connection curve, the connection curve used to connect the salient curve is essentially a function related to the transformation ⁇ of the image block, ie ⁇ ( ⁇ ), at this point, you can get the relevant optimization formula:
  • the registration module 150 selects the BFGS optimization algorithm to solve the above optimization formula to obtain the transformation position and angle of the connection curve, and then adjusts the position of the image block according to the transformation position and angle of the connection curve to obtain an image block.
  • the transformed image block is the registered image block.
  • the filling module 170 includes a structure filling unit 171. And texture synthesis unit 173.
  • the structure filling unit 171 is configured to perform structural filling on the registered image block by using a connection curve. In this embodiment, there is a gap between the registered image blocks, and the structure filling unit 171 is required to fill the registered image blocks to obtain a complete image. Specifically, the structure filling unit 171 applies structure filling between the image blocks to restore the image to be repaired according to the image structure, so that the image structure can be maintained as much as possible, and does not cause the filling as the conventional image filling method. The problem of blurred image structure of the area.
  • the above-described structure padding unit 171 includes a discretization unit 1711, a first subset extracting unit 1713, a second subset extracting unit 1715, and a merging unit 1717.
  • a discretization unit 1711 is configured to discretize the connection curve to obtain an ordered point set.
  • the first subset extracting unit 1713 is configured to extract a first subset from the ordered point set, the first subset including all points located on a gap between the image blocks.
  • the subset includes all the points on the gap in the ordered point set.
  • the second subset extracting unit 1715 is configured to extract, from the ordered point set, a second subset that has the smallest color difference from the first subset, and the length of the second subset is equivalent to the length of the first subset.
  • the process in which the second subset extracting unit 1715 extracts the second subset from the ordered point set is actually a process of searching for a pixel block similar to the gap in which the first subset is located along the entire curve.
  • the second subset extracting unit 1715 extracts a pixel strip in the whole curve including the connection curve, and uses the first subset as a target point set to deform the extracted pixel strip to obtain the deformed pixel strip.
  • the pixel strip is the smallest color difference from the pixels surrounding the first subset, that is, the pixels in the first subset in the image block, to facilitate seamless stitching of the image blocks.
  • the transformation performed by the second subset extraction unit 1715 on the extracted pixel strip, that is, the second subset may be a moving least squares deformation, or may be replaced by other algorithms, for example, a thin plate spline function method and a radial basis. Function method, etc.
  • the thin plate spline function method is an interpolation method for finding a smooth curved surface with the smallest curvature through all the control points.
  • the surface of the thin iron plate is made by a given number of "splines”. Smooth, and the minimum degree of bending is also defined by the energy function.
  • the second subset extracting unit 1715 performs the deformation of the second subset by the thin-plate spline function method, and regards the entire image to be repaired as a xoy plane in a three-dimensional space, and the displacement of the control point along the z direction And using the thin plate-like function interpolation to calculate the displacement of other points, thereby calculating the displacement of all the pixels in all the second subset to achieve image deformation.
  • the radial basis function method is mainly used to solve the interpolation problem. Since the radial basis function is only related to the distance to a certain point, it is called radial, and because the radial basis function is the basis used to approximate the function in the function space. , so called radial basis.
  • the second subset extracting unit 1715 regards the image to be repaired as the xoy plane of the three-dimensional space, and the displacement of the control point is along the difference z direction. At this time, the displacement of other points The change can be calculated by interpolation of the radial basis function, thereby calculating the displacement of all the pixels and realizing the image deformation.
  • the merging unit 1717 is configured to deform and copy the pixel strips of the second subset into the first subset, and fuse the pixels of the first subset with the surrounding pixels.
  • the merging unit 1717 seamlessly clones the pixel strip corresponding to the second subset to the gap where the first subset is located.
  • the merging unit 1717 adopts the Poisson fusion method. To make the fill more realistic and natural.
  • the texture synthesizing unit 173 is configured to perform texture synthesis on the image block after the structure is filled to obtain the restored image.
  • the texture synthesizing unit 173 performs texture synthesis on the image block in which the structure is filled to implement filling of other parts in the image to be repaired.
  • the texture synthesizing unit 173 can implement texture synthesis of the image to be repaired by using a content-aware filling method to obtain the restored image. Texture of image blocks through content-aware fill methods The synthesis is mainly to calculate the most similar blocks of all the image blocks containing it when calculating the recovery value for the missing pixels, and sum the color values corresponding to the known blocks to obtain the optimal value of the missing pixel.
  • texture synthesis unit 173 will also introduce an acceleration operation in the content-aware fill process to achieve a fast search of the closest known block by a combination of neighbor recommendation and random search.
  • the above image repairing method and device process the initially registered image block to obtain a connecting line between the image blocks, and construct a surrounding field of the image to be repaired by the connecting line, and minimize the energy pair in the surrounding field by connecting the curve
  • the image block is registered, and the image block after registration is image-filled to obtain a restored image. Since the registration of the image block does not require overlapping regions between the image blocks, and the surrounding curve is minimized by connecting the curves.
  • the energy approach achieves accurate registration based on numerical optimization, so it can be applied to the repair of any damaged image and improves the accuracy of the repair.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
  • ROM read-only memory
  • RAM random access memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供了一种图像修复方法和装置。所述方法包括：处理初始配准的图像块得到图像块之间的连接曲线；通过所述连接曲线构造待修复图像的环绕场；通过极小化连接曲线在环绕场中的能量对图像块进行配准；对所述配准后的图像块进行图像填充得到修复后的图像。所述装置包括：处理模块，用于处理初始配准的图像块得到图像块之间的连接曲线；环绕场构造模块，用于通过所述连接曲线构造待修复图像的环绕场；配准模块，用于通过极小化连接曲线在环绕场中的能量对图像块进行配准；填充模块，用于对所述配准后的图像块进行图像填充得到修复后的图像。采用本发明能够适用于任意一个损坏的图像的修复，且提高精确性。

Description

说明书
发明名称： 图像修复方法和装置 技术领域
本发明涉及图像处理技术， 特别是涉及一种图像修复方法和装置。 背景技术
图像修复指的是对受到损坏的图像进行修复重建， 使观察者看不出修复重 建所得到的图像曾被修复过。 传统的图像修复方法主要包括两类， 一类是基于 扩散的修复技术， 利用传播机制将信息从未受损区域传播到待修复的区域； 另 一类是基于纹理合成的方法， 在受损区域的周围寻找与受损区域边界像素较接 近的像素块， 然后将寻找得到的像素块在一定的结构约束下复制并抹去像素块 之间的缝隙。
传统的图像修复方法均是假设待修复的图像中各个图像块之间错位， 即各 个图像块之间存在着重叠区域。 然而， 在实际的图像修复过程中， 图像块之间 没有重叠区域的情况时有发生， 因此， 采用上述两种图像将无法适用于任意一 个损坏的图像的修复。
基于此， 又提出了一种图像修复方法， 在该方法中， 首先将没有重叠的图 像块向外插值， 使得这些图像块在外插区域重叠， 然后对外插之后的图像块进 行配准， 实现没有重叠区域的图像修复。
但是， 对图像块向外插值是一种极其不精确的手段， 通过外插区域进行配 准将进一步地扩大了误差， 并且， 这一图像修复方法仅允许图像块的形状是方 形的， 因此， 虽然能够实现没有重叠区域的图像修复， 但是， 也限制了该方法 的应用， 也不无保证图像修复的精确性。 发明内容
基于此， 有必要针对传统的图像修复方法无法适用于任意一个损坏的图像 的修复， 且精确性不高的问题， 提供一种能够适用于任意一个损坏的图像的修 复， 且提高精确性的图像修复方法。
此外， 还有必要提供一种能够适用于任意一个损坏的图像的修复， 且提高 精确性的图像修复系统。 一种图像修复方法， 包括如下步骤：
处理初始配准的图像块得到图像块之间的连接曲线；
通过所述连接曲线构造待修复图像的环绕场；
通过极小化连接曲线在环绕场中的能量对图像块进行配准；
对所述配准后的图像块进行图像填充得到修复后的图像。
在其中一个实施例中， 所述对所述初始配准的图像块进行处理得到图像块 之间的连接曲线的步骤为：
获取初始配准的图像块， 所述图像块来自待修复的图像；
沿所述初始配准的图像块边缘向里获取得到所述图像块中的突出曲线； 根据所述突出曲线得到相互关联的突出曲线， 并得到介于所述相互关联的 突出曲线之间的连接曲线。
在其中一个实施例中， 所述通过所述连接曲线构造待修复图像的环绕场的 步骤为：
从待修复图像的曲线整体中提取第一最长曲线整体， 为所述第一最长曲线 整体赋予方向；
以已有方向的曲线整体作为狄利克雷边界条件进行调和向量场的求解； 逐一对无方向的曲线整体提取第二最长曲线整体， 根据所述已有方向的曲 线整体的调和向量场确定所述第二最长曲线整体的方向， 直至无方向的曲线整 体均被 U武予方向；
通过求解所有曲线整体的调和向量场得到环绕场的方向部分， 并根据相互 关联的突出曲线数量以及突出曲线的投影计算得到环绕场的大小部分。
在其中一个实施例中， 所述通过极小化连接曲线在环绕场中的能量对图像 块进行配准的步骤为：
通过极小化连接曲线在环绕场中的能量求解得到图像块所对应的变换， 根 据所述图像块对应的变换得到连接曲线的变换位置和角度， 按照所述连接曲线 的变换位置和角度配准图像块。
在其中一个实施例中， 所述对所述配准后的图像块进行图像填充得到修复 后的图像的步骤为：
通过连接曲线对配准后的图像块进行结构填充；
对结构填充之后的图像块进行纹理合成得到修复后的图像。
在其中一个实施例中， 所述通过连接曲线对配准后的图像块进行结构填充 的步骤为：
将连接曲线离散化得到有序点集；
从所述有序点集中提取第一子集， 所述第一子集包括所有位于图像块之间 的间隙上的点；
从所述有序点集中提取与所述第一子集的颜色差最小的第二子集， 所述第 二子集的长度与所述第一子集的长度相当；
将所述第二子集的像素带变形并复制到第一子集中， 并将第一子集的像素 与周围像素融合。 一种图像修复装置， 包括：
处理模块， 用于处理初始配准的图像块得到图像块之间的连接曲线； 环绕场构造模块， 用于通过所述连接曲线构造待修复图像的环绕场； 配准模块， 用于通过极小化连接曲线在环绕场中的能量对图像块进行配准； 填充模块， 用于对所述配准后的图像块进行图像填充得到修复后的图像。 在其中一个实施例中， 所述处理模块包括：
图像块获取单元， 用于获取初始配准的图像块， 所述图像块来自待修复的 图像；
突出曲线获取单元， 用于沿所述初始配准的图像块边缘向里获取得到所述 图像块的突出曲线；
连接曲线获取单元， 用于根据所述突出曲线得到相互关联的突出曲线， 并 得到介于所述相互关联的突出曲线之间的连接曲线。 在其中一个实施例中， 所述环绕场构造模块包括：
第一提取单元， 用于从待修复图像的曲线整体中提取第一最长曲线整体， 为所述第一最长曲线整体赋予方向；
向量场求解单元， 用于以已有方向的曲线整体作为狄利克雷边界条件进行 调和向量场的求解；
循环单元， 用于逐一对无方向的曲线整体提取第二最长曲线整体， 根据所 述已有方向的曲线整体的调和向量场确定所述第二最长曲线整体的方向， 直至 无方向的曲线整体均被赋予方向；
环绕场形成单元， 用于通过求解所有曲线整体的调和向量场得到环绕场的 方向部分， 并根据相互关联的突出曲线数量以及突出曲线的投影计算得到环绕 场的大小部分。
在其中一个实施例中， 所述配准模块还用于通过极小化连接曲线在环绕场 中的能量求解得到图像块所对应的变换， 根据所述图像块对应的变换得到连接 曲线的变换位置和角度， 按照所述连接曲线的变换位置和角度配准图像块。
在其中一个实施例中， 所述填充模块包括：
结构填充单元， 用于通过连接曲线对配准后的图像块进行结构填充； 纹理合成单元， 用于对结构填充之后的图像块进行纹理合成得到修复后的 图像。
在其中一个实施例中， 所述结构填充单元包括：
离散化单元， 用于将连接曲线离散化得到有序点集；
第一子集提取单元， 用于从所述有序点集中提取第一子集， 所述第一子集 包括所有位于图像块之间的间隙上的点；
第二子集提取单元， 用于从所述有序点集中提取与所述第一子集的颜色差 最小的第二子集， 所述第二子集的长度与第一子集的长度相当；
融合单元， 用于将所述第二子集的像素带变形并复制到第一子集中， 并将 第一子集的像素与周围像素融合。
上述图像修复方法和装置， 对初始配准的图像块进行处理得到图像块之间 的连接线， 通过连接线构造得到待修复图像的环绕场， 通过极小化连接曲线在 环绕场中的能量对图像块进行配准， 进而对配准后的图像块进行图像填充得到 修复后的图像， 由于图像块的配准不需要图像块之间存在重叠区域， 并且通过 极小化连接曲线在环绕场的能量的方式在数值优化的基础上实现了精确配准， 所以能够适用于任意一个损坏的图像的修复， 且提高了修复的精确性。 附图说明
图 1为一个实施例中图像修复方法的流程图；
图 2为图 1 中处理初始配准的图像块得到图像块之间的连接曲线的方法流 程图；
图 3a为一个实施例中初始配准的图像块的示意图；
图 3b为图 3a中提取的突出曲线示意图；
图 3c为图 3b中连接曲线的示意图；
图 3d为通过图 3c构造的环绕场示意图；
图 3e为配准的图像块示意图；
图 3f为填充后的图像块示意图；
图 4为图 2中沿初始配准的图像块边缘向里获取得到图像块中的突出曲线 的方法流程图；
图 5a为一个实施例中相互关联的突出曲线示意图；
图 5b为图 5a中进行的距离计算示意图；
图 5c为图 5b中形成的关联集合示意图；
图 5d为通过图 5c的关联集合进行筛选所得到的相互关联的突出曲线示意 图；
图 6为一个实施例中通过鲁棒算法找出突出曲线中起始点的对应关系的方 法流程图；
图 7为图 1中通过连接曲线构造待修复图像的环绕场的方法流程图； 图 8a为一个实施例中配准之间的图像块示意图；
图 8b为与图 8a对应的配准后的图像块示意图；
图 9为图 1 中对配准后的图像块进行图像填充得到修复后的图像的方法流 程图；
图 10 为图 9 中通过连接曲线对配准后的图像块进行结构填充的方法流程 图；
图 11为图 10中结构填充示意图；
图 12为一个实施例中图像修复装置的结构示意图；
图 13为图 12中处理模块的结构示意图；
图 14为图 13中突出曲线获取单元的结构示意图；
图 15为图 13中连接曲线获取模块的结构示意图；
图 16为图 12中环绕场构造模块的结构示意图；
图 17为图 12中填充模块的结构示意图；
图 18为图 17中结构填充单元的结构示意图。 具体实施方式
如图 1所示， 在一个实施例中， 一种图像修复方法， 包括如下步骤： 步骤 S110, 处理初始配准的图像块得到图像块之间的连接曲线。
本实施例中， 初始配准的图像块是来自于待修复图像的图像碎片， 这些初 始配准的图像块即形成了待修复的图像。 具体的， 初始配准的图像块是用户对 图像块所进行的大致摆布。 对初始配准的图像块进行处理所得到的连接曲线介 于图像块之间所存在的间隙中， 用于相联关系处于破碎状态的图像块。
步骤 S130, 通过连接曲线构造待修复图像的环绕场。
本实施例中， 环绕场包括大小部分和方向部分， 可用于标记待修复图像中 曲线的能量大小及其方向。
步骤 S150, 通过极小化连接曲线在环绕场中的能量对图像块进行配准。 本实施例中， 对一定的优化算法来极小化连接曲线在环绕场中的能量， 进 而在这一数值优化的基础上得以保证连接曲线在环绕场中的能量至少收敛到局 部最优结果， 通过极小化后的连接曲线实现图像块的配准， 进而使得图像块的 配准不会发生偏离， 提高待修复图像中配准的精确性。
配准图像块所应用的优化算法可以是最速下降法、 牛顿法以及 BFGS算法， 在此， 不——进行列举。
例如， 最速下降法又称为梯度法， 是在求解极小化连接曲线在环绕场中的 能量这一无约束化问题时， 简单沿着梯度的负方向来搜索最优值； 牛顿法是利 用目标函数的梯度和海塞矩阵所构成的二次函数来寻找极值， 每一步之后还将 再次计算新的二次函数来进行寻优。
步骤 S170, 对配准后的图像块进行图像填充得到修复后的图像。
本实施例中， 对配准后的图像块之间存在的间隙进行填充， 以使得若干个 图像块形成完整的图像。
上述图像修复方法， 可对二维的破碎图像进行修复， 也可对三维的破碎图 像进行修复， 在此不再赘述。
如图 2所示， 在一个实施例中， 上述步骤 S110包括：
步骤 S111 , 获取初始配准的图像块， 该图像块来自待修复的图像。
本实施例中， 破碎的图像中的若干个图像块通过用户触发的操作进行粗糙 的初始配准， 以得到初始配准的图像块， 如图 3a所示。
步骤 S113 , 沿初始配准的图像块边缘向里获取得到图像块中的突出曲线。 本实施例中， 通过一定的算法或交互的手段进行图像块中线段， 即突出曲 线的提取， 例如， 由用户交互输入的智能剪刀方法或者计算图像块中突出曲线 的方法。 其中， 智能剪刀法是获取用户在图像上点集的一系列种子点， 通过在 相邻的种子点之间求取能量最小路径来自动连接种子点， 以最终得到相应的曲 线， 该曲线即为突出曲线， 如图 3b所示。
此外， 在优选的实施例中， 将通过保边滤波、 求偏导、 边界提取以及对边 界的处理等算法计算得到突出曲线。
如图 4所示， 在一个实施例中， 上述步骤 S113包括：
步骤 S1131 , 对初始配准的图像块进行保边滤波， 并求偏导得到每一图像块 对应的梯度图。
本实施例中， 用以实现保边滤波的算法可以是加权最小二乘滤波、 各向异 性扩散、 鲁棒光滑和双边滤波等， 在此不一一进行列举。
通过上述任一算法对经过初始配准的每一个图像块进行保边滤波， 并通过 保边滤波后的图像块求偏异得到各个图像块所对应的梯度图。
步骤 S 1133 , 标记图像块的边界，根据图像块对应的梯度图得到标记的边界 上的梯度峰值点。
本实施例中， 对每一图像块进行边界提取， 并标记所提取的边界。 具体的， 将假设经过初始配准的图像块是处于白色背景之下的， 此时， 通过应用一定的 边界提取算法即可实现图像块边界的提取。 例如， 所应用的边界提取算法可以 是较为简单的二值分割法或者动态轮廓法。
步骤 S 1135 , 以边界上的梯度峰值点为起点进行曲线探测得到相应的曲线， 该曲线形成待修复图像的曲线集合。
本实施例中， 沿着图像块的边界找出梯度的峰值点， 进而以这些峰值点作 为起点， 沿着梯度值最大的方向向图像块内部探测若干步得到相应的曲线， 此 时， 所得到的曲线为多个， 以形成待修复图像所对应的曲线集合。
步骤 S 1137 , 从曲线集合中提取突出曲线。
本实施例中， 计算曲线集合中每一曲线所对应的评分， 即 score(s) = len(s) +grad(s) +orth(s)-curv(s) , 其中， /6«(¾)表示曲线长度， gra< (¾)表示曲线上的 所有像素点的梯度值之和， w /z^)表示曲线在其起始点出与边界切线的夹角的正 弦值， CMrvf^)表示曲线上的平均曲率， 并且上述四项都要除以相应的最大值以保 证取值自己的取值范围为 [0, 1]
通过计算得到的评分对曲线集合中的曲线进行过滤， 以得到突出曲线。 具体的， 在计算得到曲线集合中每一曲线所对应的评分之后， 将判断计算 得到的评分是否大于阈值， 若否， 则舍弃相应的曲线， 若是则保留相应的曲线， 而最终保留的曲线即为突出曲线。 一实施例中， 该阈值可设置为 1.0。
步骤 S 115 , 根据突出曲线得到相互关联的突出曲线， 并得到介于相互关联 的突出曲线之间的连接曲线。
本实施例中， 相互关联的突出曲线是处于不同图像块上起始点之间存在着 对应关系的突出曲线， 用于连接两条相互关联的突出曲线的曲线即为连接曲线， 并且， 连接曲线也是连接两个图像块的曲线， 处于图像块之间的间隙上， 如图 3c所示。 此时， 通过上述步骤 S130所构造得到的待修复图像的环绕场如图 3d所示， 通过上述步骤 S 150极小化连接曲线在环绕场中的能量以配准得到的图像块如图 3e所示， 进而通过相应的填充即可得到如图 3f所示的完整图像， 实现了无重叠 区域也能够真实地恢复破碎的图像， 提高了图像修复的精确性。
在一个实施例中， 上述步骤 S115包括：
通过鲁棒算法找出突出曲线中起始点的对应关系， 存在对应关系的起始点 位于不同突出曲线上， 存在对应关系的起始点所在的突出曲线分别位于不同的 图像块上。
本实施例中， 上述存在对应关系的起始点所在的突出曲线即为相互关联的 突出曲线， 介于相互关联的突出曲线之间的曲线即为连接曲线。 在优选的实施 例中， 连接曲线为埃米尔特曲线， 如图 5a所框选的突出曲线即为相互关联的突 出曲线。
对待修复图像的若干个突出曲线进行配对， 以得到存在着对应关系的起始 点， 而存在着对应关系的起始点所在的突出曲线即为相互关联的突出曲线， 进 而通过光滑的埃米尔特曲线连接两条相互关联的突出曲线， 该埃米尔特曲线即 为连接曲线， 连接曲线穿过图像块之间的间隙连接两个图像块。
如图 6所示， 在一个实施例中， 上述通过鲁棒算法找出突出曲线中起始点 的对应关系的步骤包括：
步骤 S 1151 , 根据突出曲线构建待修复图像的标量场。
本实施例中， 待修复图像的标量场可通过如下公式构建：
Figure imgf000011_0001
其中， n为突出曲线的数量， r为空间的任一点， ^为曲线 j的起始点, 为 r在曲线 j在起点处的向外密切圓弧的投影， σι为控制参数， 默认取 0.05。
步骤 S1153 , 对任意突出曲线的起始点， 将任意起始点与其它起始点通过连 接曲线相互连接， 求标量场在连接曲线上的积分作为距离， 以最小距离所对应 的起始点作为元素形成关联集合， 根据元素所对应的周围像素与任意起始点周 围像素的差异得到任意突出曲线中起始点的对应关系。 本实施例中， 突出曲线包括了分别位于线段两端的起始点， 对于任意一起 始点， 将其与周围的其它起始点相连接， 连接两个起始点的线段为连接曲线， 此时， 相互之间距离最小的两个起始点即为存在着对应关系的起始点。
进一步的， 为保证对应关系的准确获取， 对于任一起始点， 首先通过光滑 的埃米尔特曲线连接该起始点与其它的起始点， 其它的起始点是与该起始点处 于不同突出曲线， 且所在的突出曲线与该起始点所在的突出曲线是处于不同图 像块的起始点， 进而求取标量场在这一埃米特曲线上的积分作为这两个起始点 之间的距离， 而不简单地应用两点之间的距离。
如图 5b所示， 在标量场中连接任一起始点与到它距离最小的另一起始点以 进行距离的计算。
此时， 将提取最小距离所对应的起始点作为元素形成关联集合， 其中， 如 图 5c所示， 所形成的关联集合包含的元素为若干个， 因此， 还需要进一步进行 元素的提取。
具体的， 关联集合中每一元素所对应的周围像素， 比对周围像素与该起始 点周围像素的差异， 进而保留差异最小的元素， 将其余的元素从关联集合中删 除， 此时， 所保留的元素， 即起始点是与该起始点存在着对应关系的， 如图 5d 所示， 被连接曲线所连接的两个框选的突出曲线即为经关联的突出曲线。
如 7所示， 在一个实施例中， 上述步骤 S130包括：
步骤 S131 , 从待修复图像的曲线整体中提取第一最长曲线整体， 为第一最 长曲线整体赋予方向。
本实施例中， 待修复图像的曲线整体是相互关联的突出曲线和连接相互关 联的突出曲线的连接曲线所形成的。 从多个曲线整体中提取最长的曲线整体， 该提取的曲线整体即为第一最长曲线整体， 并为这一第一最长曲线整体赋予一 个随机方向。
步骤 S133 , 以已有方向的曲线整体作为狄利克雷边界条件进行调和向量场 的求解。
本实施例中， 将提取的第一最长曲线整体为作狄利克雷边界条件求解调和 向量场。 步骤 S135 , 逐一对无方向的曲线整体提取第二最长曲线整体， 根据已有方 向的曲线整体的调和向量场确定第二最长曲线整体的方向， 直至无方向的曲线 整体均被赋予方向。
本实施例中， 在对提取的第一最长曲线整体赋予了方向并相应求解了调和 向量场的大小部分， 此时， 余下的曲线整体均是无方向的， 因此， 将逐一对余 下的曲线整体确定方向。
具体的， 从无方向的曲线整体中提取最长的曲线整体， 该提取的曲线整体 即为第二最长曲线整体， 根据已有方向的曲线整体的调和向量场来确定第二最 长曲线整体的方向， 并再次进行曲线整体的提取， 以及方向的确定， 直至所有 的曲线整体均被赋予方向为止， 每次所提取的最长的曲线整体即为第二最长曲 线整体。
进一步的， 根据已有方向的曲线整体的调和向量场确定第二最长曲线整体 的方向的具体过程为： 测试第二最长曲线整体在延伸方向上对应的正反两种方 向上与已有方向的曲线整体的调和向量场的一致程度， 取一致程度最高的方向 作为第二最长曲线整体的方向， 并计算已有方向的曲线整体的调和向量场， 以 此类推， 直至所有的曲线整体均被赋予方向。
步骤 S137 , 通过求解所有曲线整体的调和向量场得到环绕场的方向部分， 并根据相互关联的突出曲线数量以及突出曲线的投影计算得到环绕场的大小部 分。
本实施例中， 以所有曲线整体作为狄利克雷边界条件， 求解所有曲线整体 的调和向量场， 以构成环绕场的方向部分， 并通过如下公式计算得到环绕场的 大小部分， 即：
Figure imgf000013_0001
其中， k为相关联的突出曲线对的数量， m(j)表示与连接曲线 j相关联的曲 线索引， ^为 r在连接曲线 j在起点处的向外密切圓弧的投影， σ2为控制参数， 默认取 0.01。
在一个实施例中， 上述步骤 S150的具体过程为： 通过极小化连接曲线在环绕场中的能量求解得到图像块所对应的变换， 根 据图像块对应的变换得到连接曲线的变换位置和角度， 按照连接曲线的变换位 置和角度配准图像块。
本实施例中，在环绕场的指示下进行图像块的配准，即求对图像块的变换 T , 进而使得所有的连接曲线对应的方向与环绕场的方向尽可能一致。 由于图像块 的变换将会影响连接曲线所在位置和角度， 进而决定着连接曲线的参数， 因此， 用于连接突出曲线的连接曲线本质上是与图像块的变换 T相关的函数，即 H(T), 此时， 可得到相关的优化公式：
T = arg minr E(H(T))
Figure imgf000014_0001
f(r) = \ - t(r)T V(r) + l(\ - d(r)) 其中， /^为 H的第 i个元素， t(r)为/^在点 r处的切线方向。 为调节两项权 重的参数， 默认取 0.1 , v(r)为点 r处的向量方向， d(r)为点 r处的向量大小。
在优选的实施例中， 选用 BFGS优化算法来求解上述优化公式， 以得到连 接曲线的变换位置和角度， 进而按照连接曲线的变换位置和角度对图像块进行 位置调整， 得到图像块的变换， 变换后的图像块即为配准了的图像块。
如图 8a所示， 通过优化算法所求解得到的变换对图像块进行配准之后， 即 可得到如图 8b所示的配准的图像块。
如图 9所示， 在一个实施例中， 上述步骤 S170包括：
步骤 S171 , 通过连接曲线对配准后的图像块进行结构填充。
本实施例中， 配准后的图像块之间存在着间隙， 需要对配准后的图像块进 行填充方可得到完整的图像。 具体的， 应用图像块之间的连接线进行结构填充， 以依据图像结构还原待修复的图像， 进而使图像结构能最大可能的保持， 不会 如传统的图像填充方法一样导致填充区域的图像结构模糊的问题。
如图 10所示， 在一个实施例中， 上述步骤 S171包括：
步骤 S1711 , 将连接曲线离散化得到有序点集。
本实施例中， 对图像块之间的连接曲线进行离散化， 以得到有序点集 P = {Ρ1,Ρ2,· ..,Ρϋ}。
步骤 S1713 ,从有序点集中提取第一子集， 第一子集包括所有位于图像块之 间的间隙上的点。
本实施例中，在有序点集中选取子集，以得到第一子集 P，= { Pi,Pi+l, Pi+2.., Pi+m } , 第一子集包括了有序点集中所有位于间隙上的点， 如图 11所示。
步骤 S1715 ,从有序点集中提取与第一子集的颜色差最小的第二子集， 第二 子集的长度与第一子集的长度相当。
本实施例中， 再次在有序点集中提取子集， 以得到与第一子集的颜色差最 小的第二子集， 即如图 11所示的 P"={Pk, Pk+l, ..., Pk+m }。
从有序点集中提取第二子集的过程实际上是沿着曲线整体搜索与第一子集 所在的间隙相似的像素块的过程。
具体的， 在包含了连接曲线的曲线整体中提取一条像素带， 以第一子集为 目标点集合， 对提取的像素带进行变形， 以得到变形后的像素带， 该像素带是 与第一子集周围像素， 即第一子集中处于图像块中的像素之间的颜色差最小的， 以便于对图像块进行无缝拼接。
其中， 对提取的像素带， 即第二子集所进行的变换可以是移动最小二乘变 形， 也可以采用其它算法替代， 例如， 薄板样条函数法和径向基函数法等。
薄板样条函数法是一种插值方法， 用于寻找一个通过所有的控制点的弯曲 最小的光滑曲面， 作为薄铁板， 通过所给定的几个 "样条" 使得薄铁板的表面 是光滑的， 而弯曲最小的程度也是由能量函数定义的。
具体的， 通过薄板样条函数法对第二子集所进行的变形是将整个待修复的 图像看作一个三维空间中的 xoy平面， 控制点的位移沿着 z方向， 并应用薄板 样函数插值计算出其它点的位移， 从而计算出所有第二子集中所有像素的位移， 以实现图像变形。
径向基函数法主要用于解决插值问题， 由于径向基函数只与到某个点的距 离有关， 因此称为径向， 而又因为径向基函数是函数空间中用来逼近函数的基， 所以称为径向基。
在应用径向基函数法对第二子集变形时，把待修复图像视为三维空间的 xoy 平面， 控制点的位移沿着 Z方向， 此时， 其它点的位移变可以用径向基函数插 值计算出来， 从而计算得到所有像素的位移， 实现图像变形。
步骤 S1717,将第二子集的像素带变形并复制到第一子集中， 并将第一子集 的像素与周围像素融合。
本实施例中， 将第二子集所对应的像素带进行变形之后无缝克隆到第一子 集所在的间隙上， 在优选的实施例中， 将采用泊松融合法， 以使得填充更为逼 真、 自然。
步骤 S173 , 对结构填充之后的图像块进行纹理合成得到修复后的图像。 本实施例中， 对完成了结构填充的图像块进行纹理合成， 以实现待修复图 像中其它部分的填充。
具体的， 可采用内容感知的填充方法实现待修复图像的纹理合成， 以得到 修复后的图像。 通过内容感知的填充方法实现图像块的纹理合成主要是为丟失 像素计算恢复值时计算所有包含它的图像块的最相近已知块， 并通过组合这些 已知块所对应的颜色值求和得到这一个丟失像素的最佳值。
在优选的实施例中， 还将在内容感知的填充过程中引入加速操作， 以通过 邻居推荐和随机搜索组合来实现最相近已知块的快速搜索。 如图 12所示， 在一个实施例中， 一种图像修复装置， 包括处理模块 110、 环绕场构造模块 130、 配准模块 150和填充模块 170。
处理模块 110, 用于处理初始配准的图像块得到图像块之间的连接曲线。 本实施例中， 初始配准的图像块是来自于待修复图像的图像碎片， 这些初 始配准的图像块即形成了待修复的图像。 具体的， 初始配准的图像块是用户对 图像块所进行的大致摆布。 对初始配准的图像块进行处理所得到的连接曲线介 于图像块之间所存在的间隙中， 用于相联关系处于破碎状态的图像块。
环绕场构造模块 130, 用于通过连接曲线构造得到待修复图像的环绕场。 本实施例中， 环绕场包括大小部分和方向部分， 可用于标记待修复图像中 曲线的能量大小及其方向。
配准模块 150,用于通过极小化连接曲线在环绕场中的能量对图像块进行配 准。
本实施例中， 配准模块 150对一定的优化算法来极小化连接曲线在环绕场 中的能量， 进而在这一数值优化的基础上得以保证连接曲线在环绕场中的能量 至少收敛到局部最优结果， 通过极小化后的连接曲线实现图像块的配准， 进而 使得图像块的配准不会发生偏离， 提高待修复图像中配准的精确性。
配准模块 150 配准图像块所应用的优化算法可以是最速下降法、 牛顿法以 及 BFGS算法， 在此， 不——进行列举。
例如， 最速下降法又称为梯度法， 是在求解极小化连接曲线在环绕场中的 能量这一无约束化问题时， 简单沿着梯度的负方向来搜索最优值； 牛顿法是利 用目标函数的梯度和海塞矩阵所构成的二次函数来寻找极值， 每一步之后还将 再次计算新的二次函数来进行寻优。
填充模块 170, 用于对配准后的图像块进行图像填充得到修复后的图像。 本实施例中， 填充模块 170对配准后的图像块之间存在的间隙进行填充， 以使得若干个图像块形成完整的图像。
上述图像修复装置， 可对二维的破碎图像进行修复， 也可对三维的破碎图 像进行修复， 在此不再赘述。
如图 13所示， 在一个实施例中， 上述处理模块 110包括： 图像块获取单元 111、 突出曲线获取单元 113和连接曲线获取模块 115。
图像块获取单元 111 , 用于获取初始配准的图像块， 该图像块来自待修复的 图像。
本实施例中， 破碎的图像中的若干个图像块通过用户触发的操作进行粗糙 的初始配准， 以得到初始配准的图像块。
突出曲线获取单元 113 ,用于沿初始配准的图像块边缘向里获取得到图像块 的突出曲线。
本实施例中， 突出曲线获取单元 113 通过一定的算法或交互的手段进行图 像块中线段， 即突出曲线的提取， 例如， 由用户交互输入的智能剪刀方法或者 计算图像块中突出曲线的方法。 其中， 智能剪刀法是获取用户在图像上点集的 一系列种子点， 通过在相邻的种子点之间求取能量最小路径来自动连接种子点， 以最终得到相应的曲线， 该曲线即为突出曲线。
此外， 在优选的实施例中， 突出曲线获取单元 113 将通过保边滤波、 求偏 导、 边界提取以及对边界的处理等算法计算得到突出曲线。
如图 14所示， 在一个实施例中， 上述突出曲线获取单元 113包括滤波求偏 导单元 1131、 边界处理单元 1133、 曲线探测单元 1135和曲线提取单元 1137。
滤波求偏导单元 1131 , 用于对初始配准的图像块进行保边滤波， 并求偏导 得到每一图像块的梯度图。
本实施例中， 滤波求偏导单元 1131用以实现保边滤波的算法可以是加权最 小二乘滤波、 各向异性扩散、 鲁棒光滑和双边滤波等， 在此不一一进行列举。
滤波求偏导单元 1131通过上述任一算法对经过初始配准的每一个图像块进 行保边滤波， 并通过保边滤波后的图像块求偏异得到各个图像块所对应的梯度 图。
边界处理单元 1133 , 用于标记图像块的边界， 根据图像块对应的梯度图得 到标记的边界上的梯度峰值点。
本实施例中， 边界处理单元 1133对每一图像块进行边界提取， 并标记所提 取的边界。 具体的， 边界处理单元 1133将假设经过初始配准的图像块是处于白 色背景之下的， 此时， 通过应用一定的边界提取算法即可实现图像块边界的提 取。 例如， 边界处理单元 1133所应用的边界提取算法可以是较为简单的二值分 割法或者动态轮廓法。
曲线探测单元 1135 , 用于以边界上的梯度峰值点为起点进行曲线探测得到 相应的曲线， 该曲线形成待修复图像的曲线集合。
本实施例中， 曲线探测单元 1135沿着图像块的边界找出梯度的峰值点， 进 而以这些峰值点作为起点， 沿着梯度值最大的方向向图像块内部探测若干步得 到相应的曲线， 此时， 所得到的曲线为多个， 以形成待修复图像所对应的曲线 集合。
曲线提取单元 1137, 用于从曲线集合中提取突出曲线。
本实施例中， 曲线提取单元 1137计算曲线集合中每一曲线所对应的评分， score(s) = len(s)+grad(s)+orth(s)-curv(s) , 其中, 表示曲线长度, grad(s) 表示曲线上的所有像素点的梯度值之和， orth(s) 示曲线在其起始点出与边界切 线的夹角的正弦值， CMrvf^)表示曲线上的平均曲率， 并且上述四项都要除以相应 的最大值以保证取值自己的取值范围为 [0, 1 ]
曲线提取单元 1137通过计算得到的评分对曲线集合中的曲线进行过滤， 以 得到突出曲线。
具体的， 在计算得到曲线集合中每一曲线所对应的评分之后， 曲线提取单 元 1137将判断计算得到的评分是否大于阈值， 若否， 则舍弃相应的曲线， 若是 则保留相应的曲线， 而最终保留的曲线即为突出曲线。 一实施例中， 该阈值可 设置为 1.0。
连接曲线获取模块 115 , 用于根据突出曲线得到相互关联的突出曲线， 并得 到介于相互关联的突出曲线之间的连接曲线。
本实施例中， 相互关联的突出曲线是处于不同图像块上起始点之间存在着 对应关系的突出曲线， 用于连接两条相互关联的突出曲线的曲线即为连接曲线， 并且， 连接曲线也是连接两个图像块的曲线， 处于图像块之间的间隙上。
在一个实施例中， 上述连接曲线获取模块 115还用于通过鲁棒算法找出突 出曲线中起始点的对应关系， 存在对应关系的起始点位于不同突出曲线上， 存 在对应关系的起始点所在的突出曲线分别位于不同的图像块上。
本实施例中， 上述存在对应关系的起始点所在的突出曲线即为相互关联的 突出曲线， 介于相互关联的突出曲线之间的曲线即为连接曲线。
在优选的实施例中， 上述连接曲线为埃米尔特曲线。
连接曲线获取模块 115对待修复图像的若干个突出曲线进行配对， 以得到 存在着对应关系的起始点， 而存在着对应关系的起始点所在的突出曲线即为相 互关联的突出曲线， 进而通过光滑的埃米尔特曲线连接两条相互关联的突出曲 线， 该埃米尔特曲线即为连接曲线， 连接曲线穿过图像块之间的间隙连接两个 图像块。
如图 15所示， 在一个实施例中， 上述连接曲线获取模块 115包括标量场构 建单元 1151和起始点对应单元 1153。
标量场构建单元 1151 , 用于根据突出曲线构建待修复图像的标量场。 本实施例中， 标量场构建单元 1151可通过如下公式构建待修复图像的标量 场：
Figure imgf000020_0001
其中， n为突出曲线的数量， r为空间的任一点， ^为曲线 j的起始点, ^为 r在曲线 j在起点处的向外密切圓弧的投影， σι为控制参数， 默认取 0.05。
起始点对应单元 1153 , 用于对任意突出曲线的起始点， 将任意起始点与其 它起始点通过连接曲线相互连接， 求标量场在连接曲线上的积分作为距离， 以 最小距离所对应的起始点作为元素形成关联集合， 根据元素所对应的周围像素 与任意起始点周围像素的差异作为元素形成关联集合， 根据元素所对应的周围 像素与任意起始点周围像素的差异得到任意突出曲线中起始点的对应关系。
本实施例中， 突出曲线包括了分别位于线段两端的起始点， 对于任意一起 始点， 起始点对应单元 1153将其与周围的其它起始点相连接， 连接两个起始点 的线段为连接曲线， 此时， 相互之间距离最小的两个起始点即为存在着对应关 系的起始点。
进一步的， 为保证对应关系的准确获取， 对于任一起始点， 起始点对应单 元 1153首先通过光滑的埃米尔特曲线连接该起始点与其它的起始点， 其它的起 始点是与该起始点处于不同突出曲线， 且所在的突出曲线与该起始点所在的突 出曲线是处于不同图像块的起始点， 进而求取标量场在这一埃米特曲线上的积 分作为这两个起始点之间的距离， 而不简单地应用两点之间的距离。
此时， 起始点对应单元 1153将提取最小距离所对应的起始点作为元素形成 关联集合， 其中， 所形成的关联集合包含的元素为若干个， 因此， 还需要进一 步进行元素的提取。
具体的， 关联集合中每一元素所对应的周围像素， 起始点对应单元 1153比 对周围像素与该起始点周围像素的差异， 进而保留差异最小的元素， 将其余的 元素从关联集合中删除， 此时， 所保留的元素， 即起始点是与该起始点存在着 对应关系的， 被连接曲线所连接的两个框选的突出曲线即为经关联的突出曲线。 如图 16所示，在一个实施例中，上述环绕构造模块 130包括：第一提取单元 131、 向量场求解单元 133、 循环单元 135和环绕场形成单元 137。
第一提取单元 131 , 用于从待修复图像的曲线整体中提取第一最长曲线整 体， 为第一最长曲线整体赋予方向。
本实施例中， 待修复图像的曲线整体是相互关联的突出曲线和连接相互关 联的突出曲线的连接曲线所形成的。 第一提取单元 131 从多个曲线整体中提取 最长的曲线整体， 该提取的曲线整体即为第一最长曲线整体， 并为这一第一最 长曲线整体赋予一个随机方向。
向量场求解单元 133 ,用于以已有方向的曲线整体作为狄利克雷边界条件进 行调和向量场的求解。
本实施例中， 向量场求解单元 133 将提取的第一最长曲线整体为作狄利克 雷边界条件求解调和向量场。
循环单元 135 , 用于逐一对无方向的曲线整体提取第二最长曲线整体，根据 已有方向的曲线整体的调和向量场确定第二最长曲线整体的方向， 直至无方向 的曲线整体均被赋予方向。
本实施例中， 在对提取的第一最长曲线整体赋予了方向并相应求解了调和 向量场的大小部分， 此时， 余下的曲线整体均是无方向的， 因此， 循环单元 135 将逐一对余下的曲线整体确定方向。
具体的， 循环单元 135 从无方向的曲线整体中提取最长的曲线整体， 该提 取的曲线整体即为第二最长曲线整体， 根据已有方向的曲线整体的调和向量场 来确定第二最长曲线整体的方向， 并再次进行曲线整体的提取， 以及方向的确 定， 直至所有的曲线整体均被赋予方向为止， 每次所提取的最长的曲线整体即 为第二最长曲线整体。
进一步的， 循环单元 135 测试第二最长曲线整体在延伸方向上对应的正反 两种方向上与已有方向的曲线整体的调和向量场的一致程度， 取一致程度最高 的方向作为第二最长曲线整体的方向， 并计算已有方向的曲线整体的调和向量 场， 以此类推， 直至所有的曲线整体均被赋予方向。
环绕场形成单元 137,用于通过求解所有曲线整体的调和向量场得到环绕场 所的方向部分， 并根据相互关联的突出曲线数量以及突出曲线的投影计算得到 环绕场的大小部分。
本实施例中， 环绕场形成单元 137以所有曲线整体作为狄利克雷边界条件， 求解所有曲线整体的调和向量场， 以构成环绕场的方向部分， 并通过如下公式 计算得到环绕场的大小部分， 即：
Figure imgf000022_0001
其中， k为相关联的突出曲线对的数量， m(j)表示与连接曲线 j相关联的曲 线索引， ^为 r在连接曲线 j在起点处的向外密切圓弧的投影， σ2为控制参数， 默认取 0.01。
在一个实施例中， 上述配准模块 150还用于通过极小化连接曲线在环绕场 中的能量求解得到图像块所对应的变换， 根据图像块对应的变换得到连接曲线 的变换位置和角度， 按照连接曲线的变换位置和角度配准图像块。
本实施例中， 配准模块 150在环绕场的指示下进行图像块的配准， 即求对 图像块的变换 Τ,进而使得所有的连接曲线对应的方向与环绕场的方向尽可能一 致。 由于图像块的变换将会影响连接曲线所在位置和角度， 进而决定着连接曲 线的参数， 因此， 用于连接突出曲线的连接曲线本质上是与图像块的变换 Τ相 关的函数， 即 Η(Τ), 此时， 可得到相关的优化公式：
Τ = arg minr E(H(T))
E(H(T)) ^ l (T)f(r)dr f(r) = \ - t(r)T V(r) + l(\ - d(r)) 其中， /;为11的第 i个元素， t(r)为 /;在点 r处的切线方向。 A为调节两项权 重的参数， 默认取 0.1 , v(r)为点 r处的向量方向。
在优选的实施例中， 配准模块 150选用 BFGS优化算法来求解上述优化公 式， 以得到连接曲线的变换位置和角度， 进而按照连接曲线的变换位置和角度 对图像块进行位置调整， 得到图像块的变换， 变换后的图像块即为配准了的图 像块。
如图 17所示， 在一个实施例中， 上述填充模块 170包括结构填充单元 171 和纹理合成单元 173。
结构填充单元 171 , 用于通过连接曲线对配准后的图像块进行结构填充。 本实施例中， 配准后的图像块之间存在着间隙， 需要结构填充单元 171 对 配准后的图像块进行填充方可得到完整的图像。 具体的， 结构填充单元 171 应 用图像块之间的连接线进行结构填充， 以依据图像结构还原待修复的图像， 进 而使图像结构能最大可能的保持， 不会如传统的图像填充方法一样导致填充区 域的图像结构模糊的问题。
如图 18 所示， 在一个实施例中， 上述结构填充单元 171 包括离散化单元 1711、 第一子集提取单元 1713、 第二子集提取单元 1715和融合单元 1717。
离散化单元 1711 , 用于将连接曲线离散化得到有序点集。
本实施例中， 离散化单元 1711对图像块之间的连接曲线进行离散化， 以得 到有序点集 Ρ = {Ρ1,Ρ2,· ..,Ρη}。
第一子集提取单元 1713 , 用于从有序点集中提取第一子集， 所述第一子集 包括所有位于图像块之间的间隙上的点。
本实施例中， 第一子集提取单元 1713在有序点集中选取子集， 以得到第一 子集 P， = { Pi,Pi+l, Pi+2.., Pi+m } , 第一子集包括了有序点集中所有位于间隙上 的点。
第二子集提取单元 1715, 用于从有序点集中提取与第一子集的颜色差最小 的第二子集， 所述第二子集的长度与第一子集的长度相当。
本实施例中， 第二子集提取单元 1715再次在有序点集中提取子集， 以得到 与第一子集的颜色差最小的第二子集， 即如图 10 所示的 P"={Pk, Pk+1, · .., Pk+m }。
第二子集提取单元 1715从有序点集中提取第二子集的过程实际上是沿着曲 线整体搜索与第一子集所在的间隙相似的像素块的过程。
具体的， 第二子集提取单元 1715在包含了连接曲线的曲线整体中提取一条 像素带， 以第一子集为目标点集合， 对提取的像素带进行变形， 以得到变形后 的像素带， 该像素带是与第一子集周围像素， 即第一子集中处于图像块中的像 素之间的颜色差最小的， 以便于对图像块进行无缝拼接。 其中， 第二子集提取单元 1715对提取的像素带， 即第二子集所进行的变换 可以是移动最小二乘变形， 也可以采用其它算法替代， 例如， 薄板样条函数法 和径向基函数法等。
薄板样条函数法是一种插值方法， 用于寻找一个通过所有的控制点的弯曲 最小的光滑曲面， 作为薄铁板， 通过所给定的几个 "样条" 使得薄铁板的表面 是光滑的， 而弯曲最小的程度也是由能量函数定义的。
具体的， 第二子集提取单元 1715通过薄板样条函数法对第二子集所进行的 变形是将整个待修复的图像看作一个三维空间中的 xoy平面， 控制点的位移沿 着 z方向， 并应用薄板样函数插值计算出其它点的位移， 从而计算出所有第二 子集中所有像素的位移， 以实现图像变形。
径向基函数法主要用于解决插值问题， 由于径向基函数只与到某个点的距 离有关， 因此称为径向， 而又因为径向基函数是函数空间中用来逼近函数的基， 所以称为径向基。
在应用径向基函数法对第二子集变形时， 第二子集提取单元 1715把待修复 图像视为三维空间的 xoy平面， 控制点的位移沿差 z方向， 此时， 其它点的位 移变可以用径向基函数插值计算出来， 从而计算得到所有像素的位移， 实现图 像变形。
融合单元 1717, 用于将第二子集的像素带变形并复制到第一子集中， 并将 第一子集的像素与周围像素融合。
本实施例中， 融合单元 1717将第二子集所对应的像素带进行变形之后无缝 克隆到第一子集所在的间隙上， 在优选的实施例中， 融合单元 1717将采用泊松 融合法， 以使得填充更为逼真、 自然。
纹理合成单元 173 ,用于对结构填充之后的图像块进行纹理合成得到修复后 的图像。
本实施例中， 纹理合成单元 173对完成了结构填充的图像块进行纹理合成， 以实现待修复图像中其它部分的填充。
具体的， 纹理合成单元 173 可采用内容感知的填充方法实现待修复图像的 纹理合成， 以得到修复后的图像。 通过内容感知的填充方法实现图像块的纹理 合成主要是为丟失像素计算恢复值时计算所有包含它的图像块的最相近已知 块， 并通过组合这些已知块所对应的颜色值求和得到这一个丟失像素的最佳值。
在优选的实施例中， 纹理合成单元 173还将在内容感知的填充过程中引入 加速操作， 以通过邻居推荐和随机搜索组合来实现最相近已知块的快速搜索。
上述图像修复方法和装置， 对初始配准的图像块进行处理得到图像块之间 的连接线， 通过连接线构造得到待修复图像的环绕场， 通过极小化连接曲线在 环绕场中的能量对图像块进行配准， 进而对配准后的图像块进行图像填充得到 修复后的图像， 由于图像块的配准不需要图像块之间存在重叠区域， 并且通过 极小化连接曲线在环绕场的能量的方式在数值优化的基础上实现了精确配准， 所以能够适用于任意一个损坏的图像的修复， 且提高了修复的精确性。 本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程， 是可以通过计算机程序来指令相关的硬件来完成， 所述的程序可存储于一计算 机可读取存储介质中， 该程序在执行时， 可包括如上述各方法的实施例的流程。 其中， 所述的存储介质可为磁碟、 光盘、 只读存储记忆体（ Read-Only Memory , ROM )或随机存储记忆体（Random Access Memory, RAM )等。 但并不能因此而理解为对本发明专利范围的限制。 应当指出的是， 对于本领域 的普通技术人员来说， 在不脱离本发明构思的前提下， 还可以做出若干变形和 改进， 这些都属于本发明的保护范围。 因此， 本发明专利的保护范围应以所附 权利要求为准。

Claims

权利要求书
1、 一种图像修复方法， 包括如下步骤：
处理初始配准的图像块得到图像块之间的连接曲线；
通过所述连接曲线构造待修复图像的环绕场；
通过极小化连接曲线在环绕场中的能量对图像块进行配准；
对所述配准后的图像块进行图像填充得到修复后的图像。
2、 根据权利要求 1所述的方法， 其特征在于， 所述对所述初始配准的图像 块进行处理得到图像块之间的连接曲线的步骤为：
获取初始配准的图像块， 所述图像块来自待修复的图像；
沿所述初始配准的图像块边缘向里获取得到所述图像块中的突出曲线； 根据所述突出曲线得到相互关联的突出曲线， 并得到介于所述相互关联的 突出曲线之间的连接曲线。
3、 根据权利要求 1所述的方法， 其特征在于， 所述通过所述连接曲线构造 待修复图像的环绕场的步骤为：
从待修复图像的曲线整体中提取第一最长曲线整体， 为所述第一最长曲线 整体赋予方向；
以已有方向的曲线整体作为狄利克雷边界条件进行调和向量场的求解； 逐一对无方向的曲线整体提取第二最长曲线整体， 根据所述已有方向的曲 线整体的调和向量场确定所述第二最长曲线整体的方向， 直至无方向的曲线整 体均被 U武予方向；
通过求解所有曲线整体的调和向量场得到环绕场的方向部分， 并根据相互 关联的突出曲线数量以及突出曲线的投影计算得到环绕场的大小部分。
4、 根据权利要求 1所述的方法， 其特征在于， 所述通过极小化连接曲线在 环绕场中的能量对图像块进行配准的步骤为：
通过极小化连接曲线在环绕场中的能量求解得到图像块所对应的变换， 根 据所述图像块对应的变换得到连接曲线的变换位置和角度， 按照所述连接曲线 的变换位置和角度配准图像块。
5、 根据权利要求 1所述的方法， 其特征在于， 所述对所述配准后的图像块 进行图像填充得到修复后的图像的步骤为：
通过连接曲线对配准后的图像块进行结构填充；
对结构填充之后的图像块进行纹理合成得到修复后的图像。
6、 根据权利要求 5所述的方法， 其特征在于， 所述通过连接曲线对配准后 的图像块进行结构填充的步骤为：
将连接曲线离散化得到有序点集；
从所述有序点集中提取第一子集， 所述第一子集包括所有位于图像块之间 的间隙上的点；
从所述有序点集中提取与所述第一子集的颜色差最小的第二子集， 所述第 二子集的长度与所述第一子集的长度相当；
将所述第二子集的像素带变形并复制到第一子集中， 并将第一子集的像素 与周围像素融合。
7、 一种图像修复装置， 其特征在于， 包括：
处理模块， 用于处理初始配准的图像块得到图像块之间的连接曲线； 环绕场构造模块， 用于通过所述连接曲线构造待修复图像的环绕场； 配准模块， 用于通过极小化连接曲线在环绕场中的能量对图像块进行配准； 填充模块， 用于对所述配准后的图像块进行图像填充得到修复后的图像。
8、 根据权利要求 7所述的装置， 其特征在于， 所述处理模块包括： 图像块获取单元， 用于获取初始配准的图像块， 所述图像块来自待修复的 图像；
突出曲线获取单元， 用于沿所述初始配准的图像块边缘向里获取得到所述 图像块的突出曲线；
连接曲线获取单元， 用于根据所述突出曲线得到相互关联的突出曲线， 并 得到介于所述相互关联的突出曲线之间的连接曲线。
9、 根据权利要求 7所述的装置， 其特征在于， 所述环绕场构造模块包括： 第一提取单元， 用于从待修复图像的曲线整体中提取第一最长曲线整体， 为所述第一最长曲线整体赋予方向； 向量场求解单元， 用于以已有方向的曲线整体作为狄利克雷边界条件进行 调和向量场的求解；
循环单元， 用于逐一对无方向的曲线整体提取第二最长曲线整体， 根据所 述已有方向的曲线整体的调和向量场确定所述第二最长曲线整体的方向， 直至 无方向的曲线整体均被赋予方向；
环绕场形成单元， 用于通过求解所有曲线整体的调和向量场得到环绕场的 方向部分， 并根据相互关联的突出曲线数量以及突出曲线的投影计算得到环绕 场的大小部分。
10、 根据权利要求 Ί 所述的装置， 其特征在于， 所述配准模块还用于通过 极小化连接曲线在环绕场中的能量求解得到图像块所对应的变换， 根据所述图 像块对应的变换得到连接曲线的变换位置和角度， 按照所述连接曲线的变换位 置和角度配准图像块。
11、 根据权利要求 7所述的装置， 其特征在于， 所述填充模块包括： 结构填充单元， 用于通过连接曲线对配准后的图像块进行结构填充； 纹理合成单元， 用于对结构填充之后的图像块进行纹理合成得到修复后的 图像。
12、 根据权利要求 11所述的装置， 其特征在于， 所述结构填充单元包括： 离散化单元， 用于将连接曲线离散化得到有序点集；
第一子集提取单元， 用于从所述有序点集中提取第一子集， 所述第一子集 包括所有位于图像块之间的间隙上的点；
第二子集提取单元， 用于从所述有序点集中提取与所述第一子集的颜色差 最小的第二子集， 所述第二子集的长度与第一子集的长度相当；
融合单元， 用于将所述第二子集的像素带变形并复制到第一子集中， 并将 第一子集的像素与周围像素融合。
PCT/CN2013/083447 2013-04-19 2013-09-13 图像修复方法和装置 WO2014169561A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/765,447 US9547885B2 (en) 2013-04-19 2013-09-13 Image repairing method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310138362.2 2013-04-19
CN201310138362.2A CN103218785B (zh) 2013-04-19 2013-04-19 图像修复方法和装置

Publications (1)

Publication Number Publication Date
WO2014169561A1 true WO2014169561A1 (zh) 2014-10-23

Family

ID=48816537

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/083447 WO2014169561A1 (zh) 2013-04-19 2013-09-13 图像修复方法和装置

Country Status (3)

Country Link
US (1) US9547885B2 (zh)
CN (1) CN103218785B (zh)
WO (1) WO2014169561A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103218785B (zh) * 2013-04-19 2015-10-28 中国科学院深圳先进技术研究院 图像修复方法和装置
US10916022B2 (en) * 2017-03-27 2021-02-09 Shenzhen Institutes Of Advanced Technology Chinese Academy Of Sciences Texture synthesis method, and device for same
CN108550104B (zh) * 2018-02-28 2022-06-10 北京集光通达科技股份有限公司 图像配准方法、装置
CN109791687B (zh) * 2018-04-04 2023-01-20 香港应用科技研究院有限公司 在任意曲面上的图像修复
CN110782405B (zh) * 2019-10-14 2022-10-18 中国科学院光电技术研究所 一种基于梯度辨识的点目标和暗斑图像背景均衡方法
US11237716B2 (en) * 2019-10-14 2022-02-01 Sling TV L.L.C. Devices, systems and processes for facilitating user adaptive progressions through content

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201113831A (en) * 2009-10-02 2011-04-16 Univ Nat Chunghsing Image inpainting method based on Bezier curves
CN102324102A (zh) * 2011-10-08 2012-01-18 北京航空航天大学 一种图像场景空洞区域结构和纹理信息自动填补方法
CN103218785A (zh) * 2013-04-19 2013-07-24 中国科学院深圳先进技术研究院 图像修复方法和装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7840074B2 (en) * 2004-02-17 2010-11-23 Corel Corporation Method and apparatus for selecting an object in an image
US7840086B2 (en) * 2005-10-12 2010-11-23 The Regents Of The University Of California Method for inpainting of images
EP2248068B1 (en) * 2008-01-29 2014-04-30 Veritec, Inc. Two-dimensional symbol and method for reading same
CN102063705B (zh) * 2010-12-02 2012-08-08 天津大学 一种大区域非均匀纹理合成方法
WO2012096988A2 (en) * 2011-01-10 2012-07-19 Rutgers, The State University Of New Jersey Method and apparatus for shape based deformable segmentation of multiple overlapping objects
CN102831584B (zh) * 2012-08-02 2015-04-22 中山大学 一种数据驱动的物体图像修复系统及方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201113831A (en) * 2009-10-02 2011-04-16 Univ Nat Chunghsing Image inpainting method based on Bezier curves
CN102324102A (zh) * 2011-10-08 2012-01-18 北京航空航天大学 一种图像场景空洞区域结构和纹理信息自动填补方法
CN103218785A (zh) * 2013-04-19 2013-07-24 中国科学院深圳先进技术研究院 图像修复方法和装置

Also Published As

Publication number Publication date
US20150363906A1 (en) 2015-12-17
CN103218785B (zh) 2015-10-28
CN103218785A (zh) 2013-07-24
US9547885B2 (en) 2017-01-17

Similar Documents

Publication Publication Date Title
WO2014169561A1 (zh) 图像修复方法和装置
EP3382644B1 (en) Method for 3d modelling based on structure from motion processing of sparse 2d images
CN107767442B (zh) 一种基于Kinect和双目视觉的脚型三维重建与测量方法
Liu et al. Concrete crack assessment using digital image processing and 3D scene reconstruction
CN106611411B (zh) 一种医学图像中肋骨分割的方法及医学图像处理装置
CN1879553B (zh) 在胸部图像中检测边界的方法及装置
JP6129310B2 (ja) 画像処理装置および画像処理方法
US12106503B2 (en) System and method for mobile 3D scanning and measurement
JP2013524593A (ja) 複数カメラのキャリブレーション用の方法および構成
CN104077804A (zh) 一种基于多帧视频图像构建三维人脸模型的方法
Zheng et al. Landmark matching based retinal image alignment by enforcing sparsity in correspondence matrix
US20140147037A1 (en) Image processing apparatus and method
Ma et al. Learning from documents in the wild to improve document unwarping
EP4156096A1 (en) Method, device and system for automated processing of medical images to output alerts for detected dissimilarities
TW405059B (en) Method for combining the computer models of two surfaces in 3-D space
CN106296587A (zh) 轮胎模具图像的拼接方法
JP6615486B2 (ja) カメラキャリブレーション装置、方法及びプログラム
Ghebreab et al. Combining strings and necklaces for interactive three-dimensional segmentation of spinal images using an integral deformable spine model
Lu et al. A hybrid multimodal non-rigid registration of MR images based on diffeomorphic demons
JP2015171143A (ja) カラーコード化された構造によるカメラ較正の方法及び装置、並びにコンピュータ可読記憶媒体
Wu et al. Adaptive image registration via hierarchical voronoi subdivision
Tyle_ek et al. Refinement of surface mesh for accurate multi-view reconstruction
CN105809657A (zh) 一种角点检测方法和装置
Shah et al. Quantification and visualization of mri cartilage of the knee: A simplified approach
Gao et al. Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13882277

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 14765447

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13882277

Country of ref document: EP

Kind code of ref document: A1