WO2010024402A1 - 画像処理装置および方法、ならびに、画像表示装置 - Google Patents
画像処理装置および方法、ならびに、画像表示装置 Download PDFInfo
- Publication number
- WO2010024402A1 WO2010024402A1 PCT/JP2009/065100 JP2009065100W WO2010024402A1 WO 2010024402 A1 WO2010024402 A1 WO 2010024402A1 JP 2009065100 W JP2009065100 W JP 2009065100W WO 2010024402 A1 WO2010024402 A1 WO 2010024402A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- curved surface
- image
- edge
- unit
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
Definitions
- the present invention relates to an image processing apparatus and method, and an image display apparatus.
- Patent Document 1 discloses a technique for least-square fitting of an ideal image surface without noise by a parametric curved surface.
- a method called Kernel Regression is used, and the shape and edge of the input image can be extracted with higher accuracy than normal linear filtering.
- Patent Document 1 there is a problem that an image is too blurred due to smoothing on a low-order polynomial curved surface, and noise is likely to be generated on the image due to overfitting on a high-order polynomial curved surface.
- the present invention has been made in view of the above, and an object of the present invention is to accurately extract the shape and edge of an input image and to suppress image noise.
- the image processing apparatus uses the direction and magnitude of the gradient of the pixel value of a pixel located around the pixel to be processed in the input image to detect the edge around the pixel to be processed.
- a calculation unit that calculates at least one of the tangential direction and the normal direction, and a conversion unit that converts the coordinates of the peripheral pixels of the processing target pixel into rotated coordinates that are rotated to a coordinate axis according to the tangential direction or normal direction of the edge.
- the image processing apparatus of the present invention includes a calculation unit that calculates a gradient direction and a magnitude of a pixel value of a pixel located around a pixel to be processed in an input image, and a tangential direction or normal direction of an edge of the image.
- a storage unit that stores a filter coefficient determined by a curved surface approximated by using a curved surface model with a higher degree of freedom of variables for each of a plurality of combinations of the direction and the magnitude of the gradient of the pixel value, and the processing with reference to the storage unit
- a selection unit that selects a filter coefficient based on the direction and magnitude of the gradient obtained for the target pixel, and a convolution that calculates a pixel value of the processing target pixel after correction by filtering using the selected filter coefficient Characterized in that it comprises a calculation unit.
- the shape and edge of the input image can be extracted with high accuracy, and the noise of the image can be suppressed.
- FIG. 1 is a diagram illustrating a configuration of an image processing apparatus according to the first embodiment.
- FIG. 2 is a diagram illustrating a configuration of an image processing unit according to the first embodiment.
- FIG. 3 is a diagram illustrating an operation of the image processing unit according to the first embodiment.
- FIG. 4 is a diagram illustrating a relationship between a two-dimensional Gaussian function and an edge tangent direction and a normal direction.
- FIG. 5 is a diagram illustrating a rotation angle formed by an x-axis of an image and a long-axis direction of a two-dimensional Gaussian function.
- FIG. 6 is a diagram illustrating a curved surface fitting operation in the image processing unit according to the first embodiment.
- FIG. 7A shows an original image.
- FIG. 7A shows an original image.
- FIG. 7B is a diagram showing the shape of a two-dimensional Gaussian function at each point on the original image.
- FIG. 8A is a diagram showing a result of fitting with respect to an xy axis by a conventional Kernel Regression.
- FIG. 8B is a diagram showing a result of fitting with respect to an xy axis by a conventional Kernel Regression.
- FIG. 8C is a diagram showing a result of fitting with respect to the uv axis.
- FIG. 8D is a diagram showing a result of fitting with respect to the uv axis.
- FIG. 9 is a diagram showing the difference between the conventional method and the method of this embodiment.
- FIG. 10 is a diagram illustrating an image processing unit according to the second embodiment.
- FIG. 11 is a diagram illustrating an operation of the image processing unit according to the second embodiment.
- FIG. 12 is a diagram showing image feature classification by Harris.
- FIG. 13 is a diagram illustrating an image processing unit according to the third embodiment.
- FIG. 14 is a diagram illustrating an operation of the image processing unit according to the third embodiment.
- FIG. 15 is a diagram illustrating an image processing unit according to the fourth embodiment.
- FIG. 16 is a diagram illustrating an operation of the image processing unit according to the fourth embodiment.
- FIG. 17 is a diagram illustrating an image processing unit according to the fifth embodiment.
- FIG. 18 is a diagram illustrating an operation of the image processing unit according to the fifth embodiment.
- FIG. 19 is a diagram illustrating an image display device according to a sixth embodiment.
- the image processing apparatus 100 includes an input interface unit 101, an image processing unit 102, a storage unit 103, and an output interface unit 104.
- the input interface unit 101 is connected to an input device for inputting an image.
- the input interface unit 101 acquires an input image from the input device.
- the image processing unit 102 performs processing such as filtering on the input image. Accordingly, the image processing unit 102 generates an image from which noise has been removed.
- the storage unit 103 stores the input image acquired by the input interface unit 101, the image processed by the image processing unit 102, and the like.
- the output interface unit 104 is connected to an output device that outputs an image. The output interface unit 104 outputs the image stored in the storage unit 103 to the output device.
- At least one of the input device and the output device may be provided outside the image processing apparatus 100 or may be provided inside the image processing apparatus 100.
- the image processing unit 102 a corresponding to the image processing unit 102 in FIG. 1 includes a calculation unit 201, a parameter calculation unit 202, a conversion unit 203, and a fitting unit 204.
- the operation of the image processing unit 102a in FIG. 2 will be described with reference to FIG.
- the position in the input image is x ⁇
- the set of all the positions of the input image is ⁇ R 2
- the pixel value at the position x of the input image is I (x).
- the pixel value may be a scalar value (for example, a luminance value) or a vector value (for example, an RGB color signal).
- transposition of a matrix / vector is represented by adding a superscript “T” to the matrix / vector.
- Num (N) is the number of elements in the set N.
- step S301 the calculation unit 201 obtains the direction and magnitude of the gradient of the surrounding pixel values for each pixel of the input image.
- a discretized first-order differential operation is used to determine the direction and magnitude of the gradient of the pixel value.
- a Sobel operator can be used as the discretized first-order differential operation.
- the calculation unit 201 performs the calculation shown in Formula 1.
- d x (x) is an x-direction differential value at the position x.
- d y (x) is a y-direction differential value at the position x.
- “*” Is a convolution operation.
- d x (x) is expressed as Equation 2.
- a discretized differential operation other than the Sobel operator may be used.
- a forward difference, a backward difference, or a center difference may be used.
- the parameter calculation unit 202 calculates an image feature parameter indicating the direction and size of the edge at the position x, using d x (x) and d y (x).
- the parameter calculation unit 202 calculates a two-dimensional Gaussian function represented by an anisotropic Gaussian distribution according to the direction and size of the edge.
- the two-dimensional Gaussian function is defined as Equation 3 by the differential structure tensor H (x).
- s ⁇ N indicates the position of a point within a predetermined range (hereinafter referred to as “local neighborhood”) centering on the position x.
- h (> 0) is the standard deviation of the anisotropic Gaussian distribution.
- the two-dimensional Gaussian function is strongly influenced by the component in the normal direction of the edge. As shown in FIG. 4, the two-dimensional Gaussian function has an elliptical shape in which the direction in which the change in pixel value is small and the major axis are substantially parallel, and the direction in which the change in pixel value is large and the minor axis are substantially parallel. In the two-dimensional Gaussian function, the clearer the edge, the shorter the short diameter, and the elliptical shape collapsed in the tangential direction of the edge.
- the image feature parameter is calculated by Equation 4 from the structure tensor H (x).
- Equation 5 H (x) is expressed as Equation 5.
- the rotation angle ⁇ represents an angle formed by the x-axis of the image and the major axis direction of the two-dimensional Gaussian function.
- ⁇ + represents the length of the major axis of the ellipse by a two-dimensional Gaussian function
- ⁇ ⁇ represents the length of the minor axis.
- ⁇ + and ⁇ ⁇ are eigenvalues of the structure tensor.
- the long axis of the two-dimensional Gaussian function substantially coincides with the tangential direction of the edge.
- the short axis of the two-dimensional Gaussian function substantially coincides with the normal direction of the edge.
- the image feature parameter is not stably calculated due to noise included in the image. Therefore, as shown in Expression 6, a structure tensor convolved with respect to a point in the local vicinity N centered on the position x may be used.
- the local neighborhood N may be an arbitrary shape. For example, a 5 ⁇ 5 (pixel is a unit; hereinafter omitted) centered around the position x may be used as the local neighborhood N.
- step S303 the conversion unit 203 converts the coordinates of the position s within the local neighborhood N with respect to the position x (hereinafter referred to as local coordinates s) into the local coordinates u based on the rotation coordinates in accordance with the rotation angle ⁇ .
- the coordinate conversion from the xy coordinates of the image to the uv local coordinates of the two-dimensional Gaussian function is expressed by Equation 7.
- step S304 the fitting unit 204 obtains parameters of the curved surface to be fitted by curved surface fitting by the least square method.
- curved surface fitting using a curved surface model of an nth order polynomial used in Patent Document 1 will be described.
- an n-order polynomial curved surface model is expressed by Equation 8.
- a curved surface fitting by a curved surface model is performed on the local neighborhood N with the position x as the center.
- the output pixel value after the curved surface fitting, an I (x) a 0.
- the curved surface fitting disclosed in Patent Document 1 has the following problems. For example, it is assumed that there is a diagonal edge. In order to fit this with a quadric surface, parameters a 0 , a 1 , a 2 , a 3 , a 4 , and a 5 are all required. On the other hand, if the edge is in the vertical direction, only the parameters a 0 , a 1 , and a 3 are required. This is because the t component of local coordinates is not necessary for the vertical edge. Similarly, if the edge is in the horizontal direction, only the parameters a 0 , a 2 and a 5 are required.
- the fitting of Patent Document 1 has a direction dependency on the edge. If the number of parameters required for fitting increases, the stability of fitting may be reduced. If it is attempted to cover all directions, more parameters than necessary are used, which may cause overfitting.
- the stability of the fitting by the least square method is determined by the number of parameters to be estimated and the number of sample points that can be used for fitting. If an excessive number of parameters is used with respect to the number of sample points, the fitting becomes unstable.
- the major axis direction of the two-dimensional Gaussian function corresponds to the tangential direction of the edge.
- the minor axis direction of the two-dimensional Gaussian function corresponds to the normal direction of the edge.
- the change in pixel value is large in the normal direction of the edge and small in the tangential direction of the edge. This characteristic is considered by curved surface fitting. Since the change in the pixel value is large in the normal direction of the edge, the number of curved surface parameters increases. As a result, the degree of freedom is increased and the fitting is performed with high accuracy. On the other hand, since the change in the pixel value is small in the tangential direction of the edge, the number of parameters of the curved surface is small. As described above, the curved surface model is expressed as Equation 9.
- u is the long axis of the two-dimensional Gaussian function
- v is the short axis of the two-dimensional Gaussian function.
- the degree of freedom of the curved surface is set only in the short-axis direction of the two-dimensional Gaussian function, and a zero-order polynomial is set in the long-axis direction.
- the sharpness of the edge is adjusted in the minor axis direction, and the noise removal performance is improved in the major axis direction.
- fitting independent of the edge direction is possible. Further, the sharpness of the edge can be increased with fewer parameters than in Patent Document 1, and the stability of the fitting can be improved even if the same sample point is used.
- Patent Document 1 For example, in Patent Document 1, six parameters are required to express an edge with a quadric surface accuracy. On the other hand, in this embodiment, three parameters are required. In Patent Document 1, 15 parameters are required to express an edge with a quartic surface accuracy. On the other hand, in this embodiment, 5 parameters are required. Thus, regarding the number of required parameters, the difference between the two becomes larger as the degree of freedom of the curved surface is increased.
- the curved surface fitting is performed using a least square method.
- the curved surface model is defined as follows.
- Equation 10 a quartic polynomial curved surface model is shown, but a polynomial curved surface model of 0th to 5th or 7th order or higher may be used. Further, instead of a polynomial, a curved surface model based on a sine wave as shown in Expression 11 may be used.
- the pixel value of the pixel at the position s in the local neighborhood N centered on the position x is I (x + s) (where s ⁇ N).
- the coordinate conversion by the rotation angle ⁇ with respect to the local coordinate s is obtained by Equation 12 through a coordinate conversion step using a rotation matrix.
- Equation 13 the pixel value I (x + s) and the curved surface f (R ⁇ 1 ( ⁇ ) s) are associated with each other.
- the least square method is a method for obtaining a parameter that minimizes the square error in this correspondence, and is defined as Equation 13.
- a ⁇ (x) is a fitting parameter by the least square method.
- K (x, s) is a weight at the point s.
- a two-dimensional Gaussian function is used.
- Arbitrary shapes can be used for the local neighborhood N. For example, a 5 ⁇ 5 tap rectangular area centered on the position x can be used.
- Expression 13 is expressed in matrix form as Expression 14.
- N ⁇ s 0 ,..., S n ⁇ .
- Equation 15 is called a normal equation. In the case of the linear least square method, this is the optimal solution.
- the inverse matrix can be numerically calculated by LU decomposition or singular value decomposition. Here, it is assumed that a ⁇ (x) is obtained as shown in Equation 16.
- Equation 17 The output pixel after fitting is given by Equation 17.
- step S601 the fitting unit 204 calculates the matrix P of Expression 14.
- step S602 the fitting unit 204 calculates the matrix W of Expression 14 using the image feature parameter of Expression 4.
- step S603 the fitting unit 204 calculates (P T WP) ⁇ 1 using LU decomposition, singular value decomposition, or the like.
- step S604 the fitting unit 204 calculates a ⁇ (x) in Expression 15.
- FIGS. 8A and 8B show the results obtained by performing the first-order differentiation in the x direction and the y direction, respectively, by performing curved surface fitting with respect to FIG. 7B by the least square method.
- the curved surface parameters are dispersed on each axis, and the model parameters depend on the rotation angle of the two-dimensional Gaussian function. For this reason, when trying to reproduce edges in all directions at the same level, parameters for all quadric surfaces are required.
- FIGS. 8C and 8D perform curved surface fitting with respect to FIG. 7B by the least square method, and are linear in the u direction (major axis direction of the two-dimensional Gaussian function) and the v direction (minor axis direction of the two-dimensional Gaussian function), respectively.
- the result of differentiation is shown. Referring to FIGS. 8C and 8D, it can be seen that the axis of the curved surface model coincides with the principal component axis of the edge. For this reason, the edge components are concentrated on the v-axis (the short axis of the two-dimensional Gaussian function). For this reason, only the v-axis needs to be used when reproducing edges in all directions.
- fitting by Kernel Regression is performed on the xy axis.
- fitting by Kernel Regression is performed on the uv axis. Since the uv axis is the principal component axis of the edge, the degree of freedom can be distributed in the maximum amplitude direction (principal component) of information.
- filtering with excellent edge reproducibility was performed by rotating the curved surface model in the tangential direction of the edge. This is because, at the edge portion, information is concentrated in the short axis direction of the ellipse by the above-described two-dimensional Gaussian function, that is, the normal direction of the edge, and thus a curved surface giving a degree of freedom to the short axis component is well applied.
- the image includes not only an edge region but also a flat region and a corner region (such as a corner or a tip). In these areas, information is not necessarily concentrated on the short axis component.
- the region is classified according to the image feature, and a suitable curved surface model is assigned according to the classification.
- the image feature is obtained using the differential value in the x direction and the differential value in the y direction. More specifically, the image feature is obtained using the structure tensor described in the first embodiment.
- the operation by the image processing unit 102b in FIG. 10 is different from the operation in FIG. 3 in that at least the curved surface model selection step S1103 is performed before the coordinate conversion step and the curved surface fitting step. Other steps will be described with a focus on differences from FIG.
- step S1102 parameters such as the tangent direction of the edge representing the local features of the image, the major axis and the minor axis of the ellipse are calculated using the structure tensor. Harris et al. Perform feature classification of images from structural tensors (C. Harris and M. Stephens (1988), “A Combined Corner and Edge Detector”, Proc. Of the 4th ALVEY Vision Conference 7, 14). ).
- each part of the image has an edge region (Edge) and a flat region (Flat) as shown in FIG. 12 according to the eigenvalues ⁇ + and ⁇ ⁇ . ) And a corner area (Corner). If one of the eigenvalues ⁇ + and ⁇ ⁇ is large and the other is small, the ellipse by the two-dimensional Gaussian function has a collapsed shape. In that case, the image portion is an edge region. If both eigenvalues are large, the ellipse by the two-dimensional Gaussian function becomes smaller in an isotropic circle.
- the image portion is a corner (corner or tip). If both eigenvalues are small, the ellipse by the two-dimensional Gaussian function becomes isotropic and large. In that case, it indicates that the image portion is a flat region.
- the curved surface model of the first embodiment is a curved surface model using only the short axis component of an ellipse by a two-dimensional Gaussian function. This is suitable for the edge region among the above regions based on the fact that information is concentrated in the edge normal direction in the edge region.
- the following edge region curved surface model can be considered.
- the corner region and the flat region an ellipse based on a two-dimensional Gaussian function is isotropic, and information is not concentrated on the short axis (v axis). Therefore, it is appropriate to use both the u-axis and the v-axis in the corner region and the flat region.
- the corner region has a large change in pixel value, and thus it is suitable that the degree of freedom of the curved surface model is high.
- the following corner area curved surface model can be considered.
- Equation 20 a flat region curved surface model composed of a low-order polynomial as shown in Equation 20 can be considered.
- the selection unit 1001 classifies the target pixel into a plurality of predetermined classes (regions).
- the selection unit 1001 selects a curved surface model corresponding to the plurality of classified classes on a one-to-one basis as the curved surface model of the target pixel.
- the selection unit 1001 classifies the target pixel based on the eigenvalues ⁇ + and ⁇ ⁇ of the structure tensor. The classification is performed according to the classification described with reference to FIG. Then, the selection unit 1001 selects the curved surface model according to the classification result.
- step S1105 the fitting unit 204 performs fitting by the least square method using the curved surface model selected in step S1103.
- the third embodiment is different from the first embodiment in that a filter selection unit 1301 and a convolution operation unit 1302 are provided instead of the conversion unit 203 and the curved surface fitting unit 204.
- the first embodiment it is necessary to solve a normal equation for each pixel to be filtered. Specifically, it is necessary to numerically calculate the inverse matrix by LU decomposition or singular value decomposition.
- a result obtained by solving a normal equation in advance is stored in an LUT (Look Up Table). Thereby, feasibility in a circuit etc. is improved.
- the operation of the image processing unit 102c of FIG. 13 shown in FIG. 14 includes at least a filter selection step S1403 and a convolution calculation step S1404 in place of the coordinate conversion step S303 and the curved surface fitting step S304. Is different. The other steps are the same as in FIG.
- step S1403 the filter selection unit 1301 selects an appropriate filter from the LUT obtained by solving the normal equation based on the image feature parameter calculated in step S1402.
- Equation 15 Y represents a pixel value as shown in Equation 14, and changes according to the input image.
- P r WP P ⁇ 1 P r W depends only on the image feature parameters ⁇ + , ⁇ ⁇ , and ⁇ , and does not depend on the image.
- the two-dimensional Gaussian function is represented by the differential autocorrelation coefficient of the input image, as shown in Equation 21.
- Equation 23 The two-dimensional Gaussian function is also rewritten as Equation 23.
- Equation 24 The matrix W is expressed as Equation 24.
- Equation 25 the matrix W depends only on the image feature parameters.
- Equation 25 the matrix P is expressed as Equation 25.
- Equation 26 also depends only on the image feature parameters.
- ,..., L) are calculated and registered in the LUT (filter bank).
- the corresponding X ( ⁇ + , ⁇ ⁇ , ⁇ ) 1 is selected from the LUT based on the calculated image feature parameters ( ⁇ + , ⁇ ⁇ , ⁇ ).
- step S1404 the convolution operation unit 1302 performs a convolution operation with the pixel value vector Y using the filter X ( ⁇ + , ⁇ ⁇ , ⁇ ) l selected in the filter selection step S1403. Then, the convolution operation unit 1302 calculates an output pixel to which the curved surface is fitted by the least square method. Specifically, the convolution operation unit 1302 performs a matrix calculation represented by Equation 28.
- a filtering process adapted to the edge of the input image is performed using a two-dimensional Gaussian function.
- the tip of the triangle cannot be expressed by a two-dimensional Gaussian function, the image becomes dull. Therefore, in the fourth embodiment, it is possible to solve the problem that the tip portion becomes dull by making the two-dimensional Gaussian function robust from the pixel value of the pixel at the tip portion of the triangle.
- the operation of the image processing unit 102d in FIG. 15 is different from the operation described in FIG. 3 in that it includes a weight calculation step S1606 after at least the curved surface fitting step S1605. Other steps will be described with a focus on differences from FIG.
- Step S1605 is basically the same as in the first embodiment, but the fitting unit 204 calculates the matrix W according to Equation 29.
- the weight calculation unit 1501 weights the pixel value of the input image in the local vicinity according to the distance from the processing target pixel located at the center of the local coordinates. For example, in the local vicinity, the weight is decreased as the difference from the pixel value of the processing target pixel of the input image is larger, and the weight is increased as the difference is smaller.
- the two-dimensional Gaussian function applies to the step edge portion of the image, but may not apply to the tip portion or corner. When fitting by the least square method in a region where such a two-dimensional Gaussian function is not applicable, the image is greatly distorted due to outlier pixel values that do not apply to the function, and as a result, the adverse effect that the corner becomes dull appears. .
- Such an outlier region can be said to be a region of pixels having pixel values that are different from the pixel value of the pixel to be processed.
- the weight based on the difference in pixel values can be defined as follows using a Gaussian function as shown in Equation 30.
- ⁇ I > 0 is a standard deviation of the Gaussian function, and is a parameter indicating how much the pixel value is different to reduce the weight.
- Equation 30 the weight of Equation 30 also reacts to the block distortion, and the block distortion is not eliminated. Therefore, deblocking weights such as Equation 31 can be used so that the pixel value difference weights are not used at the block boundaries.
- (Fifth embodiment) 17 that are the same as those of the image processing unit 102a in FIG. 2 are denoted by the same reference numerals in the image processing unit 102e of the present embodiment illustrated in FIG.
- the fifth embodiment is different from the first embodiment in that a difference calculation unit 1701 is provided.
- the difference between the pixel value of the curved surface fitted in the first to fourth embodiments and the pixel value of the input image is taken, and reproducibility of a fine texture or the like is performed by fitting again to this difference. Can be increased.
- the operation of the image processing unit 102e in FIG. 17 is described in FIG. 3 in that it includes a difference calculation step S1806, a coordinate calculation step S1807, and a curved surface fitting step S1808 at least after the curved surface fitting step S1805.
- the operation is different.
- Other steps will be described with a focus on differences from FIG.
- the pixel I (x) can be modeled as Equation 32.
- Equation 33 S (x) is a skeleton component
- T (x) is a texture component
- n (x) is noise.
- the degree of freedom of the curved surface is set in the normal direction of the edge, not in the tangential direction. This is equivalent to extracting the skeleton component, ignoring the irregularities in the tangential direction of the edge. Therefore, the curved surface fitting result by the least square method can be regarded as a skeleton component of the image as shown in Equation 34.
- step S1806 the difference calculation unit 1701 calculates the difference between the pixel value of the input image and the pixel value of the skeleton component. First, from Equation 32 and Equation 34, the texture component becomes Equation 35.
- step S1808 the curved surface fitting unit 204 calculates a texture component by performing curved surface fitting on the local coordinates subjected to coordinate transformation in step S1807 by the least square method with respect to the difference obtained in step S1806.
- the texture curved surface model is expressed by Equation 38.
- the model is set in the normal direction of the edge, but here the model is set in the tangential direction of the edge. This is because the texture component is extracted in the tangential direction of the edge orthogonal to the normal direction of the edge from which the skeleton component of the image is removed.
- the least square method for the difference is expressed as Equation 39.
- Equation 40 When this is converted into a matrix form, it is expressed as Equation 40.
- Equation 41 the normal equation is expressed as Equation 41.
- the inverse matrix can be numerically calculated by LU decomposition or singular value decomposition.
- b ⁇ (x) is obtained as in Expression 42.
- the image processing unit 102e adds the curved surface model and the textured curved surface model to generate an output image. That is, the output image after filtering is expressed by Equation 44.
- FIG. 19 is a diagram illustrating an image display apparatus according to the present embodiment.
- the image display apparatus according to this embodiment includes an image processing apparatus 1900 and a display unit 1903 that displays an image.
- the same reference numerals are given to the same portions as those of the image processing unit 102c in FIG.
- the image processing apparatus 1900 includes a decoding unit 1901 that decodes input data and obtains an input image, a filter selection unit 1902, and an LUT 1903.
- the filter selection unit 1902 and the LUT 1903 perform the same operation as the filter selection unit 1301 in FIG.
- the filter selection unit 1902 changes the filter to be selected according to the encoding method of the input data.
- the LUT 1903 holds filter coefficients calculated in advance.
- a filter coefficient is prepared for each encoding method.
- the LUT 1903 is a curved surface model in which the displacement on the coordinate axis in the direction in accordance with the tangential direction or the normal direction of the edge of the image is a variable, and a free variable indicating the displacement on the coordinate axis in the tangential direction of the edge.
- the input data is H.264.
- a filter coefficient used when encoding is performed according to an encoding method with high encoding efficiency such as H.264 a filter coefficient based on a curved surface obtained by approximation by a curved surface model based on a high degree of freedom is held in the LUT 1903. Yes.
- the encoding efficiency is H.264. Low compared to H.264, H.264.
- a filter coefficient used in the case of being encoded by an encoding method such as H.263, H.263 is used.
- a filter coefficient generated by a curved surface model based on a lower degree of freedom than that encoded by H.264 is held in the LUT 1903.
- the filter may be switched according to the size of the input image. For example, in order to use when the image size is small, a filter coefficient is generated so as to reduce the size of the target region used in the convolution calculation, and is stored in the LUT 1903. Further, in order to use when the size of the input image is large, filter coefficients that increase the size of the target used in the convolution calculation are generated and held in the LUT 1903. *
- suitable image processing can be performed according to the encoding method and image size of the input image.
- various encoding methods and image sizes are distributed over the Internet. Even if the encoding method and image size of the input data are different, a high-quality image can be obtained.
- an image processing program for executing each processing step in the image processing apparatus according to each embodiment described above is a CD-ROM, a flexible disk (FD), a CD-R in an installable or executable file. , Recorded on a computer-readable recording medium such as a DVD (Digital Versatile Disk), and provided as a computer program product.
- the image processing program according to each embodiment may be provided by being incorporated in advance in a ROM or the like.
- the image processing program executed by the image processing apparatus has a module configuration including each part of the above-described image processing unit 102 or image processing apparatus 1900, and is not illustrated as actual hardware.
- the CPU processor
- each unit of the image processing unit 102 or the image processing device 1900 is loaded on a main storage device (not shown), and the image processing unit 102 or the image processing device 1900 is loaded. These units are generated on the main storage device.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
Description
図1に示すように、画像処理装置100は、入力インターフェース部101と、画像処理部102と、記憶部103と、出力インターフェース部104と、を備える。
例えば、斜め方向のエッジがあるとする。これを2次曲面でフィッティングするには、パラメータa0、a1、a2、a3、a4、a5が全て必要である。これに対し、縦方向のエッジであれば、パラメータa0、a1、a3のみでよい。縦方向のエッジに関しては、局所座標のt成分が必要ないからである。同様に、横方向のエッジであれば、パラメータa0、a2、a5のみでよい。
図10に示す本実施形態の画像処理部102bの各部について、図2の画像処理部102aと同一部分は同一符号で示す。第2の実施形態が第1の実施形態と異なる点は、選択部1001を備える点である。
図13に示す本実施形態の画像処理部102cの各部について、図2の画像処理部102aと同一部分は同一符号で示す。第3の実施形態が第1の実施形態と異なる点は、変換部203及び曲面フィッティンブ部204に代えて、フィルタ選択部1301及び畳み込み演算部1302を備える点である。
図15に示す本実施形態の画像処理部102dの各部について、図2の画像処理部102aと同一部分は同一符号で示す。第4の実施形態が第1の実施形態と異なる点は、重み算出部1501を備える点である。
図17に示す本実施形態の画像処理部102eの各部について、図2の画像処理部102aと同一部分は同一符号で示す。第5の実施形態が第1の実施形態と異なる点は、差分算出部1701を備える点である。
図19は、本実施形態に係る画像表示装置を示す図である。本実施形態の画像表示装置は、画像処理装置1900と、画像を表示する表示部1903を有する。図19に示す画像処理装置1900の各部について、図13の画像処理部102cと共通する部分には同一の符号を付して、詳細な説明を省略する。
Claims (10)
- 入力画像の処理対象画素の周辺に位置する画素の画素値の勾配の方向及び大きさを用いて、前記処理対象画素周辺でのエッジの接線方向および法線方向のうち少なくとも一方を演算する演算部と、
前記処理対象画素の周辺画素の座標を、前記エッジの接線方向または法線方向に従った座標軸に回転させた回転座標に変換する変換部と、
前記処理対象画素の周辺での局所的な画素値の分布を、前記回転座標での位置を変数とする曲面モデルで近似した曲面を求め、前記曲面における前記処理対象画素の位置の画素値から、補正後の前記処理対象画素の画素値を求めるフィッティング部と、
を備えることを特徴とする画像処理装置。 - 前記処理対象画素の周辺の画素値の勾配の方向及び大きさを用いて、前記処理対象画素を予め定められた複数のクラスに分類し、前記複数のクラスに対応する曲面モデルを当該処理対象画素の曲面モデルとして選択する選択部を更に備え、
前記フィッティング部は、前記選択部により選択された曲面モデルに従って前記曲面を求める
ことを特徴とする請求項1に記載の画像処理装置。 - 前記フィッティング部は、前記回転座標において、前記エッジの法線方向の座標軸上での変位を示す変数の自由度の方が、前記エッジの接線方向の座標軸上での変位を示す変数の自由度よりも高い前記曲面モデルを用いることを特徴とする請求項1に記載の画像処理装置。
- 前記入力画像の周辺の画素に対して、前記処理対象画素からの距離に応じた重みを与える重み算出部を更に備え、
前記フィッティング部は、前記入力画像の画素値に対して前記重みを乗じた画素値に基づいて近似した曲面を求めることを特徴とする請求項1に記載の画像処理装置。 - 前記重み算出部は、前記処理対象画素の画素値との差が大きい画素値の画素程、小さな重みを与える
ことを特徴とする請求項4に記載の画像処理装置。 - 入力画像の処理対象画素の周辺に位置する画素の画素値の勾配の方向及び大きさを演算する演算部と、
画像のエッジの接線方向または法線方向に従った方向の座標軸上での変位を変数とする曲面モデルであって、前記エッジの接線方向の座標軸上での変位を示す変数の自由度よりも前記エッジの法線方向の座標軸上での変位を示す変数の自由度の方が高い前記曲面モデルを用いて近似した曲面によって定まるフィルタ係数を、画素値の勾配の方向及び大きさの複数の組合せ毎に記憶する記憶部と、
前記記憶部を参照し、前記処理対象画素について求めた勾配の方向及び大きさに基づいてフィルタ係数を選択する選択部と、
選択された前記フィルタ係数によるフィルタリングによって補正後の前記処理対象画素の画素値を算出する畳み込み演算部と、
を備えることを特徴とする画像処理装置。 - 入力データを復号して入力画像を得る復号部をさらに備え、
前記記憶部は、符号化方式毎にフィルタ係数を保持し、
前記選択部は、前記入力データの符号化方式に応じて前記フィルタ係数を選択する
ことを特徴とする請求項6に記載の画像処理装置。 - 入力データを復号して入力画像を得る復号部をさらに備え、
前記記憶部は、画像のサイズ毎にフィルタ係数を保持し、
前記選択部は、前記入力画像のサイズに応じて前記フィルタ係数を選択する
ことを特徴とする請求項6に記載の画像処理装置。 - 入力画像の処理対象画素の周辺に位置する画素の画素値の勾配の方向及び大きさを用いて、前記処理対象画素周辺でのエッジの接線方向および法線方向のうち少なくとも一方を演算する演算ステップと、
前記処理対象画素の周辺画素の座標を、前記エッジの接線方向または法線方向に従った座標軸に回転させた回転座標に変換する変換ステップと、
前記処理対象画素の周辺での局所的な画素値の分布を、前記回転座標での位置を変数とする曲面モデルで近似した曲面を求め、前記曲面における前記処理対象画素の位置の画素値から、補正後の前記処理対象画素の画素値を求めるフィッティングステップと、
を備えることを特徴とする画像処理方法。 - 請求項6に記載の画像処理装置と、
画像を表示する表示部と、
を更に備えたことを特徴とする画像表示装置。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/061,270 US9092870B2 (en) | 2008-08-29 | 2009-08-28 | Techniques to suppress noises in an image to precisely extract shapes and edges |
JP2010526796A JP5487106B2 (ja) | 2008-08-29 | 2009-08-28 | 画像処理装置および方法、ならびに、画像表示装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2008-222696 | 2008-08-29 | ||
JP2008222696 | 2008-08-29 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2010024402A1 true WO2010024402A1 (ja) | 2010-03-04 |
Family
ID=41721565
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2009/065100 WO2010024402A1 (ja) | 2008-08-29 | 2009-08-28 | 画像処理装置および方法、ならびに、画像表示装置 |
Country Status (3)
Country | Link |
---|---|
US (2) | US20100054606A1 (ja) |
JP (1) | JP5487106B2 (ja) |
WO (1) | WO2010024402A1 (ja) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101200378B1 (ko) | 2011-08-30 | 2012-11-12 | 인하대학교 산학협력단 | 국소 각 위상을 이용한 강건한 텍스처 특징 추출 방법 |
JP2017502323A (ja) * | 2013-12-17 | 2017-01-19 | 深▲せん▼市華星光電技術有限公司Shenzhen China Star Optoelectronics Technology Co., Ltd. | 映像信号取得方法及び映像信号取得装置 |
JP2019096264A (ja) * | 2017-11-28 | 2019-06-20 | Kddi株式会社 | 画像処理装置、画像処理方法、及びプログラム |
CN111353470A (zh) * | 2020-03-13 | 2020-06-30 | 北京字节跳动网络技术有限公司 | 图像的处理方法、装置、可读介质和电子设备 |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5075861B2 (ja) * | 2009-03-16 | 2012-11-21 | 株式会社東芝 | 画像処理装置及び画像処理方法 |
JP5665508B2 (ja) * | 2010-11-30 | 2015-02-04 | キヤノン株式会社 | 画像処理装置及び方法、並びにプログラム及び記憶媒体 |
US10593007B1 (en) | 2015-06-11 | 2020-03-17 | Digimarc Corporation | Methods and arrangements for configuring industrial inspection systems |
KR101905993B1 (ko) * | 2016-10-31 | 2018-10-10 | 현대자동차주식회사 | 차량용 내장재 및 그 성형 방법 |
JP6777507B2 (ja) * | 2016-11-15 | 2020-10-28 | Kddi株式会社 | 画像処理装置および画像処理方法 |
CN106780352B (zh) * | 2016-12-16 | 2020-06-09 | 珠海赛纳打印科技股份有限公司 | 图像旋转方法、装置及图像形成设备 |
CN107644398B (zh) * | 2017-09-25 | 2021-01-26 | 上海兆芯集成电路有限公司 | 图像插补方法及其相关图像插补装置 |
CN111178118B (zh) * | 2018-11-13 | 2023-07-21 | 浙江宇视科技有限公司 | 图像采集处理方法、装置及计算机可读存储介质 |
CN112986964B (zh) * | 2021-02-26 | 2023-03-31 | 北京空间机电研究所 | 基于噪声邻域密度的光子计数激光点云自适应去噪方法 |
CN113326855B (zh) * | 2021-06-22 | 2022-07-12 | 长光卫星技术股份有限公司 | 基于二维高斯曲面拟合的夜间灯光遥感图像特征提取方法 |
CN115512341B (zh) * | 2022-09-15 | 2023-10-27 | 粤丰科盈智能投资(广东)有限公司 | 基于高斯分布拟合的目标检测方法、装置及计算机介质 |
CN116060269B (zh) * | 2022-12-08 | 2024-06-14 | 中晟华越(郑州)智能科技有限公司 | 回型产品喷涂方法 |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11113885A (ja) * | 1997-10-08 | 1999-04-27 | Oki Electric Ind Co Ltd | 個体識別装置およびその方法 |
JP2001283252A (ja) * | 2000-03-29 | 2001-10-12 | Minolta Co Ltd | 点群に面をフィッティングする方法および装置 |
JP2004125690A (ja) * | 2002-10-04 | 2004-04-22 | Dainippon Printing Co Ltd | 丸み自動計測方法、マスクパターン品質測定装置 |
JP2004133551A (ja) * | 2002-10-08 | 2004-04-30 | Noritsu Koki Co Ltd | 画像処理方法、画像処理装置、画像処理プログラム、および画像処理プログラムを記録した記録媒体 |
JP2006025105A (ja) * | 2004-07-07 | 2006-01-26 | Sharp Corp | 動画像再生装置 |
JP2006059346A (ja) * | 2004-08-20 | 2006-03-02 | Mitsubishi Electric Research Laboratories Inc | 画像内の画素をフィルタリングするシステムおよび方法 |
US20070047838A1 (en) * | 2005-08-30 | 2007-03-01 | Peyman Milanfar | Kernel regression for image processing and reconstruction |
JP2007201533A (ja) * | 2006-01-23 | 2007-08-09 | Toshiba Corp | ボケ変換装置及び方法 |
JP2007293550A (ja) * | 2006-04-24 | 2007-11-08 | Konica Minolta Sensing Inc | ポリゴンメッシュ編集方法、装置、システム及びプログラム |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6229905B1 (en) * | 1997-03-26 | 2001-05-08 | Oki Electric Industry Co., Ltd. | Animal identification based on irial granule analysis |
US6992665B2 (en) * | 2000-03-29 | 2006-01-31 | Minolta Co., Ltd. | Method and device for fitting surface to point group, modeling device, and computer program |
US7826535B2 (en) * | 2002-04-11 | 2010-11-02 | Broadcom Corporation | Adaptive pixel processing |
JP4042563B2 (ja) * | 2002-12-27 | 2008-02-06 | セイコーエプソン株式会社 | 画像ノイズの低減 |
US8200028B2 (en) * | 2007-12-07 | 2012-06-12 | Csr Technology Inc. | System and method for detecting edges in a video signal |
JP2010087614A (ja) * | 2008-09-29 | 2010-04-15 | Toshiba Corp | 画像処理装置、画像処理方法、及び、プログラム |
JP5075861B2 (ja) * | 2009-03-16 | 2012-11-21 | 株式会社東芝 | 画像処理装置及び画像処理方法 |
-
2009
- 2009-08-20 US US12/544,356 patent/US20100054606A1/en not_active Abandoned
- 2009-08-28 JP JP2010526796A patent/JP5487106B2/ja not_active Expired - Fee Related
- 2009-08-28 WO PCT/JP2009/065100 patent/WO2010024402A1/ja active Application Filing
- 2009-08-28 US US13/061,270 patent/US9092870B2/en not_active Expired - Fee Related
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11113885A (ja) * | 1997-10-08 | 1999-04-27 | Oki Electric Ind Co Ltd | 個体識別装置およびその方法 |
JP2001283252A (ja) * | 2000-03-29 | 2001-10-12 | Minolta Co Ltd | 点群に面をフィッティングする方法および装置 |
JP2004125690A (ja) * | 2002-10-04 | 2004-04-22 | Dainippon Printing Co Ltd | 丸み自動計測方法、マスクパターン品質測定装置 |
JP2004133551A (ja) * | 2002-10-08 | 2004-04-30 | Noritsu Koki Co Ltd | 画像処理方法、画像処理装置、画像処理プログラム、および画像処理プログラムを記録した記録媒体 |
JP2006025105A (ja) * | 2004-07-07 | 2006-01-26 | Sharp Corp | 動画像再生装置 |
JP2006059346A (ja) * | 2004-08-20 | 2006-03-02 | Mitsubishi Electric Research Laboratories Inc | 画像内の画素をフィルタリングするシステムおよび方法 |
US20070047838A1 (en) * | 2005-08-30 | 2007-03-01 | Peyman Milanfar | Kernel regression for image processing and reconstruction |
JP2007201533A (ja) * | 2006-01-23 | 2007-08-09 | Toshiba Corp | ボケ変換装置及び方法 |
JP2007293550A (ja) * | 2006-04-24 | 2007-11-08 | Konica Minolta Sensing Inc | ポリゴンメッシュ編集方法、装置、システム及びプログラム |
Non-Patent Citations (1)
Title |
---|
HIROYUKI TAKEDA ET AL.: "Kernel Regression for Image Processing and Reconstruction", IEEE TRANS. ON IMAGE PROCESSING, vol. 16, no. 2, February 2007 (2007-02-01), pages 349 - 366 * |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101200378B1 (ko) | 2011-08-30 | 2012-11-12 | 인하대학교 산학협력단 | 국소 각 위상을 이용한 강건한 텍스처 특징 추출 방법 |
JP2017502323A (ja) * | 2013-12-17 | 2017-01-19 | 深▲せん▼市華星光電技術有限公司Shenzhen China Star Optoelectronics Technology Co., Ltd. | 映像信号取得方法及び映像信号取得装置 |
JP2019096264A (ja) * | 2017-11-28 | 2019-06-20 | Kddi株式会社 | 画像処理装置、画像処理方法、及びプログラム |
JP7140491B2 (ja) | 2017-11-28 | 2022-09-21 | Kddi株式会社 | 画像処理装置、画像処理方法、及びプログラム |
CN111353470A (zh) * | 2020-03-13 | 2020-06-30 | 北京字节跳动网络技术有限公司 | 图像的处理方法、装置、可读介质和电子设备 |
CN111353470B (zh) * | 2020-03-13 | 2023-08-01 | 北京字节跳动网络技术有限公司 | 图像的处理方法、装置、可读介质和电子设备 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2010024402A1 (ja) | 2012-01-26 |
US20100054606A1 (en) | 2010-03-04 |
JP5487106B2 (ja) | 2014-05-07 |
US20110205235A1 (en) | 2011-08-25 |
US9092870B2 (en) | 2015-07-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5487106B2 (ja) | 画像処理装置および方法、ならびに、画像表示装置 | |
US9679357B2 (en) | Image processing device, and an image processing method | |
JP4727720B2 (ja) | 画像処理方法および画像処理装置 | |
JP4517872B2 (ja) | 画像処理装置、画像処理方法、画像処理方法のプログラム及び画像処理方法のプログラムを記録した記録媒体 | |
CN103745439B (zh) | 图像放大方法以及装置 | |
EP2188774B1 (en) | System and method for scaling images | |
JP4811462B2 (ja) | 画像処理方法、画像処理プログラム、画像処理装置、及び撮像装置 | |
JP2005332130A (ja) | 画像処理装置、画像処理方法、画像処理方法のプログラム及び画像処理方法のプログラムを記録した記録媒体 | |
Rajagopalan et al. | A variational approach to recovering depth from defocused images | |
CN104700360B (zh) | 基于边缘自适应的图像缩放方法及系统 | |
JP2010087614A (ja) | 画像処理装置、画像処理方法、及び、プログラム | |
WO2005109340A1 (ja) | 画像拡大装置、及びプログラム | |
Kim et al. | Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems | |
WO2012029238A1 (ja) | 画像処理システム、画像処理方法および画像処理プログラム | |
JP2008217526A (ja) | 画像処理装置、画像処理プログラム及び画像処理方法 | |
JP2014526111A (ja) | 画像のノイズ除去、圧縮および補間のための異方性勾配の正則化 | |
WO2013089261A1 (ja) | 画像処理システム及び画像処理方法 | |
CN105608684B (zh) | 双边数字图像滤波器的加速方法和系统 | |
WO2012132090A1 (ja) | 画像の正置からの回転角度検知装置および方法 | |
JP4868249B2 (ja) | 映像信号処理装置 | |
JP6316010B2 (ja) | 画像処理装置、画像処理方法 | |
JP5478200B2 (ja) | 画像処理装置、画像処理方法、および画像処理プログラム | |
JP4104475B2 (ja) | 輪郭補正装置 | |
JP4635652B2 (ja) | 画像処理装置、画像処理方法、画像処理方法のプログラム及び画像処理方法のプログラムを記録した記録媒体 | |
WO2013011797A1 (ja) | 劣化復元システム、劣化復元方法およびプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09810050 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010526796 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13061270 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 09810050 Country of ref document: EP Kind code of ref document: A1 |