JP2018022385A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus, image processing method and image processing program capable of improving the identification accuracy of the texture.SOLUTION: An image processing apparatus includes: a conversion unit which generates a plurality of sub-bands with a wavelet coefficient as an element by performing multiple resolution analysis with wavelet transformation to an input image; a setting unit which hierarchically sets dot areas being a range of the element associated with one pixel of the input image in the sub-band on the basis of the redundancy of the wavelet transformation and the decomposition level being the frequency of repetition processing of the wavelet transformation; a derivation unit which drives a model parameter of the connection probability density function with the subset of the wavelet coefficient constituted by dividing the sub-band set with the dot area as a variable number; and an identification unit which identifies the texture of each pixel by obtaining the occurrence probability density of each pixel of the input image using the derived model parameter.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  Conventionally, there is known a technique for dividing a region of an image having a texture for each texture by modeling a change in pixel values using a Markov Random Fields (MRF) in a local region of the texture. It has been. In addition, research is being conducted on a technique for modeling a feature space composed of wavelet coefficients obtained by performing wavelet transform on an image having a texture using a Gaussian MRF (GMRF). Further, as a means for analyzing an image having a texture, research is also being conducted on a technique for obtaining a wavelet coefficient that is robust against a shift of an image signal by performing wavelet transform using a wavelet frame.

JP 2005-352753 A

  However, in the conventional technique, the texture identification accuracy may not be sufficient. The present invention has been made in consideration of such circumstances, and provides an image processing apparatus, an image processing method, and an image processing program capable of improving the texture identification accuracy.

  To achieve the above object, an image processing apparatus according to an aspect of the present invention generates a plurality of subbands having wavelet coefficients of an input image as elements by performing multi-resolution analysis by wavelet transform on the input image. A conversion unit, and a dot area (region indicating an element range) associated with one pixel of the input image in the subband generated by the conversion unit, the redundancy of the wavelet transform by the conversion unit, and the conversion A setting unit that is set hierarchically based on a decomposition level that is the number of times the wavelet transform is repeated by the unit, and a subset of wavelet coefficients by dividing the subband in which the dot area is set by the setting unit To derive the model parameters of the joint probability density function with the subset as a variable And output unit, by using the model parameters derived by the deriving unit obtains the occurrence probability density of each pixel of the input image, and a recognition unit that identifies the texture of each pixel of the input image.

  According to one embodiment of the present invention, texture identification accuracy can be improved.

It is a figure which shows an example of a structure of the image processing apparatus 100 in embodiment. It is a figure which shows typically a mode that a subband is produced | generated by wavelet frame decomposition | disassembly. It is a figure which shows typically the setting process of the dot area DT. It is a figure which shows an example of the subset _Ukl ^ (j, s). It is a figure which shows typically the setting process of area _Ac ^ (j) which integrated the dot area corresponding to sample area _Ac . 5 is a flowchart showing a flow of processing of the image processing apparatus 100 in the present embodiment. It is a figure which shows an example of the input image used for experiment. It is a figure which shows the texture image of Brodaz used for experiment. It is the figure which compared the correct identification rate of the class label in each of this proposal method, and the 1st conventional method and the 2nd conventional method. It is a figure which shows an example of the input image used for experiment. It is a figure showing the area | region of four texture classes in an input image with a different density value, respectively. It is a figure which shows an example which divided the area | region of the input image for every class label in this proposal technique. It is a figure which shows an example which divided the area | region of the input image for every class label in the 1st conventional method. It is a figure which shows an example which divided the area | region of the input image for every class label in the 2nd conventional method.

  Hereinafter, embodiments of an image processing apparatus, an image processing method, and an image processing program of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a configuration of an image processing apparatus 100 according to the embodiment. The image processing apparatus 100 according to the present embodiment performs wavelet transform on the input image and identifies the texture of the input image. The texture is, for example, a pattern (change in brightness, color, etc.) representing the texture of an object, which is shown in part or all of the input image. In addition, the input image may be a partial area of an image (for example, a single pixel or a set of a plurality of pixels) or an entire area. In the present embodiment, various images including an input image are data in which one-dimensional or multi-dimensional pixel values are associated with pixels arranged in a two-dimensional grid (for example, a matrix). And For example, when the image is a digital image generated by a camera or the like, the pixel value may be a part or all of luminance, hue, brightness, and saturation. The pixel value of the input image may be regarded as a wavelet coefficient of the image or a value based on the wavelet coefficient.

  The image processing apparatus 100 includes a control unit 110 and a storage unit 130, for example. The control unit 110 includes, for example, a wavelet transform unit 112, a setting unit 114, a derivation unit 116, and a texture identification unit 118. The control unit 110 is realized by a processor such as a CPU (Central Processing Unit) executing a program stored in a program memory. Also, some or all of the components of the control unit 110 may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array).

  The storage unit 130 includes, for example, a nonvolatile storage medium such as a ROM (Read Only Memory), a flash memory, an HDD (Hard Disk Drive), and an SD card, and a volatile storage medium such as a RAM (Random Access Memory) and a register. And may be realized. The storage unit 130 stores sample area information 132 and the like in addition to the program executed by the processor described above. The sample area information 132 is information used in the process of the derivation unit 116 described later.

[Wavelet frame decomposition]
The wavelet transform unit 112 performs redundant wavelet transform on the input image. That is, the wavelet transform unit 112, among pixels included in the input image, a certain pixel of interest (hereinafter referred to as a pixel of interest) and another pixel located in the vicinity of the pixel of interest (hereinafter referred to as a neighborhood pixel). And the wavelet transform of the pixel range of the target pixel and neighboring pixels (hereinafter referred to as a frame) is derived, and the entire frame is converted into an element corresponding to the position of the target pixel in the frame. Perform processing to associate the wavelet coefficients. Redundant wavelet transform refers to generating a portion where a frame for a certain pixel of interest and a frame for another pixel of interest overlap each other more than a certain amount. The degree of frame overlap is an example of “redundancy”. Hereinafter, redundant wavelet transform will be described as “wavelet frame decomposition”. The mother wavelet used for wavelet frame decomposition may be, for example, a Haar wavelet, but is not limited thereto, and may be another mother wavelet. In the present embodiment, as an example, it is assumed that Haar wavelets are used for wavelet frame decomposition.

  For example, the wavelet transform unit 112 generates a plurality of subbands in which wavelet coefficients are associated with elements corresponding to the positions of all the pixels of the input image. In this case, even if a position shift of the input image occurs at the time of wavelet frame decomposition, wavelet coefficients corresponding to the total number of pixels of the input image are derived, so the position of the element associated with each wavelet coefficient corresponding to the position shift Just by shifting, the state of those changes does not change.

  Wavelet coefficients derived by performing wavelet frame decomposition have low-frequency components in the column direction, low-frequency components in the column direction, and low in the row direction due to the characteristics of the multi-resolution analysis of the wavelet transform. They are classified into those having a frequency component and those having a high frequency component in the row direction. Therefore, the wavelet transform unit 112 generates four types of subbands. In the present embodiment, the wavelet transform unit 112 performs wavelet frame decomposition on the input image, and with respect to one type of subband (LL of decomposition level-1 in FIG. 2) obtained as a result, Further, wavelet frame decomposition may be performed to generate multiple (two or more times) decomposed subbands. As described above, | 3J | +1 subbands are obtained by multi-resolution analysis that repeats wavelet frame decomposition. Here, J is a value (hereinafter referred to as a decomposition level) representing the total number of times that an input image has been subjected to multiple wavelet frame decomposition. In the present embodiment, the decomposition level J is represented by a negative value. For example, when wavelet frame decomposition is performed twice in multiple, the decomposition level J is −2.

  Hereinafter, subbands having wavelet coefficients of decomposition level j having low frequency components in the row direction and high frequency components in the column direction as sub-bands (j, LH). A subband having a wavelet coefficient at decomposition level j having a high frequency component in the row direction and a low frequency component in the column direction is referred to as a subband (j, HL). A subband having a wavelet coefficient of decomposition level j having a high frequency component in the row direction and having a high frequency component in the column direction is referred to as a subband (j, HH). A subband having a wavelet coefficient at decomposition level j having a low frequency component in the row direction and a low frequency component in the column direction is referred to as a subband (j, LL).

  For example, when the Haar wavelet described above is used as a mother wavelet, in the wavelet frame decomposition, in the input image, if the row direction is x and the column direction is y, it corresponds to the pixel coordinates (x, y) of the input image. The wavelet coefficient of the element can be expressed as the following formula (1).

In formula (1), u _xy ^ (J, LL) represents a wavelet coefficient in the subband (J, LL), and u _xy ^ (j, s) represents a wavelet coefficient in the subband (j, s). ing. Moreover, the wavelet coefficient represented by the above mathematical formula (1) is derived by the following mathematical formulas (2) and (3) when using the Haar wavelet.

  The asterisk symbol * shown in the mathematical expressions (2) and (3) represents an element of a set composed of the low frequency component L and the high frequency component H, that is, * ∈ {L, H}. In addition, the said numerical formula (2) and (3) shall satisfy | fill following numerical formula (4) and (5).

Here, in the above formulas (2), (3), (4), and (5), when the decomposition level on the left side is j, xy ′ and x′y on the right side are the coordinates of the pixel in the input image ( x, y + 2 ^{(| j | -1)} ) and coordinates (x + 2 ^{(| j | -1)} , y). Further, u _xy ^ (0, LL) = u _xy represents a pixel value corresponding to the pixel coordinates (x, y) in the input image. Hereinafter, a pixel or element corresponding to the coordinates (x, y) of the input image is referred to as a pixel (x, y) or an element (x, y).

  FIG. 2 is a diagram schematically illustrating how subbands are generated by wavelet frame decomposition. As illustrated, for example, a total of four subbands are generated at the decomposition level j = −1, and a predetermined subband is generated among the subbands generated at the decomposition level j = −1 at the decomposition level j = −2. The band (for example, the subband (−1, LL) from which the highest frequency component is removed) is further converted into four subbands. Therefore, at the decomposition level j = −2, the subbands at the decomposition level j = −1 (subband (−1, LH), subband (−1, HL), subband (−1, HH in the illustrated example) )) And four subbands converted from subband (-1, LL), a total of seven subbands are generated.

[Dot area setting process]
In the subband generated by the wavelet transform unit 112, the setting unit 114 hierarchically maps the dot area DT associated with one pixel of the input image based on the wavelet frame decomposition redundancy and the subband decomposition level. Set for each decomposition level. The dot area DT includes a target element corresponding to the coordinates of the target pixel that is one pixel of the input image and neighboring elements of the target element.

For example, when the wavelet frame decomposition using the Haar wavelet is performed once, the pixel value u _xy of the pixel (x, y) included in the input image has four elements {(x , Y), (x−1, y), (x, y−1), (x−1, y−1)} are used to derive wavelet coefficients. Further, when the decomposition level-1 subband is subjected to wavelet frame decomposition, four elements {(x, y), (x-1, y), (x, y-1), Each of (x−1, y−1)} is used to derive wavelet coefficients of four different elements included in the decomposition level-2 subband. That is, the pixel value u _xy of the pixel of _interest (x, y) included in the input image (decomposition level 0) is used to derive wavelet coefficients of 16 elements in the subband of decomposition level-2.

In the present embodiment, in consideration of the above-described overlapping use of the pixel value u _xy of the input image according to the decomposition level, for example, for a subband, a size (element based on the subband decomposition level j) A dot area DT ^ (j) having (number) is set.

FIG. 3 is a diagram schematically showing the dot area DT setting process. In the figure, DT ^ (-1) represents a dot area set in a subband of decomposition level-1, and DT ^ (-2) is set in a subband of decomposition level-2. This represents the dot area. For example, when the Haar wavelet is used as the mother wavelet in the wavelet frame decomposition, the setting unit 114 sets the dot area DT ^ (j in the subband corresponding to the pixel (x, y) of the input image according to a certain decomposition level j. ) Is expanded in the range of the number of elements from (x, y) to 2 ^{| j |} × 2 ^{| j |} in the minus direction of the respective axes in the row direction and the column direction.

When this embodiment is not applied, in the illustrated example, since the pixel (6, 6) in the input image is the pixel of interest, the dot area DT ^ (-1) set in the subband of decomposition level-1 is This is an area of only the target element (6, 6) corresponding to the coordinates of the target pixel (6, 6) in the input image. That is, one element is associated with one pixel. On the other hand, in the present embodiment, the dot area DT ^ (-1) is the coordinate of the pixel of interest (6, 6) in the input image because it is expanded to the range of the number of elements of 2 ^{| j |} × 2 ^{| j |} Element (6, 6) corresponding to the element (5, 6) and (6, 5) adjacent to the element of interest (6, 6), and elements (5, 6) and (6, 5). It is a region in a range where elements (5, 5) adjacent to each other are combined. Also, the dot area DT ^ (-2) set in the subband at the separation level-2 is an area in a range of a total of 16 elements (4 elements in the row direction × 4 elements in the column direction). That is, the number of elements included in the dot area DT ^ (j) set in the subband of the decomposition level j is 4 ^{| j |} . As a result, a plurality of elements are hierarchically associated with one pixel of the input image. In addition, when a mother wavelet other than the Haar wavelet is used for wavelet frame decomposition, the setting unit 114 determines the size of the dot area DT based on the frame size (decomposition redundancy) of the other mother wavelet. May be further enlarged.

The deriving unit 116 divides the subband elements in which the dot area DT is set by the setting unit 114 into subsets. Then, the derivation unit 116 derives parameters used for the model expression of the local conditional probability density function p (u _mknl ^ (j, s)) of the GMRF model for each divided subset.

[Process to divide into subsets]
For example, the derivation unit 116 _converts the wavelet coefficient u _xy ^ (j, s) of the subband (j, s) at the decomposition level j in which the dot area DT ^ (j) is set to the dot area DT ^ (j). _Are divided into the same number (for example, 4 ^{| j |} ) of subsets U _kl ^ (j, s).

FIG. 4 is a diagram illustrating an example of the subset U _kl ^ (j, s). In the example shown in the figure, the subband (−1, s) when the decomposition level j = −1 is divided into four subsets U ₀₀ ^ (− 1, s) to U ₁₁ ^ (− 1, s). ing. As shown in the gray squares of (a) to (d) in the figure, the elements included in the respective subsets U _kl ^ (− 1, s) do not overlap with the elements included in the other subsets. It is divided into. When the decomposition level is j, the deriving unit 116 extracts elements from the reference elements (for example, any one of the four corners) every (2 ^{| j |} -1) to form respective subsets. For example, the deriving unit 116 extracts every other element from the reference elements at the decomposition level-1, and extracts every third element from the reference elements at the decomposition level-2. As described above, the derivation unit 116 may determine an interval of thinning out elements at the time of forming a set according to the decomposition level j so that the elements do not overlap between the subsets U _kl ^ (j, s). For example, the larger the decomposition level j of the sub-band (j, s) to be divided (the larger the absolute value), the wider the interval for thinning out elements at the time of forming a set, and the smaller the decomposition level j (the smaller the absolute value). However, the same number of subsets U _kl ^ (j, s) as the number of elements in the dot area DT ^ (j) may be formed with a tendency to narrow the thinning-out interval at the time of forming the set. Note that it is not necessary to divide all elements included in the subset U _kl ^ (j, s) so as not to overlap with elements included in other subsets, so that some elements overlap each other. It may be divided.

Further, the thinning-out interval of elements may be changed according to the type of mother wavelet. For example, a mother wavelet with a larger frame (a larger number of pixels to be duplicated when decomposing wavelet frames) has a wider interval for thinning out elements when forming a set, and a mother wavelet with a smaller frame (smaller number of pixels to be duplicated when decomposing wavelet frames) As the number of sub-sets U _kl ^ (j, s) equal to the number of elements included in the dot area DT ^ (j) at the decomposition level j may be formed with a tendency to narrow the thinning-out interval at the time of forming the set. . Note that the number of elements included in the dot area DT ^ (j) and the number of the subsets U _kl ^ (j, s) are not necessarily the same and may be different.

For example, each of the subsets U _kl ^ (j, s) divided into 4 ^{| j |} pieces can be expressed by the following formula (6).

Here, m _k = m · 2 ^{| j |} + k, n _l = n · 2 ^{| j |} + l, (m, n = 0, 1,...). Subset _U kl ^ (j, s) in the, some element of interest _(m k, _{n l)} 2 order neighborhood of, i.e., a set of neighboring elements of eight surrounding the element of interest _(m k, _{n l)} from the periphery thereof When T _kl ^ (j, s) is assumed, the probability density function p (u _mknl ^ (j, s)) for the subset U _kl ^ (j, s) is expressed by equations (7) and (8). Can be represented. These mathematical expressions indicate that the occurrence probability of the element of interest (m _k , n _l ) follows a local conditional probability distribution that depends only on its neighboring elements T _kl ^ (j, s).

In Equations (7) and (8), μ _kl ^ (j, s) is a parameter representing the average of the wavelet coefficients of all elements included in the subset U _kl ^ (j, s). Further, (σ _kl ^ (j, s)) ² is a parameter representing the dispersion of wavelet coefficients of all elements included in the subset U _kl ^ (j, s). Β _{kl, τ} ^ (j, s) is a parameter representing an interaction between elements included in the subset U _kl ^ (j, s). The interaction represents, for example, an action that the element of interest (m _k , n _l ) receives from each element τ of the set of neighboring elements T _kl ^ (j, s).

[Model parameter derivation process]
The deriving unit 116 calculates the average μ _kl ^ (j, s), variance (σ _kl ^ (j, s)) ² , and interaction β _{kl, τ} ^ (j, s) shown in Equation (7). One parameter is derived from each of, for example, 4 ^{| j |} subsets U _kl ^ (j, s) divided. Hereinafter, a set of these three parameters (μ _kl ^ (j, s), (σ _kl ^ (j, s)) ² , β _{kl, τ} ^ (j, s)) is combined into one parameter Θ _{kl. , C} ^ (j, s).

For example, the derivation unit 116 assumes a joint probability density function p (U _kl ^ (j, s)) having each subset U _kl ^ (j, s) as a variable, and this combined probability density function p (U _kl). ^ (J, s)) is approximated to a pseudo-likelihood function (quasi-likelihood function), and the joint probability density function p (U _kl ^ (j, s)) approximated to the pseudo-likelihood function is maximized. In this way, the parameter Θ _{kl, c} ^ (j, s) is derived.

[Binding probability density function]
Hereinafter, the joint probability density function p (U _kl ^ (j, s)) used for derivation of the parameter Θ _{kl, c} ^ (j, s) will be described. The derivation unit 116 assumes that each subset U _kl ^ (J, LL) and each subset U _kl ^ (j, s) are independent from each other, thereby selecting one part from each subband. A set of sets (U _i ^ (J, LL),..., U _i ^ (j, s),..., U _i ^ (− 1, HH)) = U _i (i = 1, 2,..., I) Is composed of I combinations. Here, U _i ^ (J, LL) ∈ {U _kl ^ (J, LL)} and U _i ^ (j, s) ∈ {U _kl ^ (j, s)} are established. In addition, I is represented by, for example, 4 ^{| J |} · _{ｊ j} (4 ^{| 3j |} ). The deriving unit 116 assumes joint probability density functions p (U _kl ^ (J, LL)) and p (U _kl ^ (j, s)) each having a subset in each subband as variables, By taking the sum of the relevant joint probability density functions with a subset selected from each subband as a variable, I kinds of joint probability density functions p (U _i ) are defined. The following mathematical formula (9) is a defining formula representing the joint probability density function p (U _i ).

The joint probability density function p (U _kl ^ (j, s)) having each subset U _kl ^ (j, s) as a variable is defined by a pseudo likelihood function, for example. Equation (10) represents a pseudo-likelihood function of the joint probability density function p (U _kl ^ (j, s)).

[Sample area settings]
On the other hand, in the present embodiment, an input image to be subjected to wavelet frame decomposition has a pixel area (hereinafter referred to as sample area A) in which a texture identification class (hereinafter referred to as texture class c) is estimated or designated in advance. _c )) is set. For example, the sample area _Ac may be set by designating coordinates represented by the row direction x and the column direction y in the input image. Position information of the sample area A _c with respect to the input image may be pre-stored in the storage unit 130 as a sample area information 132.

For example, when there are C texture classes c to be identified, in the present embodiment, a sample area _Ac is set for each of the texture classes c (c = 1, 2,..., C).

The range of sample area A _c is deriving unit 116 may set. For example, the derivation unit 116 applies the above-described GMRF model and the division and integration method to roughly divide the input image into several small regions, and uses the model parameters derived from these small regions to calculate those small values. The sample area _Ac may be set adaptively in a region where the regions are integrated. The division integration method is a method of dividing one image into a plurality of regions having similar characteristics, and may be performed based on, for example, a method described in a prior art document.

The regions within each sub-band corresponding to the sample area A _c, referred to as sample area A _{c ^} (j). Since the sample area A _c ^ (j) is an area set by integrating the dot areas DT ^ (j) corresponding to the pixels of the sample area A _c of the input image, the range of the area is larger than the sample area A _c. Expands. 5, the setting processing of the sample corresponding to the sample area A _c area A _{c ^} (j) is a diagram schematically showing, represent elements of the sample area in gray boxes. As shown in the figure, the sample area A _c ^ (− 1) at the decomposition level j = −1 is a union of elements in the dot area DT ^ (− 1) corresponding to each pixel included in the sample area A _c. It is the area of the corresponding range. Sample area A _c (−2) at decomposition level j = −2 is an area in a range corresponding to the union of elements in dot area DT (−2) corresponding to each pixel included in sample area A _c It is.

Under such a premise, a set of all elements included in the dot area DT ^ (j) corresponding to the pixel of interest (x, y) of the input image in each subband of decomposition level j is represented by _Bxy ^. Define (j). Here, B _xy ^ (j) is, for example, {(x−b ₁ ^ (j), y−b ₂ ^ (j)) | b ₁ ^ (j) ∈ {0,..., 2 ^{| j |} − 1}, b ₂ ^ (j) ε {0,..., 2 ^{| j |} -1}}. The subband sample area A _c ^ (j) corresponding to the sample area A _c which is a partial area of the input image is defined as the following equation (11) at the decomposition level j.

The derivation unit 116 uses the subset of wavelet coefficients that are elements of the sample area A _c ^ (j) expanded according to the decomposition level j shown by the above equation (11), and performs parameter Θ by the maximum likelihood method. _{Deriving kl, c} ^ (j, s). That is, the derivation unit 116 maximizes the joint probability density function p (U _kl ^ (j, s) | (m _k , n _l ) ∈A _c ^ (j)) shown in Equation (10). Obtain an estimate of the parameter Θ _{kl, c} ^ (j, s). Accordingly, the derivation unit 116 applies the derived parameter Θ _{kl, c} ^ (j, s) to the above-described equation (9), thereby obtaining the joint probability density function of the equation (9) for the sample area A _c . . Hereinafter, the parameter Θ _{kl, c} ^ (j, s) derived using the wavelet coefficients of the sample area A _c ^ (j) will be referred to as an identification model parameter. The identification model parameter may be stored in the storage unit 130 as part of the sample area information 132.

For example, when a certain pixel belongs to the texture class c = 1, the identification parameter Θ _{kl, c = 1} ^ (j derived (estimated) from the sample area A _{c = 1} of the texture class c = 1 , s) values of the probability density of the pixels is calculated using the value of (below in equation (12)) is derived from the sample area a _{c ≒ 1} other texture class c ≒ 1 (estimated) It becomes larger than the value of the occurrence probability density of the pixel calculated in the same manner using a set of parameters for identification.

[Texture identification processing]
The texture identification unit 118 identifies the texture class c of the pixel of the input image based on the occurrence probability density of each pixel calculated using the parameters derived by the derivation unit 116. For example, the texture identification unit 118 uses the parameter set Θ _{i, c} derived for each sample area A _c to determine the occurrence probability density value (related to the texture class c) of the pixel of interest (x, y) to be identified. The occurrence probability density of the pixel) is calculated for each texture class c, and the parameter set Θ _{i, c} having the highest occurrence probability density is specified. Then, the texture identification unit 118 determines the texture class c corresponding to the specified parameter set Θ _{i, c} as the texture class of the pixel of interest (x, y). Depending on the determination of the texture class of the pixel of interest (x, y), the class label w _xy of the texture class is _assigned . The class label of each texture class may be associated with the parameter Θ _{kl, c} ^ (j, s) derived by the deriving unit 116 and stored in the storage unit 130 as a part of the sample area information 132, for example. .

For example, when expressed as a set of theta _{i, c} of the model parameters for a subset _{U i} with respect to a certain texture class _{c, Θ i, c = (} Θ i, c ^ (J, LL), ..., Θ i _{, C} ^ (-1, HH)). Where Θ _{i, c} ^ (J, LL) ∈ {Θ _{kl, c} ^ (J, LL)} and Θ _{i, c} ^ (j, s) ∈ {Θ _{kl, c} ^ (j, s) }.

When the set of model parameters is expressed as described above, the class label w _xy of the pixel of interest (x, y) is the element of the dot area of the pixel shown in Equation (12) using the parameter set Θ _{i, c} Based on the value of the combined probability density function (occurrence probability density of the pixel of interest) defined by the square of the probability density function with σ as a variable, it is determined by Equation (13).

In Equation (12), ε _J and ε _j are coefficients representing weights for the joint probability density for each of the maximum decomposition level J and a certain decomposition level j. In Expression (12), by using the wavelet coefficients of all the elements included in the dot area DT ^ (j) expanded according to the decomposition level j, an isolated pixel (isolated point) whose class label is different from the surrounding pixels. ) Is suppressed.

  The texture identification unit 118 may identify the texture class c of the pixel of the input image based on the occurrence probability density of the pixel of interest calculated using the identification model parameter stored in advance in the storage unit 130. Good.

  FIG. 6 is a flowchart showing a processing flow of the image processing apparatus 100 according to the present embodiment. The processing of this flowchart may be repeatedly performed at a predetermined cycle, for example.

  First, the wavelet transform unit 112 sets a processing target for wavelet frame decomposition (step S100). For example, the wavelet transform unit 112 sets a digital image generated by a camera or the like as a processing target. This digital image may be stored in advance in the storage unit 130 or may be acquired via a predetermined interface (not shown) at the start of processing.

  Next, the wavelet transform unit 112 performs wavelet frame decomposition on the set processing target (step S102), and generates | 3J | +1 subbands (step S104).

  Next, the setting unit 114 sets a dot area DT ^ (j) associated with one pixel of the input image for each of | 3J | +1 subbands generated by the wavelet transform unit 112 (step S106).

Next, the derivation unit 116 has the same number of elements in the subband in which the dot area DT ^ (j) is set by the setting unit 114 as the number of elements in the dot area DT ^ (j) (for example, 4 ^{| j |} ) To a subset U _kl ^ (j, s) (step S108).

Next, the deriving unit 116 derives the model parameter Θ _{kl, c} ^ (j, s) from the elements of the sample area A _c ^ (j) for each of the divided subsets U _kl ^ (j, s). (Step S110).

Next, the deriving unit 116 sets a set of parameters Θ _{i, c} of the joint probability density function p (U _i ) with a set of I sets of subsets U _i as variables (step S112).

Next, the texture identification unit 118 determines the pixel of interest (x, y) based on Expression (13) using the parameter set Θ _{i, c} of the joint probability density function p (U _i ) derived by the derivation unit 116. ) To identify the texture class c of the pixel (step S114). The processing of this flowchart is terminated by identifying all the target pixels.

[Verification example]
The applicant of the present application conducted the following experiment, and verified the identification accuracy of the texture of the input image. FIG. 7 is a diagram illustrating an example of an input image used in the experiment. As shown in the figure, the input image used in the experiment has four types of textures, and is represented by, for example, an 8-bit grace scale of 384 × 384 pixels. Further, as shown in the figure, a sample area _Ac is set for each region having a different texture. FIG. 8 is a diagram showing a Brodazz texture image used in the experiment. The illustrated image was obtained from the Southern California University image database site, and 715 input images were prepared by combining 13 Brodazz texture images. Note that these texture images are adjusted so that the average luminance value is the same level.

In the identification method proposed in this embodiment and the conventional identification method, the class label is determined after unifying the subscript _kl of the model parameter Θ _{kl, c} ^ (j, s) of each subband for each decomposition level. . The conventional identification method is, for example, a method that does not enlarge the dot area DT in the texture identification method in the present embodiment (hereinafter referred to as the first conventional method) and a subband wavelet coefficient that has no redundancy. And a method of deriving one GMRF model and identifying each pixel in a class having the highest occurrence probability density (hereinafter referred to as a second conventional method). The second conventional method is based on the method described in Reference Document 1, for example. In the second conventional method, the dot area is not enlarged and the subband elements are not divided.
[Reference 1] Nurishiraji Mahadad, Nobuaki Takao, Hideki Noda: "Texture recognition using MRF model in wavelet feature space", IEICE D-II, J83-D-II, 10, pp.1995-2002 (2000)

For example, when the subscripts _{kl of} the model parameters Θ _{kl, c} ^ (j, s) are unified, Equation (14) holds when j> J.

Further, when the subscripts _{kl of} the model parameters Θ _{kl, c} ^ (j, s) are unified, Equation (15) is established when j = J.

Based on the above formulas (14) and (15), combinations of model parameters for obtaining class labels are limited to Π _j (4 ^{| j |} ). However, as described above, k, lε {0, 1,..., 2 ^{| j |} −1}. Further, in the proposed method, ε _{j is set} to the following formula (16) using ln (p (u _xy | Θ _{i, c} )) which is a logarithm of the formula (12) in the determination of the class label. Thereby, it is possible to suppress the occurrence of calculation errors such as underflow during the calculation.

As for ε _J , a value obtained by replacing j in Equation (16) with J was used. In this case, the logarithm of Expression (12) can be expressed as Expression (17).

  In any method, the region of the experimental image is divided by assigning one of the four class labels to each pixel of the input image. Moreover, about the decomposition | disassembly level, it verified about two cases to -2 and -3. The class label was not corrected by post-processing such as expansion and contraction.

  FIG. 9 is a diagram comparing the correct classification rates of class labels in the proposed method, the first conventional method, and the second conventional method. The correct identification rate is a ratio of the number of pixels to which a class label is correctly assigned in one input image. In the figure, “highest” indicates the value of the input image having the highest positive identification rate among the input images in all 715 images, and “lowest” indicates the most positive identification in the input images in all 715 images. The value of the input image having a low rate is represented, and “average” represents the average of the correct identification rates of all input images. As shown in the figure, the proposed method has the best correct recognition rate in any result. Further, when the decomposition level was set to -3 rather than -2, the average positive identification rate was higher. On the other hand, in the second conventional method, there is relatively no difference in the average of the positive identification rates depending on the decomposition level.

  Here, in the result of the proposed method with J = −3, an input image (No. 413) having the highest correct identification rate and an input image (No. 364) having the lowest correct identification rate are obtained. 10 (a) and 10 (b), respectively. FIG. 11 is a diagram in which four texture class regions in the input image are represented by different density values. FIG. 11 shows the result of area division when the correct identification rate is 100%. The four types of class labels assigned by the methods of J = -3 to the input image (No. 413) and the input image (No. 364) shown in FIG. When represented by density values, each image is as follows.

  FIG. 12 is a diagram showing an example in which the input image is divided into regions for each class label in the proposed method. FIG. 13 is a diagram illustrating an example in which an input image is divided into regions for each class label in the first conventional method. FIG. 14 is a diagram showing an example in which an input image is divided into regions for each class label in the second conventional method.

  As shown in FIGS. 12 and 13, the proposed method for enlarging the dot area DT has fewer isolated points than the first conventional method, and each divided region may be grouped. Recognize. In particular, image no. In the result of the proposed method for 413 (FIG. 12A), the four areas are more clearly divided than the result of the first conventional method (FIG. 13A). On the other hand, No. 1 shown in FIG. The input image 364 is considered to have similar spatial frequency distributions in the upper left and lower right regions and the upper right and lower left regions in the figure. Therefore, the image No. 1 shown in FIG. In the result of the proposed method for 364, many pixels are classified into two groups that almost match the combination of these regions, and the number is higher than that of the other methods. Compared with the result of the proposed method for 413 (FIG. 12A), the correct identification rate was lower. A similar tendency can be seen for the result of the first conventional method (FIG. 13B). In the second conventional method, no. 413 and no. For both input images of 364, the division accuracy for each texture was relatively low compared to other methods.

  According to the embodiment described above, by performing wavelet frame decomposition on the input image, the wavelet transform unit 112 that generates a plurality of subbands having the wavelet coefficients of the input image as elements, and the wavelet transform unit 112 generate the subbands. In the subband, a setting unit 114 that hierarchically sets the dot area DT associated with one pixel of the input image based on the redundancy of the wavelet frame decomposition and the decomposition level that is the number of repetitions of the wavelet frame decomposition. And a derivation unit 116 for deriving a model parameter of a joint probability density function using the subset as a variable by dividing the subband in which the dot area DT is set by the setting unit 114 to form a subset of the wavelet coefficients. And the model derived by the deriving unit 116 A texture identification unit 118 that identifies the texture of each pixel by determining the occurrence probability density of each pixel of the input image using a set of parameters, and thereby assigns a class label to the pixel of the input image to provide a texture. The occurrence of isolated points can be suppressed when identifying. As a result, texture identification accuracy can be improved.

  Further, according to the above-described embodiment, the dot area DT is enlarged based on the number of pixels (redundancy) to be overlapped at the time of wavelet frame decomposition and the number of wavelet frame decomposition processes (decomposition level) when class labels are identified. Thus, since the wavelet coefficients of all the elements in the enlarged dot area DT are associated with one pixel in the input image, the generation of isolated points can be further suppressed.

  Further, according to the above-described embodiment, it is possible to eliminate the need for class label correction by post-processing such as expansion and contraction for an image whose texture is identified by the proposed method. As a result, the proposed method shown in the above-described embodiment can be easily applied even when the texture class is 3 or more.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, In the range which does not deviate from the summary of this invention, various deformation | transformation and substitution Can be added.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 110 ... Control part 112 ... Wavelet transformation part 114 ... Setting part 116 ... Derivation part 118 ... Texture identification part 130 ... Storage part 132 ... Sample area information

Claims (16)

By performing multi-resolution analysis by wavelet transform on the input image, a conversion unit that generates a plurality of subbands having wavelet coefficients of the input image as elements, and
In the subband generated by the conversion unit, the dot area that is the range of the element corresponding to one pixel of the input image is set to the redundancy of the wavelet conversion by the conversion unit, and the repetition of the wavelet conversion by the conversion unit. Based on the decomposition level that is the number of times of processing, a setting unit that sets hierarchically,
A derivation unit for deriving a model parameter of a joint probability density function using the subset as a variable by dividing the subband in which the dot area is set by the setting unit to form a subset of wavelet coefficients;
An identification unit for identifying the texture of each pixel of the input image by obtaining the occurrence probability density of each pixel of the input image using the model parameter derived by the deriving unit;
An image processing apparatus comprising: The setting unit is an element in the subband, and sets the dot area including a target element corresponding to coordinates of one pixel of the input image and a neighboring element of the target element.
The image processing apparatus according to claim 1. The setting unit increases the number of the neighboring elements and increases the dot area as the decomposition level increases.
The image processing apparatus according to claim 2. The setting unit increases the neighboring elements in one direction of each of the two-dimensional axes in the subband based on the element of interest in the subband.
The image processing apparatus according to claim 3. The neighboring elements include a number of elements corresponding to a value obtained by subtracting the number of the elements of interest from a value obtained by subtracting the number of the elements of interest from a value having a base of 2 and a product of 2 and the value of the decomposition level as a power index.
The image processing apparatus according to claim 3 or 4. The converter is
The change in pixel value between pixels is converted into a wavelet coefficient having a frequency component for each of the two-dimensional axial directions of the input image,
A wavelet coefficient having a low frequency component in one direction of the axial direction and a high frequency component in the other direction of the axial direction has a first subband associated with the pixel, and a high frequency in the one direction of the axial direction. A second subband having a component and a wavelet coefficient having a low frequency component in the other direction of the axial direction associated with the pixel, and a wavelet coefficient having a high frequency component in the bidirectional direction of the axial direction. And a fourth subband in which a wavelet coefficient having a low frequency component in both directions in the axial direction is associated with the pixel.
The image processing apparatus according to claim 1. The derivation unit divides at least one of the subbands generated by the conversion unit into a subset of a number corresponding to a decomposition level that is the number of repetitions of the wavelet transform by the conversion unit, and the divided Deriving model parameters of the joint probability density function with a subset as a variable,
The image processing apparatus according to claim 1. The model parameters include mean, variance, and interaction between elements of wavelet coefficients that are elements of the subset divided by the derivation unit.
The image processing apparatus according to claim 1. The model parameter includes a model parameter stored in a storage unit,
The image processing apparatus according to claim 1. The joint probability density function having the subset as a variable is a function having a variable of one element of the subset, and approximated by the power of the probability density function defined by the model parameter.
The image processing apparatus according to claim 1. The joint probability density function with the subset as a variable is a joint probability density function with the subset set as a variable by taking the sum of two or more of the joint probability density functions.
The image processing apparatus according to claim 1. The identification unit is a wavelet coefficient included in the dot area corresponding to one pixel of the input image instead of the subset which is a variable of the joint probability density function having the model parameter derived by the derivation unit. The set of wavelet coefficients belonging to the same type of subband as the subband to which the subset belongs is used as a variable of the joint probability density function to calculate the occurrence probability density for the one pixel, and the one pixel Identify the texture of the
The image processing apparatus according to claim 1. The transform unit generates a plurality of subbands having the wavelet coefficients of the input image as elements by performing multi-resolution analysis by wavelet transform on an input image including pixels for which a texture is designated,
In the subband generated by the conversion unit, the setting unit sets a dot area that is a range of the element to be associated with a pixel for which the texture is designated, and the redundancy of the wavelet transform by the conversion unit, and the conversion unit Hierarchically set based on the decomposition level that is the number of iterations of the wavelet transform by
The derivation unit is a subband in which the dot area is set by the setting unit, and forms a subset of wavelet coefficients by dividing the subband of the input image including pixels in which the texture is specified. Then, a model parameter of a joint probability density function with the subset as a variable is derived, and the derived model parameter is stored in the storage unit in association with the texture type of the pixel for which the texture is specified.
The image processing apparatus according to claim 1. The derivation unit includes: an element included in the dot area associated with the pixel for which the texture is designated among elements of the subset obtained by dividing the subband of the input image including the pixel for which the texture is designated. Deriving model parameters of joint probability density function with set as variable,
The image processing apparatus according to claim 13. Computer
By performing multi-resolution analysis by wavelet transform on the input image, a plurality of subbands having the wavelet coefficients of the input image as elements are generated,
In the generated subband, the dot area that is a range of the element corresponding to one pixel of the input image is based on the redundancy of the wavelet transform and the decomposition level that is the number of repetitions of the wavelet transform. Set hierarchically,
By dividing the sub-band in which the dot area is set to form a subset of wavelet coefficients, a model parameter of a joint probability density function with the subset as a variable is derived,
Identifying the texture of each pixel of the input image by determining the occurrence probability density of each pixel of the input image using the derived model parameter;
Image processing method. On the computer,
By performing multi-resolution analysis by wavelet transform on the input image, a plurality of subbands having the wavelet coefficients of the input image as elements are generated,
In the generated subband, a dot area that is a range of the element corresponding to one pixel of the input image is determined based on the redundancy of the wavelet transform and the decomposition level that is the number of repetitions of the wavelet transform. , Set hierarchically,
By dividing the sub-band in which the dot area is set and forming a subset of wavelet coefficients, the model parameter of the joint probability density function with the subset as a variable is derived,
By identifying the occurrence probability density of each pixel of the input image using the derived model parameter, the texture of each pixel of the input image is identified.
Image processing program.