JP2000216979A - Image extraction method and device and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To highly accurately and quickly extract a subject with various images. SOLUTION: When the dispersion value of concentration values is a threshold or higher in the neighborhood of a reference point, it is judged that an image pattern has texture priority characteristics, and when the dispersion value is less than the threshold, it is decided that changes in the concentration values is smooth (S11). When the dispersion value is the threshold or higher (S11), a texture featured value is loaded (S12), and a featured value suitable for area growth (for example, texture energy) is calculated (S13), and a distance DT related to the texture featured value is calculated (S14). When the dispersion value is less than the threshold (S11), a color component featured value is loaded (S16), and a feature value suitable for area growth (for example, a featured vector calculated from local histogram related to the color components or the list of central color components or the like) is extracted (S17), and a distance DC related to the color components is calculated (S18). The area growth based on the corresponding featured value is operated (S15, S19), and an update position based on the new boundary line of a core area obtained by unifying the area growth results from each reference point is decided (S20).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image extracting method and apparatus, and a storage medium, and more specifically, to a method and apparatus for extracting or detecting a desired figure or pattern of an object separately from a background from an image. And a storage medium for storing program software of the method.

[0002]

2. Description of the Related Art Conventionally, as a method of cutting out a specific subject from an image, a region having a predetermined range of color component values (or shading values) including a pixel value of a point on an object or a background designated by a user is selected. A method of removing a background or repeatedly specifying an area of an extraction subject (JP-A-8-7107, JP-A-8-16779, JP-A-7-334675, and JP-B-7-34219, etc.); A method in which a rough rough contour region or a local region including a contour to be extracted is designated, and the designated region is thinned or clustered to obtain and cut out a target contour
-240884, JP-A-7-225847, JP-A-6-251149, JP-A-7-107
266, JP-A-8-7075, JP-A-8-7075
No. 77336, Japanese Patent Publication No. 6-42068, and Japanese Patent Publication No. 8-20725).

A method of generating a cut-out mask image that is almost similar to the shape of an object only by setting a closed curve (or a polygonal boundary line) so as to roughly surround an image portion to be cut out by using only color components (Patent 264236
No. 8 and Japanese Patent Application Laid-Open No. 7-322054) are also known.

Further, the input image is divided into regions of similar colors or regions having similar spatial frequencies, and the user performs an operation of selecting an image region including a region to be extracted from the divided regions, thereby obtaining a desired image region. (See JP-A-6-149959 and JP-A-5-21)
6992 and JP-A-5-35868).
Are known.

[0005]

A method of designating a region to be extracted while selecting a region having a predetermined range of color component values is performed in a case where a subject is locally composed of regions having many color components, or When the boundary between the background and the subject is low in contrast and cannot be easily distinguished only by pixel characteristics, it takes a great deal of trouble (designation of a color component selection range and designation of a point in an extraction target). As described in Japanese Patent Publication No. 7-34219, even if the operability is improved by providing a means for arbitrarily changing the size of the extraction area, the background is extracted in principle only with the grayscale values. In some cases, it is difficult to extract a target subject separately.

In the method of specifying a rough outline including an outline to be extracted, when the width of the outline is small, the coarse outline is taken with great care so as not to protrude from the true outline. Contour lines must be set. Further, when the width of the rough contour is large, even if the contour of the target object is obtained by thinning or clustering, if the boundary of the target object has a low contrast, and the contour of the target within the coarse contour is complicated. It is difficult to extract a true contour when the image has a complicated shape. A method of successively extracting a boundary line by designating a local region and moving the local region along an outline to be extracted while performing region division or boundary determination therein (Japanese Patent Publication No. 6-42068). JP, JP-B-8-20725 and JP-A-6-251149
Still require considerable attention and effort in user operations.

In the method of designating a closed curve (a region roughly including) that roughly encloses an image portion to be cut out, a method based on the area ratio of a region having the same color component included in the closed curve (Japanese Patent No. 2642368) is also used. When there is an area in the background within the closed curve that is the same color as the subject, and
There is a problem that erroneous extraction such as extraction of a background portion is liable to occur when the closed curve region has an area twice or more as large as the cutout target region.

A method in which a user selects an area to be extracted from the result of area division based on color components and area division based on spatial frequency components in advance (JP-A-6-149959, JP-A-5-216992 and JP-A-5-35
868), when the optimal region division is not performed, the number of divided small regions belonging to the region to be extracted increases, and the user's labor for selection increases. If the boundary line of the area division does not match the contour to be extracted,
It takes too much work to fix. Further, since the calculation is performed on the entire image, the processing load may be unnecessarily excessive due to the region division in the background part which is originally unnecessary, and the result cannot be obtained at high speed.

An object of the present invention is to provide an image extracting method and apparatus and a storage medium which solve such inconveniences.

[0010] That is, an object of the present invention is to provide an image extraction method and apparatus and a storage medium which can extract a desired image with high speed and high accuracy.

[0011]

According to the image extracting method of the present invention, a nucleus specified by at least one of a predetermined point, a line segment and an area is set in an image, and at least one nucleus constituting the nucleus is set. In a local region including two reference points, an evaluation function including a texture feature amount is calculated, and a boundary of a predetermined region in the image is determined by updating the shape of the nucleus based on the value of the evaluation function. And

[0012] An image extracting apparatus according to the present invention comprises a setting means for setting a nucleus specified by at least one of a predetermined point, a line segment, and a region in an image, and at least one reference point constituting the nucleus. Evaluation function calculating means for obtaining a function value of a predetermined evaluation function including at least one of a feature amount of a color component and a texture feature amount in a local region, and updating the shape of the nucleus based on the function value of the evaluation function A boundary line updating unit that determines a boundary line of a predetermined area in the image.

With such a configuration, generally, it is possible to collectively extract a region to be extracted from an image composed of various patterns and color components with a small amount of calculation and with high accuracy.

[0014] The image extracting method according to the present invention further includes a nucleus setting step of setting a nucleus specified by at least one of a predetermined point, a line segment, and a region in the image, and a representative point set on the nucleus. A reference point setting step with the reference point as
A first distance between two points in the feature space based on the feature amount of the color component for the point of interest and the reference point given by a predetermined method, and a first distance between the two points in the feature space based on the texture feature amount A similarity calculating step of calculating a similarity between the neighboring area image of the reference point and the neighboring area image of the target point based on the feature distance obtained by weighting and adding the second distance;
If the similarity is greater than a predetermined threshold, the point of interest or a nearby area is merged into a predetermined extraction area including a reference point, and if the point of interest is equal to or less than the threshold and the point of interest is included in the extraction area, By deleting from the extraction area,
The method is characterized by comprising an extraction area updating step of updating a boundary line of the extraction area, and a boundary line displaying step of displaying a boundary line of the updated extraction area.

With this configuration, it is possible to relatively evaluate the superiority of the color component characteristic amount and the superiority of the texture component characteristic amount, and generate the necessary and optimal characteristic amount for determining the boundary of the extraction region. it can. As a result, it is possible to automatically perform image extraction while minimizing user's operations.

The evaluation function is expressed as a function of a predetermined co-occurrence probability value related to the texture feature amount in the vicinity of the reference point, and the reference point is moved by a predetermined amount in a direction in which the function value of the evaluation function is maximized. By doing so, the shape of the nucleus is updated. As a result, it is possible to efficiently and accurately determine the boundary of the pattern (texture) that forms the boundary of the extraction target.

The evaluation function is a predetermined energy function including a term giving a minimum value at the boundary of the texture feature. This provides a dynamic contour model that applies the so-called active contour method in the texture feature space and automatically converges to the contour even when the contour to be extracted corresponds to the texture boundary. (Or an outline) can be automatically extracted.

The extraction area updating step includes a threshold updating step of increasing or decreasing the threshold value by a predetermined method every time the boundary line is updated. Thus, a desired target image can be automatically or semi-automatically extracted from an image including various patterns and color components.

The threshold updating step includes an edge ratio calculating step of calculating a ratio of a point on a predetermined boundary line on the extraction area as an edge point, and the edge ratio calculating step based on one of a value of the edge ratio and a change rate thereof. Update the threshold. Thus, a desired target image can be automatically extracted from an image including various patterns and color components.

When the value of the edge ratio reaches a predetermined reference value or more, or when the absolute value of the change rate of the edge ratio reaches a predetermined reference value or less, the extraction area update processing is automatically stopped. Thus, a desired target image can be automatically extracted from an image including various patterns and color components.

The threshold value is updated based on a signal from a predetermined instruction selecting means. This makes it possible to directly and directly reflect the user's intention and obtain the extraction processing result easily and reliably.

The first distance is a weighted value relating to one of the difference value between the value at the reference point and the value at the point of interest for each color component separated into a plurality of components and the absolute value of the difference. The second distance is represented by a power sum.
It is represented by a weighted power-sum of one of the difference between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components and the absolute value of the difference. This makes it possible to appropriately evaluate the difference between the feature amounts of the color component and the texture feature component at the reference point and the point of interest. Further, it is possible to increase the calculation efficiency for detecting the difference between the reference point and the point of interest regarding the texture feature component.

The weighting factor for weighting and adding the first distance and the second distance is determined based on the variance of gray values near the reference point. This allows
It is possible to automatically discriminate a portion where the texture feature amount is superior in the image and a portion where the color or luminance level is relatively uniform, and to give an optimal evaluation function to each of them. ) Can be improved in detection accuracy.

[0024]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a schematic block diagram showing an embodiment of the present invention. Reference numeral 10 denotes an image input device, 12 denotes an image storage device, 14 denotes an image display device, 16 denotes an image processing device, and 18 denotes an instruction selecting device including a pointing device such as a mouse. The image input device 10 includes an image pickup device such as an image scanner or a digital camera, or an image data transfer device connected to an image database.

In this embodiment, image data including an object to be separated and extracted from the background is input from the image input device 10 and stored in the image storage device 12. Image storage device 12
Is displayed at a predetermined position on the screen of the image display device 14 in a predetermined format.

FIG. 2 shows a schematic block diagram of the image processing apparatus 16. 20 is a kernel setting device, 22 is a texture feature amount extracting device, 24 is a color component feature amount extracting device, 26 is a feature amount distance calculating device for calculating a feature amount distance in a feature space,
28 is a region growing device for growing a region based on the feature distance, 30 is a boundary line updating device for obtaining a boundary line of a nucleus obtained as a result of region growing, 32 is a mask data generating device, and 34 is an extracted image for generating image data. It is a generating device. Although not shown, the image processing device 16 includes a storage device that temporarily stores data being processed.

In this embodiment, the "nucleus" refers to a seed that grows a line segment including a point or a curve or a region of an arbitrary shape. 3A, 3B, and 3C show an example of an initial nucleus. FIGS. 3A and 3C show an example in which the user designates with the instruction selecting device 18 while looking at the image displayed on the screen of the image display device 14, and FIG. An example in which four sides of an outer frame of a target image are nuclei will be described. The user may specify an arbitrary side (a plurality of sides) of the four sides constituting the frame of the frame.

FIG. 4 shows an operation flowchart of this embodiment. The operation of the present embodiment will be described with reference to FIG.

Image data is input from the image input device 10 to the image storage device 12, and displayed on the screen of the image display device 14 (S1). Looking at the displayed input image, the user
A nucleus is set (S2). For example, when setting the nucleus in a curved shape, the user sets a curve of an arbitrary shape using the instruction selecting device 18 so as to surround the target to be cut out in the image displayed on the image display device 14. This curve may be a closed curve surrounding the extraction target or a curve of any shape near the extraction target and not closed. Specifically, while checking the position of a cursor (not shown) displayed on the image display device 14, an instruction selection device (mouse) 18
When the cursor is moved on the screen so as to draw a locus of a curve surrounding the extraction target while pressing the button, a smooth curve connecting the sampling points obtained by sampling points on the locus at predetermined intervals is generated. This is the core of the early days. In addition, when the mouse is clicked at any of a plurality of points outside the extraction target, the nucleus setting device 20 connects the click points with a straight line to generate a nucleus.

After the generation of the nucleus, the reference points constituting the nucleus are sampled (S3). For example, when a nucleus is set as a curve, points on the curve are discretely sampled at predetermined intervals and used as reference points. 3 (a), (b),
The point of the black circle arranged on the nucleus of (c) is the reference point.
When a nucleus is designated as a point, that point may be used as a reference point, or a plurality of positions near the point may be automatically sampled as reference points.

A feature quantity is extracted in the vicinity including the reference point or the point of interest (arbitrary sampling point not on the nucleus, usually adopted from the vicinity of the reference point) (S4). The point of interest refers to a point at which it is determined whether or not to merge with a region including the reference point, and is usually a point outside the extraction region and near the reference point. In this embodiment, a texture feature amount and a color component feature amount are employed as feature amounts.

A method for extracting texture features by the texture feature extraction device 22 will be described. In the present embodiment, the gray level value of the input image (if the input image data is a color image, it is converted into a gray image) is subjected to Gabor wavelet conversion, and the texture feature is extracted based on the conversion coefficient.

The Gabor wavelet is given by the following equation (1)
Sine wave having a constant directional component and a spatial frequency is modulated by a Gaussian function, and is specified by a scaling level index m and a directional component index n. That is,

[0035]

[Equation 1] g_mn(x, y) = (a^−m/ 2πσ_xσ_y) exp (h_mn(x, y)) h_mn(x, y) = {(xcosθ_n+ ysinθ_n)^Two/ a^2mσ_x ^Two+ (-xsinθ_n+ yco
sθ_n)^Two/ a^2mσ_y ^Two} / 2-i2πW (xcosθ_n+ ysinθ_n) / a^m  Where (x, y) is the position in the image and a is the scaling
Factor, θ_nIs the directional component of the filter, W is the basic space circumference
The wave numbers are shown. σ_x, Σ_yAre the filter functions
A parameter giving the magnitude of the spread of a number in the x and y directions
Data.

As a wavelet, this set of filters has similar functional shapes and different main directions and sizes. In this wavelet, the function form is localized in the spatial frequency domain and the real space domain, and
Simultaneous uncertainty regarding position and spatial frequency is minimized,
It is known that the function is the most localized function in both real space and frequency space (JG Daugman, “Un
certaintylation for res
solution in space, spatial
frequency, and orientati
on optimized by two-dimen
sional visual coral fi
liters ", Journal of Optica
l Society of America A, v
ol. 2, pp. 1160-1169, 198
5).

In the present embodiment, θ _n is 0 degree, 30 degrees, 60 degrees.
Taking values in six directions of degrees, 90 degrees, 120 degrees and 150 degrees, a is set to 2, and m is given as an integer from 1 to 4. σ _x and σ _y are desirably set such that the filters appropriately and homogeneously overlap each other in the Fourier domain, and there is no bias toward a specific spatial frequency. For example, if the half-value levels for the amplitude maximum after Fourier transform are designed to be in contact with each other in the Fourier domain,

[0038]

２ _u = − (a−1) U _H / {(a + 1) (2ln2) ^1/2 }

[0039]

Σ _v = tan (π / 2N) {U _H ² − (2ln2) σ _u ² } / {(2ln2) U
_H ² − (2ln2) ² σ _u ² } ^1/2

[0040]

[Equation 4] a = (U_H/ U_L ^{) −1 / (M−1)}  Where U_H, U_LIs covered by the wavelet transform.
Where M is the maximum and minimum value of the spatial frequency band
Gives the number of scaling levels in the range.

By performing a two-dimensional convolution operation between each filter g _mn (x, y) and the input grayscale image, Gabo
An r wavelet transform is performed. That is,

[0042]

(Equation 5) Here, I is an input image, and W _mn is a Gabor wavelet transform coefficient. _{W m n (m = 1,} ···, 4; n =
The set of 1,..., 6) is used as a feature vector, and W _mn (x, y) is obtained at each point. In addition, * shows taking a complex conjugate. In order to mainly reduce the calculation amount, the following feature amount may be obtained from W _mn and used in a region growing process based on a distance function value in a feature space between a reference point and a point of interest, which will be described later.

For example, as texture energy,

[0044]

(Equation 6) Is used, the average uniformity of spatial frequency in a certain range
A combination is represented. Since it is a scalar quantity,
The amount of calculation at the time can be greatly reduced.

In addition, in the area near each reference point or point of interest,
A set of mean and variance values for each component of
If, μ_mn, Σ_mnNear the reference point or the point of interest, respectively.
[Μ] when the mean and variance of₁₁, Σ₁₁, Μ
₁₂, Σ₁₂, ... μ₄₆, Σ ₄₆])
May be used. In addition, in a region near the reference point,
Evaluation criteria, for example, if the homogeneity is high in the local area,
The average value in the place area is the largest, otherwise the control
Last (the difference between the maximum value and the minimum value is large and takes each value
If the frequency is above a certain level)
W corresponding to the constant (m, n)_mnThe texture features and
Or from the one giving the maximum value in the order of the values
Pieces (about 2 or 3 pieces) of W_mnSelectively extract it,
Reduced feature vectors (eg,
If [| W_m1n1|, | W_m2n2|, | W _m3n3|])
It can also be used as a texture feature at a point or near a point of interest.
No. Furthermore, it is defined in the following color component feature quantity.
Of the distribution shape of the histogram
Near each of the Gabor wavelet transform coefficients
And may be expressed as a feature vector.

The feature value extracted by the color component feature value extracting device 24 will be described. As the color components, RGB component values or components decomposed into hue, saturation, lightness, and the like are generally used. Here, a histogram is taken for each color component in a nearby area given by a rectangular area of a predetermined size (for example, 10 pixels vertically and 10 pixels horizontally) centered on the reference point (or point of interest), and the reference point (or point of interest) is taken. In each case, a color component characteristic amount is obtained as a characteristic vector based on the distribution shape or a characteristic amount given in a list format of representative values of color components.

When expressing the distribution shape of the histogram as a feature vector, for example, for each range (called a bin) obtained by equally dividing the range from 0 to 255 for each of the RGB component values by N (for example, N = 20). The number of pixels having the corresponding color component is counted, and the frequency is obtained.
With respect to the frequency of each color component, a feature vector V _{R with} N elements,
_{V G,} V _B is obtained. For ^{_{example, V R = [V 1 R}} ,
^{_{V 2 R, ···, V N}} R] represents the distribution shape of the histogram for the R component.

The feature value of the representative component value of the color component is a color component value such that the distance in the color space is greater than or equal to a reference value, for example, a color component value that is the center value of each bin of the histogram. Is represented. The distance D _c in the color space used herein, the sum of the absolute values of differences of the components, i.e., D = | R (x, y) -R (x ', y') | + | G (x , y) -G (x ', y') | + | B (x, y)
-B (x ', y') | is typically used. Therefore, the list of representative colors near the reference point is (R ₁ , G ₁ , B ₁ ), (R ₂ ,
G ₂ , B ₂ ),..., (R _k , G _k , B _k ).

After the feature value is extracted, the feature value distance calculating device 26
Calculates the feature distance between the reference point and the point of interest (S
5). FIG. 5 is a flowchart showing the operation of the feature distance calculation by the feature amount distance calculation device 26 and the region growth by the region growth device 28. In the present embodiment, the variance of the gray values having values in the range of 0 to 255 near the reference point is obtained, and if the variance is equal to or larger than a threshold value (for example, 30), it is determined that the image pattern has a texture superior feature. If it is less than the threshold value, it is determined that the change in the gray level is gradual (S11).

If the variance is equal to or larger than the threshold value (S11), the texture feature is loaded from a primary storage device (not shown) (S12), and a feature (eg, texture energy) suitable for region growth is calculated (S13). The distance _DT related to the texture feature amount is obtained (S14). The feature amount conversion process (S13) can be omitted depending on the processing speed, required accuracy, and the like. In that case, the texture feature amount is Gab
or as a set of wavelet transform coefficients.

If the variance value is less than the threshold value (S11), the color component feature is loaded from a primary storage device (not shown) (S1).
6), the feature amount suitable for region growing (e.g., such as a list of feature vectors or representative color component obtained from the local histogram of the color component) to extract (S17), obtains a distance D _C for the color component (S18). Conversion of feature value (S
17) can be omitted depending on the processing speed and required accuracy. In that case, the color component feature amount is given as RGB component values.

The determination in S11 is not limited to the variance, and it is also possible to determine which feature is superior based on another evaluation value scale. The distance _DT related to the texture feature amount is a scalar amount of the texture feature amount (eg, texture energy or a Gabor wavelet transform coefficient value selected and extracted).
In this case, it is preferable to use the absolute value of the difference between the values at the reference point and the point of interest. Similarly, if the texture feature amount is a vector amount, it is preferable to use the sum of absolute differences between corresponding components. However, the Euclidean distance, the Minkowski distance, the Chi-Nikho distance, and the Kolmokov-Smirnov distance may be used as other measures of the feature amount distance.

The processing shown in FIG. 5 is an example, in which a distance is defined as a weighted sum of the first and second distances, and a predetermined evaluation value γ relating to the superiority of the texture (or color component) is defined as a weighting factor. The value shown in the following equation may be calculated. That is,

[0054]

D = D_c+ ΓD_T  However, if γ is set as, for example, γ = β (α / σ (x, y) + μ (x, y)), the local area around the point (x, y) is set.
The average value μ (x, y) of the texture feature is high in the
The higher the homogeneity 1 / σ (x, y), the higher the value of the coefficient γ
Becomes larger. α and β are constants.

Next, the region growing device 28 grows the region based on the feature distance and updates the reference point position (S6).
The region growth is based on the distance D between the reference point and the point of interest in the feature space.
Is smaller than or equal to the threshold value, the point of interest is merged with the extraction region including the reference point, and if it is larger than the threshold value, the point of interest is excluded from the extraction region. The boundary update device 30
The nucleus boundary line data obtained in the region growing process is obtained, and the result of the successive update is displayed on the screen of the image display device 14.

In the processing shown in FIG. 5, the region is grown based on the corresponding feature amount (S15, S19), and the update based on the new boundary line of the core region obtained by integrating the region growth results from each reference point is performed. The position is determined (S20). In particular,
The update position of each reference point is given as a predetermined sampling point on the boundary of a new nucleus region obtained by region growth. In the region growth based on the color component feature, when the feature based on the representative component value of the color component is used, the update of the reference point position is not essential.

The reference point is set in accordance with a predetermined method (for example, from the boundary line of a new nucleus region in the process of region growth).
Sampling (so that the distances between adjacent reference points along the boundary are approximately equal). Therefore, the number of reference points generally increases or decreases.

FIGS. 3 (d), (e) and (f) show FIG.
The intermediate results from (a), (b) and (c) are shown. FIG.
The hatched portions in (d), (e) and (f) indicate regions in the growth process. FIG. 3 (g) shows the final result after region growth. The growth area converges on the contour of the subject. In the example shown in FIG. 3, the nucleus is initially set in the background portion, so that after the growth is completed, the nucleus becomes a donut-shaped region having a hole in the region to be extracted.

In this embodiment, for example, the threshold is monotonically increased while the user is pressing the left button of the mouse, and the threshold is monotonically decreased while the user is pressing the right button. Then, the threshold is changed (S7). In the meantime, the region growth result is displayed on the screen of the image display device 14 by the outer boundary line of the nuclear region or the like (S
8). As a result, an image can be extracted that reflects the user's intention. As a method of changing the threshold value, a method of increasing or decreasing linearly as a function of the time during which a mouse button or the like is pressed, or tanh which is a hyperbolic function
A method of increasing or decreasing so as to saturate to a certain level as in (x) is effective. The latter is effective and suitable when the upper or lower limit of the threshold is to be restricted in advance.

In this embodiment, the outline of the extraction target and the image of the extraction target can be automatically extracted depending on the combination of the background and the design of the extraction target. FIG. 6 shows an example in which the flowchart of FIG. 4 is modified as described above. The process from the input of the image (S11) to the start of the region growing (S26) is the same as the flowchart shown in FIG. However, here, region growth is performed according to a predetermined initial threshold value from a nucleus (or a nucleus set by the user) consisting of all or a part of the frames of the input image, and an evaluation value for determining the validity of the growth result is obtained. As an example), the edge ratio of a point on the boundary line of the nucleus region updated by the region growth is calculated, and the region growth threshold is controlled based on the calculated value (S27). Specifically, for example, when the edge ratio is less than a predetermined threshold value Th _E (for example, 0.5), the threshold value is monotonously increased at a predetermined ratio for each processing cycle of region growth or for each unit time, and becomes equal to or greater than Th _E. Then, a threshold for region growth is given according to a monotonically increasing function (such as a hyperbolic function tanh) that gradually reaches a predetermined saturation level.

In this case, if the initially set nucleus is a closed curve surrounding the extraction target, the inner boundary of the donut-shaped region obtained as a result of region growth is the target for calculating the edge ratio. If all of the initially set nuclei are inside the extraction target, it goes without saying that the outermost boundary is the target of calculating the edge ratio. In addition, when the initially set nucleus is an isolated point, a fragmentary line segment, or an isolated small area in the vicinity of the extraction target, the validity of the area growth result cannot always be determined only by the edge ratio. However, the process may be automatically interrupted (S28) to display the intermediate result (S28), and the boundary line of the nuclear region may be displayed (S29).

The conditions for the processing interruption or automatic stop (S28) are as follows: when the edge ratio exceeds a predetermined level (for example, 90%) or when the absolute value of the rate of change becomes equal to or less than a predetermined value (for example, 10%). And the like. Processing stop may be determined using an index other than the edge ratio, for example, a boundary line length or a texture edge ratio. The texture edge is obtained by extracting the edge of the two-dimensional distribution data of the texture feature amount (texture energy or Gabor wavelet transform coefficient value, etc.).
It is obtained by performing a spatial differentiation operation using an el filter or the like and further performing a binarization process.

In calculating the edge ratio, if the boundary line of the nucleus obtained as a result of the region growth is in contact with the image frame, it is desirable not to target that portion. This is because it is generally considered that the case where the contour to be extracted matches the contour of the image frame is small.

If the user finds that the outline obtained by the region growth is incomplete, the processing can be continued further or the incomplete part can be corrected by the method exemplified in the following procedure. For example, by using the instruction selection device 18 to specify and select a boundary of a necessary portion or an appropriate portion, the portion is fixed (for example, a method of double-clicking a mouse button or a method of displaying the image on the screen of the image display device 14). There is a way to click the icon of the confirmation button that is displayed.)
Accordingly, the region growing process is automatically continued for the remaining portion, or the region growing is performed according to the user's instruction.

The outline extracted by the above method is the outline of the extraction target or the outline of the background region excluding the extraction target. When the contour forms a closed curve, the mask data generating device 32 outputs the mask data (generally, binary data) representing the area inside the closed curve to each pixel inside the mask area as “1” and each outside pixel as “1”. Is set to '0'.) (S9, S30). The extraction image generation device 34 generates image data indicating the extraction target image by taking the logical product of the mask data and the original image,
The extraction result is displayed as an image (S10, S31). For example, a display method such as setting a region other than the extraction target to a specific color or blinking the extracted outline is displayed so that the user can easily visually recognize the extraction result.

In this embodiment, G is used as the texture feature amount.
Although the feature values in the local spatial frequency domain obtained by the abor wavelet transform are used, the present invention is not limited to such feature values. For example, as another characteristic amount, as illustrated below, instead of information such as a pixel value of a certain pixel or a mere density gradient (edge intensity), an image pattern of a local two-dimensional region (or a global region) is not included. What is necessary is just a feature amount representing a feature (however, it is desirable that the feature amount can be defined for each pixel), and the same applies to the following embodiments. For example, feature amounts (RM Haralick e) such as energy, entropy, correlation, and local uniformity obtained by a density co-occurrence matrix
ta1. , "Textural features
forage classificatio
n ", IEEE Trans. Syst. Ma.
n, Cybern. , Vol. SMC-3, p
p. 610-621, 1973) or mean, variance, Skewness and Kurtosis for gray values.
(PC Chen and TP
alvidis, "Segmentation by
Texture Using a Co-Ocurr
ence Matrix and a Sp1it-a
nd-Merge Algorithm ", Comp
Uter Graphics and Image P
processing, vol. 10, pp. 172
-182, 1979).

Each of the above processes may be formed in a predetermined program format that can be executed by a computer, or each process may be performed by a predetermined gate array (FPGA and A
Obviously, the present invention can also be realized in a form in which hardware such as an SLC or the like, or a predetermined computer program and a partial hardware module for realizing some elements shown in FIG. 1 are mixed. When a hardware module is included, each component does not necessarily have to have the same configuration as that shown in FIG. It goes without saying that a device having a function is included in the technical scope of the present invention. This is the same in the following embodiments.

FIG. 7 is a schematic configuration block diagram showing another example of the configuration of the image processing apparatus 16. 40 is an initial contour setting device, 42 is a texture feature amount extraction device, 44 is a co-occurrence probability field calculation device that calculates a co-occurrence probability field based on the texture feature amount, and 46 is a texture boundary direction based on the co-occurrence probability field. A texture boundary direction calculating device, 48 is a sampling point moving device, 50 is a movement stop determining device, 52 is a boundary position precise estimating device, 54 is a boundary line updating device, 56 is a mask data generating device, and 58 is an extracted image generating device. . The provision of a primary storage device (not shown) for temporarily storing data being processed is the same as that of FIG.

FIG. 8 shows an operation flowchart of the configuration shown in FIG. As a processing example, FIG. 3 (b) → (e) →
(G).

After the input and display of the image (S41), the user selects a closed curve (including a rectangular frame or the like) surrounding the extraction target or a closed curve roughly representing the shape of the extraction target inside the extraction target as an initial contour. Is set using the instruction selecting device 18 (S42). The initial contour is not limited to a closed curve, and may be a closed curve that roughly surrounds the extraction target as shown in FIG.

An outline image generating device (not shown) superimposes and displays the initial outline on the screen of the image display device 14 in a specific color or pattern so as to be easily recognized. The reference points are sampled at predetermined intervals on the initial contour, for example, at intervals such that the distance along the contour becomes substantially constant between adjacent points (S43).

Next, similarly to the first embodiment, the Gabor wavelet transform is performed on the gray value of the input image (when the input image data is a color image, it is converted into a gray image), and the conversion is performed. The texture feature amount data at each point in the image is extracted by the coefficient (S44). As described above, in addition to the texture energy, the feature amount used here is a feature vector including a set of an average value and a variance value of each component in each reference point or an area near the point of interest, or a feature vector in the vicinity of the reference point. extracting a plurality of W _mn in the order of W _mn, or its absolute value corresponding to the predetermined evaluation go reference value specific to provide maximum ones (m, n) in the region,
A reduced-dimensional feature vector constituted by the above is used.

At each reference point, a texture co-occurrence probability field H (x, y, D, θ) defined below is determined for the texture feature of a local region including the point (S45).
The texture co-occurrence probability field H (x, y, D, θ) is obtained by calculating the feature amount of one (reference) as i, the feature amount of the other (comparison control) as j, and the texture feature amount changing near the boundary of the texture. Assuming that a threshold value serving as a reference for the magnitude change amount is Th, this is the probability that two points satisfying | i−j |> Th exist, and is given by the following equation. That is,

[0074]

(Equation 8)However, (x_a, Y_a) Is at an arbitrary position in the local area A.
A reference point, (x_b, Y _b) Indicates a comparison in the local area A.
Points are indicated. D is a local region including the point (x, y)
Is the distance between two points, and θ is the other point viewed from one point
Indicates the direction. N is the area of the region A (the number of pixels). V
_x(Θ), V_y(Θ) is calculated for each θ as shown in FIG.
It is determined.

In this embodiment, in order to estimate the direction in which the texture boundary exists, the above-described probability H is obtained by using only the difference between the two texture features larger than a certain threshold value Th as a basic constraint condition. Distinguish between positive and negative. The direction opposite to the specific direction is also distinguished.

On the other hand, in the co-occurrence matrix, a constraint condition is that a pixel which is separated by a distance D in the q direction from a pixel having a density value i has a density value j, and there are two points satisfying such a condition. Find the probability density. Particular Gabor wavelet transform coefficients _{(e.g., W m n (x, y} )) when used as the texture feature, texture co-occurrence probability H is H
_mn (x, y, D, q).

When the user instructs the start of image extraction,
It is instructed whether the initial contour basically converges to the extraction target contour by contracting or expands. In this case, the basic movement direction of each reference point on the contour is the direction toward the center of the contour in the former case, and the direction from the center toward the reference point in the latter case.

Texture co-occurrence probability H (x, y, D, θ)
推定 based on the value of
(X, y) is determined as θ that maximizes the value in argmax as shown in the following equation (S46). That is,

[0079]

９ (x, y) = argmax _θ {NH (x, y, D, θ) P (x, y, D, θ)} P (x, y, D, θ) = H (x, y, D, θ) / (H (x, y, D, θ) + H (x, y,
D, θ + π)) NH (x, y, D, θ) indicates the number of pairs that satisfy the condition for the existence of a texture boundary, and P (x, y, D, θ)
Indicates a probability indicating the superiority of the direction θ with respect to θ + π as the direction in which the texture boundary exists.

In Equation 9, the texture co-occurrence probability H is obtained for each Gabor wavelet transform coefficient without using the texture feature amount as a specific scalar amount (such as texture energy) as shown in the first embodiment.
May be calculated. Further, the range of the value of the distance D may be changed. In this case, H is obtained for each value of D, and the product N
The direction θ corresponding to a specific Gabor transform coefficient W _mn (x, y) that gives the maximum value of H (x, y, D, θ) P (x, y, D, θ) is obtained.

When the texture co-occurrence probability is 0, for example, when two points satisfying the calculation conditions do not exist in the local area near the reference point, the center direction of the initial contour may be selected as θ.

Next, it is determined whether or not the reference point position can be moved (S47). The determination condition of the movement stop is, for example, that the estimated texture boundary direction is the above-described basic movement direction (when the initial contour surrounds the extraction target, the convergence direction of the initial contour and the initial contour are extracted). Inconsistent with the direction of inflation if set inside the object (eg,
180 degrees or more) and / or the change width and variance of the texture feature amount in the area near the reference point position (10 × 10 rectangular area or the like) are larger than the reference value, or the immediately preceding movement direction and its movement In a case where the new moving direction is largely different from the later estimated new moving direction (for example, a case where the moving direction differs by 180 degrees or more), the reference point is stopped at the position before the movement without moving the corresponding reference point.

If movement is possible, the selected movement direction θ
Next, the position of each reference point is step-moved by a predetermined amount (S4).
8) Repeat the process of estimating the texture boundary direction based on the texture co-occurrence probability and determining whether or not to move at the reference point of the movement destination by the above-described method.

A curve that smoothly connects the reference points after the movement is
The image display device 1 generates a contour line to be extracted by spline interpolation of a predetermined order (typically 3rd order or 5th order).
4 (S49). As a spline function, a B-spline or the like is typically used, but is not limited to a specific one.

After the above-described series of processing, especially after the convergence of the movement of the reference point, or after the above-described repetition processing has been automatically performed a fixed number of times, instructions from the user (mouse button operation and keyboard operation) After a predetermined number of times, the boundary position of the extraction target is accurately estimated (S50). In the present embodiment, since the movement operation of the reference point position is performed by the step movement, it is necessary to estimate the boundary position with a resolution equal to or less than the movement step amount, and therefore, this precise estimation is performed. Specifically, in the vicinity area of a certain range of the reference point position, the maximum position of the edge strength or, as described above, the distance defined in the color space for the feature amount of the color component is equal to or more than the predetermined threshold from the reference point. The position of a point is determined.

The contour between the reference points thus precisely determined is similarly spline-interpolated to generate a target contour line for image extraction (S51). However, before a contour line to be finally extracted is determined, an incomplete portion may be modified in the same manner as described above. For example, the instruction selecting device 18
A contour tracing method such as specifying and selecting a necessary part or an appropriate part boundary line and confirming the part and automatically continuing the above-described processing for the remaining part or tracking the maximum value of the edge strength is performed. You may use together.

Finally, mask data of the object to be extracted or the background to be removed or its outline data is generated and stored in a predetermined storage means, and the process is terminated (S52).

In the same manner as described above, the variance of the grayscale value of the image is obtained near each reference point, and the processing based on the above-described texture feature amount according to the value is performed, and the processing in the configuration shown in FIG. Any of the area growing processes based on the color components may be appropriately selected and executed. In the latter case, the area growth in the area near the reference point is
Although it is easy to perform the process with a fixed threshold value, when performing automatic setting or automatic stop determination of the region growth threshold value, the following may be performed. That is, since it is known in advance that the contour to be extracted is determined by either the mode of convergence or expansion of the initial contour, it is viewed from the center direction of the initial contour in the case of convergence and from the center of the initial contour in the case of expansion. The reference point direction is defined as a main growth direction, a fixed angle range including the main growth direction is defined as a region growth range, an edge ratio is obtained for a boundary portion of the grown region, and a graph is obtained based on the edge ratio or its change rate. The control of the threshold value or the stop of the region growth may be performed as described in connection with No. 2. After area growth, a new reference point is set from the boundary of the area after growth, or a boundary part in the main growth direction range after growth is held as a part of the new boundary. In the former case, an appropriate position (for example, from the boundary of a predetermined range including the main growth direction)
A point in the main growth direction) may be sampled and set.

FIG. 10 is a schematic block diagram showing a third configuration example of the image processing apparatus 16. 60 is an initial contour setting device, 62 is a texture feature amount extraction device, 64 is an edge intensity distribution calculation device, 66 is an energy function calculation device that calculates an energy function based on the texture feature amount and the edge intensity distribution of the input image, and 68 is An outline deforming device 70 for deforming the outline based on the calculated energy function, 70 is a mask data generating device, and 72 is an extracted image generating device for generating extracted image data. As in the previous example, a primary storage device for temporarily storing data being processed is provided.

The operation of the configuration shown in FIG. 10 will be described. Here, the target image is extracted by the active contour method that converges to the texture edge position related to the feature amount in the texture feature space. Here, the texture edge is obtained by extracting, as an edge, a portion where the value of two-dimensional distribution data of a predetermined texture feature value changes rapidly. The texture edge can be obtained, for example, by performing a spatial differentiation operation on the distribution data of the texture feature amount using a Sobel filter or the like, and further performing a binarization process.

The active contour method is a method of extracting a contour of an object from edge information so that a predetermined energy function expressing a smooth contour and being on an edge as a constraint condition is minimized. This is a contour extraction method in which a contour model is transformed to converge on a contour on an object. As a method developed from this, an inward or outward external force is applied to a point on the contour of the active contour based on the difference between the image of the area near the initial contour and the feature amount of the image relating to the local area of the subject part. Can be considered.

Referring to the flowchart shown in FIG.
The operation of the configuration shown in FIG. 10 will be described. As a processing example,
3 (b) → (e) → (g).

The steps from the input of the image to the sampling of the reference point (S61 to S63) are the same as S41 to S43 in FIG. A texture feature and an edge intensity distribution are extracted as the feature (S64, S65), and an energy function is calculated from both (S66).

The energy function has a term related to the texture feature amount, which is different from the above-described embodiment and the conventional example. The energy function is specifically defined as follows. The energy calculation device 66 calculates the following evaluation function value E at each reference point position v (s _i ).

[0095]

E = Σ ｛E _int (v (s _i )) + E _image (v (s _i )) + E
_texture (v (s _i )) + E _ext (v (s _i ))｝ Here, the terms E _int and E _image also exist in the conventional active contour method, and the other items are It is newly introduced. s _i denotes the value of the circumferential length along the outline that corresponds to the i-th sampling point position on the contour. E _{in t} is the contour line model v (s) = (x
(S), y (s)) are internal energies to be smoothed, and are given by the following equations. That is,

[0096]

E _int (v (s)) = α (s) | dv / ds | ² + β (s) | d ² v / ds ² | E _image is energy representing a force attracted to a so-called gray-scale edge. Yes, edge intensity distribution calculation device 64
Using the calculation result of

[0097]

(Equation 12)_image(v (s)) =-w_I｜ ▽ I (v (s)) ｜^Two  Is required. Similarly, E_textureIs
With the energy representing the force drawn to the texture edge
And is given by the following equation: That is,

[0098]

(Equation 13)_texture(v (s)) =-w_T｜ ▽ T (v (s)) ｜^Two  In Expressions 12 and 13, I (v (s)) and T (v
(S)) is a pixel value and a text on v (s), respectively.
It represents the key feature. Α (s) and β (s) in Equation 11 are circles
It should be set for each location according to the shape of the contour line.
However, it may be a constant. W_T, W_IThe texture
Determined by weighting the edge or shading edge
Is a number.

E _ext (v (s)) corresponds to an external force and is appropriately set.

[0100]

E _ext (v (s)) = -γ {lnP _{in, s} (T (v (s)))-lnP
_{out, s} (T (v (s)))} γ is a positive constant. P _in (T (v (s)), P _out
(T (v (s)) is an inner contour region and an outer neighboring region of an initial contour line or an updated contour line thereof.
Average value μ _{in, Ns} , μ of each texture feature
_{out, Ns} and variances σ _{in, Ns} , σ _{out, Ns} are represented as follows. That is,

[0101]

P _{in, s} = (1 / {(2π) ^1/2 σ _{in, Ns} }) exp {-(T (v
(s))-μ _{in, Ns} ) ² / (2σ ² _{in, Ns} )} P _{out, s} = (1 / {(2π) ^1/2 σ _{out, Ns} }) exp {-(T (v (s ))-μ
_{out, Ns)} representing the _{^{2 / (2σ 2 out, Ns}} )} Ns neighboring region to obtain an average value and variance of the texture feature for point v (s). The inner side is closer to the center of gravity of the initial contour, and the outer side is farther from the center of gravity of the initial contour.

By applying equation 15, equation 14 becomes
If there is a high probability that the sampling points on the contour are on the image in the area near the inner side of the initial (or updated) contour, an external force that expands outward acts on the contour, and conversely, it is on the image in the area near the outer side. If the probability is high, it means that an external force in the direction of contracting inward acts. Further, Equation 15 can be regarded as a function for evaluating the difference between the texture feature amount at the sampling point v (s) on the contour and the texture feature amount in the inner or outer neighboring region. It is also possible to use a scale for evaluating the difference between other feature amounts regarding the region. In Equation 15, the Gaussian model is used as the probability model, but another model may be used. E
_{In ext} (v (s)), instead of the texture feature amount T, Expression 15 may be evaluated for the image data I. Further, another component item may be added as an external force.

When the texture feature amounts _cover the entire Gabor wavelet transform coefficient W _mn described with reference to FIG.

[0104]

(Equation 16) Is defined as a weighted sum for each component. Here, p is a positive constant. α _mn may be a constant regardless of the definition of the above equation.

In the configuration shown in FIG. 10, the contour deforming device 68 sequentially moves the sampling points on the initial contour until the energy function value defined as described above converges to the minimum value (S67). A curve that smoothly connects the points after the movement is generated by spline interpolation or the like (S6).
8). Thereby, the outline of the target to be extracted is automatically determined. The movement amount of each sampling point can be obtained by a variation method or the like which is a known method in the active contour method. .

The obtained outline is superimposed on the input image on the display screen, displayed so as to be easily recognized, and can be modified as required. After that, the mask data generation device 70
Generates contour data, image data to be extracted or mask data representing an area thereof, and generates an extracted image generating device 72.
Store the extracted image data in a recording medium or a storage device (S72).

In the process in which the initial contour converges on the object to be extracted, the relative weight between the energy term relating to the gray level value (or color component value) of the image and the term relating to the texture feature amount is as described with reference to FIG. It is desirable that the determination be made based on the result of evaluating the superiority of the texture feature based on the variance of the gray value near the sampling point.
Specifically, weighting is performed as given by γ in Equation 7.

[0108]

As can be easily understood from the above description, according to the present invention, extremely rough instructions (designation by dots, designation by an approximate contour line or a rectangular frame, etc.) regarding an extraction target or a non-extraction target, etc. , A color component (or gray value) feature amount and a texture feature amount, or at least the texture feature amount is used to determine a local characteristic of the image pattern (for example, whether the texture feature component is superior or Given an appropriate evaluation function according to the possibility of the existence of a texture boundary, and based on the value of the evaluation function, the boundary line of the extraction region is estimated. Extraction (image clipping or contour line extraction) becomes possible. Further, the labor required for specifying the extraction target is reduced, and automation such as batch extraction is also possible.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of an embodiment of the present invention.

FIG. 2 is a schematic configuration block diagram of a first example of an image processing device 16;

FIG. 3 is a diagram showing the progress of the processing system of the present embodiment.

4 is an operation flowchart of the image processing apparatus 16 shown in FIG.

FIG. 5 is an operation flowchart of a feature distance calculation by a feature distance calculating device and a region growing by a region growing device.

FIG. 6 is a flowchart in which FIG. 4 is modified for automatic extraction.

FIG. 7 is a schematic configuration block diagram of a second example of the image processing device 16;

8 is an operation flowchart of the image processing apparatus 16 shown in FIG.

FIG. 9 is a table showing a numerical example of V _x (θ) and V _y (θ) in Expression 8.

FIG. 10 is a schematic block diagram of a third example of the image processing apparatus 16;

11 is an operation flowchart of the image processing apparatus 16 shown in FIG.

[Explanation of symbols]

DESCRIPTION OF THE PREFERRED EMBODIMENTS 10: Image Input Device 12: Image Storage Device 14: Image Display Device 16: Image Processing Device 18: Instruction Selection Device 20: Kernel Setting Device 22: Texture Feature Extraction Device 24: Color Component Feature Extraction Device 26 : Feature amount distance calculation device 28: Region growth device 30: Boundary line update device 32: Mask data generation device 34: Extracted image generation device 40: Initial contour setting device 42: Texture feature value extraction device 44: Co-occurrence probability field calculation device 46: Texture boundary direction calculating device 48: Sampling point moving device 50: Moving stop determining device 52: Boundary position precise estimating device 54: Boundary line updating device 56: Mask data generating device 58: Extracted image generating device 60: Initial contour setting device 62: Texture feature extraction device 64: Edge intensity distribution calculation device 66: Energy function calculation device 68: Ring Linear transformation device 70: the mask data generation unit 72: extraction image generating apparatus

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Fumiaki Takahashi 3-30-2 Shimomaruko, Ota-ku, Tokyo F-term (reference) in Canon Inc. 5C076 AA02 AA33 BA03 BB04 BB25 CA11 5L096 AA02 AA06 BA07 CA07 CA24 DA01 EA37 EA39 FA06 FA14 FA15 FA39 FA45 FA53 FA66 GA08 GA22 GA23 GA51

Claims (41)

[Claims] 1. A nucleus specified by at least one of a predetermined point, a line segment, and a region is set in an image, and a texture feature amount is determined in a local region including at least one reference point constituting the nucleus. An image extraction method, comprising: calculating an evaluation function including the evaluation function; and updating the shape of the nucleus based on the value of the evaluation function to determine a boundary of a predetermined region in the image. 2. The image extraction method according to claim 1, wherein the evaluation function uses a feature amount of a color component and a texture feature amount as factors. 3. The evaluation function calculates a first distance between two points in a feature space based on a feature amount of a color component between a point of interest given by a predetermined method and a reference point forming the nucleus. ,
A weighted addition of the second distance between the two points in the feature space based on the texture feature amount, and a neighborhood image of the reference point and a neighborhood image of the target point based on the function value of the evaluation function. When the similarity is greater than a predetermined threshold, the point of interest or its neighboring area is merged with a predetermined extraction area including the reference point,
The image according to claim 2, wherein when the similarity is equal to or less than the threshold and the point of interest is included in the extraction region, the shape of the nucleus is updated by deleting the extraction point from the extraction region. Extraction method. 4. The evaluation function calculates a first distance between two points in a feature space based on a feature amount of a color component between a point of interest given by a predetermined method and a reference point constituting the nucleus. ,
And selecting one of a second distance between the two points in the feature space based on the texture feature amount based on a predetermined criterion, and an image of an area near the reference point based on a function value of the evaluation function. And calculating the similarity between the vicinity area image of the point of interest and the similarity, if the similarity is greater than a predetermined threshold, merging the point of interest or its neighboring area with a predetermined extraction area including the reference point,
The image according to claim 2, wherein when the similarity is equal to or less than the threshold and the point of interest is included in the extraction region, the shape of the nucleus is updated by deleting the extraction point from the extraction region. Extraction method. 5. The evaluation function is expressed as a function of a predetermined co-occurrence probability value related to the texture feature amount in the vicinity of the reference point, and the reference point is set in a direction in which the function value of the evaluation function becomes maximum. The method according to claim 1, wherein the shape of the nucleus is updated by moving the nucleus by a predetermined amount. 6. The image extraction method according to claim 1, wherein when the value of the edge ratio of a point on the boundary line of the nucleus reaches a predetermined reference value or more, the update processing of the extraction area is automatically stopped. 7. The image extraction method according to claim 1, wherein when the absolute value of the rate of change of the edge ratio of a point on the boundary line of the nucleus reaches a predetermined reference value or less, the update processing of the extraction area is automatically stopped. . 8. The image extracting method according to claim 3, wherein the threshold is updated based on a signal from a predetermined instruction selecting unit. 9. The evaluation function is a predetermined energy function including a term that gives a minimum value at the boundary of the texture feature amount, and moves the position of each reference point so that the function value of the evaluation function becomes minimum. 2. The image extraction method according to claim 1, wherein the shape of the nucleus is updated by performing the process. 10. The first distance relates to one of a difference value between a value at the reference point and a value at the point of interest for each color component decomposed into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted power sum, and the second distance is a difference value between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components, and the absolute value of the difference. 4. The image extraction method according to claim 3, wherein the image extraction is represented by a weighted sum of powers concerning one of the values. 11. The first distance relates to one of a difference value between a value at the reference point and a value at the target point for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted sum of powers, and the second distance is a value at the reference point of an energy value represented by a predetermined weighted sum of powers for each texture feature component decomposed into a plurality of components, and 4. The image extraction method according to claim 3, wherein the image extraction method is represented by a predetermined function value relating to one of a difference value of a value at a point and an absolute value of the difference. 12. The method according to claim 3, wherein the weighting factor for weighting and adding the first distance and the second distance is determined based on a variance value related to a gray value near the reference point. Image extraction method. 13. A nucleus setting step of setting a nucleus specified by at least one of a predetermined point, a line segment, and an area in an image, and setting a reference point using a representative point set on the nucleus as a reference point. A step, a first distance between two points in a feature space based on a feature amount of a color component for a point of interest given by a predetermined method and the reference point, and a second distance in a feature space based on a texture feature amount. A similarity calculating step of calculating a similarity between a neighboring area image of the reference point and a neighboring area image of the target point based on a feature distance obtained by weighting and adding a second distance between the points; Is larger than a predetermined threshold, the point of interest or a nearby area is merged with a predetermined extraction area including a reference point, and when the point of interest is equal to or less than the threshold and the point of interest is included in the extraction area, the extraction area is To be removed from Than,
An image extraction method, comprising: an extraction area update step of updating a boundary line of the extraction area; and a boundary line display step of displaying a boundary line of the updated extraction area. 14. The method according to claim 1, wherein the similarity calculating step includes:
14. The image extraction method according to claim 13, wherein one of the distance and the second distance is selected to be the feature amount distance. 15. The method according to claim 1, wherein the extracting area updating step includes a threshold updating step of increasing or decreasing the threshold by a predetermined method every time the boundary line is updated.
3. The image extraction method according to 3. 16. The threshold updating step includes: an edge ratio calculating step of obtaining a ratio at which a point on a predetermined boundary line on the extraction area is an edge point; and one of a value of the edge ratio and a change rate thereof. 16. The image extracting method according to claim 15, further comprising: an updating step of updating the threshold value by using a command. 17. The apparatus according to claim 13, wherein said threshold value is updated based on a signal from a predetermined instruction selecting means.
2. The image extraction method according to 1. 18. The first distance relates to one of a difference value between a value at the reference point and a value at the point of interest for each color component decomposed into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted power sum, and the second distance is a difference value between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components, and the absolute value of the difference. 14. The image extraction method according to claim 13, wherein the image extraction method is represented by a weighted sum of powers concerning one of the values. 19. The first distance relates to one of a difference value between a value at the reference point and a value at the target point for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted sum of powers, and the second distance is a value at the reference point of an energy value represented by a predetermined weighted sum of powers for each texture feature component decomposed into a plurality of components, and 14. The image extraction method according to claim 13, wherein the image extraction method is represented by a predetermined function value regarding one of a difference value of a value at a point and an absolute value of the difference. 20. A weighting coefficient for weighting and adding the first distance and the second distance is determined based on a variance value regarding a gray value near the reference point.
2. The image extraction method according to 1. 21. An initial contour setting step of setting an initial contour existing inside or outside of an object to be extracted, and a plurality of predetermined representative points set on the initial contour as reference points. A reference point setting step, and a first distance between two points in the feature space based on the feature amount of the color component with respect to the point of interest given by the predetermined method and the feature point in the feature space based on the texture feature amount A similarity calculating step of calculating a similarity between the neighboring area image of the reference point and the neighboring area image of the target point based on the feature distance obtained by weighting and adding the second distance between the two points; When the similarity is larger than a predetermined threshold, the point of interest or its neighboring area is merged into a predetermined extraction area including the reference point, and when the similarity is equal to or less than the threshold and the point of interest is included in the extraction area, The extraction area An extraction area updating step of updating the boundary line of the extraction area by a predetermined number of times by deleting the initial contour from the object, and when the entire initial contour is outside the object,
The threshold is determined based on the updated edge ratio on the inner boundary line of the extracted region, and the edge ratio on the outer boundary line of the updated extracted region is determined when the entire initial contour is inside the object. A threshold setting step of determining the threshold based on the threshold value. 22. The method according to claim 21, wherein the similarity calculating step includes:
22. The image extraction method according to claim 21, wherein one of the distance and the second distance is selected and set as the feature amount distance. 23. The first distance relates to one of a difference value between a value at the reference point and a value at the point of interest for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted power sum, and the second distance is a difference value between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components, and the absolute value of the difference. 22. The image extraction method according to claim 21, wherein the image extraction method is represented by a weighted sum of powers concerning one of the values. 24. The first distance relates to any one of a difference value between a value at the reference point and a value at the point of interest for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted sum of powers, and the second distance is a value at the reference point of an energy value represented by a predetermined weighted sum of powers for each texture feature component decomposed into a plurality of components, and 22. The image extraction method according to claim 21, wherein the image extraction method is represented by a predetermined function value regarding one of a difference value of a value at a point and an absolute value of the difference. 25. A weighting coefficient for weighting and adding the first distance and the second distance is determined based on a variance value relating to a gray value near the reference point.
2. The image extraction method according to 1. 26. A setting means for setting a nucleus specified by at least one of a predetermined point, a line segment and an area in an image, and a color component of a local area comprising at least one reference point constituting the nucleus. Evaluation function calculation means for obtaining a function value of a predetermined evaluation function including at least one of a feature amount and a texture feature amount; and a boundary of a predetermined region in the image by updating the shape of the nucleus based on the function value of the evaluation function. An image extracting apparatus comprising: a boundary updating unit that determines a line. 27. The evaluation function includes a first distance and a texture between two points in a feature space based on a feature amount of a color component for a point of interest given by a predetermined method and a reference point constituting the nucleus. A weighted addition of the second distance between the two points in the feature space based on the feature amount by a predetermined method, or one of them is selected based on a predetermined criterion, and is obtained.
The boundary line updating unit obtains the similarity between the neighboring area image of the reference point and the neighboring area image of the target point based on the function value of the evaluation function. If the similarity is greater than a predetermined threshold, The point of interest or its neighboring area is merged with a predetermined extraction area including the reference point, and when the similarity is equal to or less than the threshold and the point of interest is included in the extraction area, the point is deleted from the extraction area. The image extraction device according to claim 26, wherein the shape of the nucleus is updated. 28. The first distance relates to one of a difference value between a value at the reference point and a value at the point of interest for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted power sum, and the second distance is a difference value between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components, and the absolute value of the difference. 27. The image extraction device according to claim 26, wherein the image extraction device is represented by a weighted sum of powers concerning one of the values. 29. The first distance relates to one of a difference value between a value at the reference point and a value at the target point for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted sum of powers, and the second distance is a value at the reference point of an energy value represented by a predetermined weighted sum of powers for each texture feature component decomposed into a plurality of components, and 27. The image extraction device according to claim 26, wherein the image extraction device is represented by a predetermined function value regarding one of a difference value of a value at a point and an absolute value of the difference. 30. A weighting coefficient for weighting and adding the first distance and the second distance is determined based on a variance value regarding a gray value near the reference point.
An image extraction device according to claim 1. 31. A setting means for setting a predetermined extraction nucleus in an image, and a feature based on a color component feature amount for a point of interest given by a predetermined method in the image and a reference point constituting the extraction nucleus. A predetermined feature amount distance obtained by weighting and adding a first distance between two points in the space and a second distance between the two points in the feature space based on the texture feature amount by a predetermined method. A similarity calculating means for calculating a similarity between the neighboring area image of the reference point and the neighboring area image of the point of interest, and extracting the point of interest or its neighboring area when the similarity is greater than a predetermined threshold value An image extracting apparatus, comprising: merging means for merging into an area. 32. The image extraction device according to claim 31, wherein the similarity calculation means selects one of the first distance and the second distance and sets the selected distance as the feature amount distance. 33. The first distance relates to one of a difference value between a value at the reference point and a value at the target point for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted power sum, and the second distance is a difference value between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components, and the absolute value of the difference. 32. The image extraction device according to claim 31, wherein the image extraction device is represented by a weighted power sum regarding one of the values. 34. The first distance relates to any one of a difference value between a value at the reference point and a value at the point of interest for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted sum of powers, and the second distance is a value at the reference point of an energy value represented by a predetermined weighted sum of powers for each texture feature component decomposed into a plurality of components, and The image extraction device according to claim 31, wherein the image extraction device is represented by a predetermined function value regarding one of a difference value of a value at a point and an absolute value of the difference. 35. A weighting coefficient for weighting and adding the first distance and the second distance is determined based on a variance value relating to a gray value near the reference point.
An image extraction device according to claim 1. 36. A nucleus setting means for setting a nucleus specified by at least one of a predetermined point, a line segment, and a region in an image, and a reference using a predetermined representative point set on the nucleus as a reference point. A point setting means, a first distance between two points in a feature space based on a feature amount of a color component and a corresponding point in a feature space based on a texture feature amount for a point of interest given by a predetermined method and the reference point. A similarity calculating means for calculating a similarity between a neighboring area image of the reference point and a neighboring area image of the target point based on a feature distance obtained by weighting and adding a second distance between the two points by a predetermined method; And if the similarity is greater than a predetermined threshold, the point of interest or a nearby area is merged with a predetermined extraction area including the reference point, and the point of interest is equal to or less than the threshold and the point of interest is included in the extraction area. Must be deleted from the extraction area. And an extraction area updating means for updating a boundary line of the extraction area by a predetermined number of times. 37. The image extraction apparatus according to claim 36, wherein the similarity calculation means selects one of the first distance and the second distance and sets the selected distance as the feature amount distance. 38. The first distance relates to one of a difference value between a value at the reference point and a value at the point of interest for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted power sum, and the second distance is a difference value between the value at the reference point and the value at the point of interest for each texture feature component decomposed into a plurality of components, and the absolute value of the difference. 37. The image extraction device according to claim 36, wherein the image extraction device is represented by a weighted sum of powers regarding one of the values. 39. The first distance relates to any one of a difference value between a value at the reference point and a value at the point of interest for each color component separated into a plurality of components, and an absolute value of the difference. The second distance is represented by a weighted sum of powers, and the second distance is a value at the reference point of an energy value represented by a predetermined weighted sum of powers for each texture feature component decomposed into a plurality of components, and 37. The image extraction device according to claim 36, wherein the image extraction device is represented by a predetermined function value relating to one of a difference value of a value at a point and an absolute value of the difference. 40. A weighting coefficient for weighting and adding the first distance and the second distance is determined based on a variance value relating to a gray value near the reference point.
An image extraction device according to claim 1. 41. A storage medium storing the program software of the image extraction method according to claim 1. Description: