JP2000251083A - Device and method for extracting contour and program recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device and a method for extracting contour capable of presenting plural pieces of candidate data capable of subjectivity evaluation to a user so as to select any one concerning data processing for which the evaluation based on the subjectivity of the user is requested to the output result. SOLUTION: A color shading axis candidate calculating part 32' calculates plural color shading axes. A contour curve generating part 33 generates a contour curve candidate list corresponding to the list of candidates of the said plural color shading axes. The contour curve candidate list generated by this contour curve generating part 33 is displayed on a display device. Then, the user is made select any axis by a candidate list selecting part 71 and the color shading axis selected by the user is sent to the contour curve generating part 33. In the contour curve generating part 33, a new contour curve in an update block is generated from an image, a representative point 1, a representative point 2 and the color shading axis. In an update block synthesizing part 34, the contour curve of the updated block is synthesized with a closed curve and generated as a new closed contour curve.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a contour extracting apparatus and method for extracting a contour of an object of interest from an image. Also, the present invention relates to a program recording medium.

[0002]

2. Description of the Related Art In a contour extraction process for extracting a contour of an object of interest from an image, there are roughly two types of conventional methods which operate so that an area can be extracted more accurately by an operator's intervention input. There is.

[0003] In the first prior art, contour information automatically extracted by some processing is automatically extracted but unnecessary information, or information that could not be automatically extracted but is required is identified by an operator. Is a type that determines and performs an operation of deleting or adding them. As specific examples of the prior art belonging to this, “Image contour detection method” (Japanese Patent Laid-Open No. 3-291770), “Image editing device” (Japanese Patent Laid-Open No.
75076).

In the second prior art, an operator determines that initial input is not sufficient for input information for performing a contour extraction process, and operates so as to provide a more accurate input and recalculate. Kind of thing. As a specific example of the prior art belonging to this, "Image contour detection method"
(Japanese Unexamined Patent Publication No. 4-152481), "Pattern Mask Creation Method" (Japanese Unexamined Patent Publication No. 1-155476), "Cut-out Mask Creation Method" (Japanese Unexamined Patent Publication No. 4-68763), "Intelligent Sc"
issors for Image Composition ”(Eric N. Mortensen
and William A. Barrett, Computer Graphics Proceedin
g, Annual Conference Series, 1995, ACM SIGGRAPH, p
p.191-198).

[0005] For each operation of the above prior art,
A brief description will be given.

"Image contour detection method" (Japanese Patent Laid-Open No. 3-291
770), an edge is extracted using an edge detection operator in the first automatic extraction, and for an outline that could not be extracted by the edge detection processing, the operator designates both ends of the outline, and a line segment connecting the two ends as an outline. Works to be added.

"Image editing apparatus" (Japanese Patent Laid-Open No. 1-150707)
Step 6) performs a process of extracting, as an object region, a region surrounded by a schematic line by an operator among regions in an image which has been divided into regions by color quantization in advance. Then, if there is an extraction error, the operator can specify the corresponding areas one by one and correct whether the areas are added to or deleted from the target object area.

[0008] "Picture mask making method" (Japanese Patent Laid-Open No. 1-1)
55476), “Cut-out mask making method” (Japanese Unexamined Patent Application Publication No.
-68763) is a technique similar to chroma key, and operates so as to detect the boundary between the colors of the outline area by inputting the colors at both ends of the outline, that is, one or both of the foreground and background colors. If necessary, it can be operated by partially changing the color input.

[0009] "Intelligent Scissors for Image Compo"
"sition" operates to search for and extract contour points between them by giving some relay points on the contour. This search method searches for a route connecting relay points so as to pass over the edge as much as possible, based on a search rule using edge information of an image. Quite complicated contour shapes can be automatically extracted.
Therefore, this method has an advantage that a contour can be extracted with a relatively small number of relay points.

"Image contour detection method" (JP-A-4-152)
481) also gives some relay points on the contour,
It operates to search for and extract contour points between them. This method operates to apply a bias in the search direction depending on the position of the relay point when performing the contour search. Therefore, by changing the relay point, it is easy to extract a contour that advances in a desired direction.

In the first prior art, the accuracy of the final contour largely depends on the accuracy of the first automatic extraction, and it cannot be expected that the contour will be made more accurate by the subsequent correction operation of the operator. There was a problem.

[0012] For example, "Image contour detection method" (Japanese Patent Laid-Open No.
-291770), as described above, has a drawback in the accuracy of the shape, since manual addition of a contour can only be performed with a line segment.

An "image editing apparatus" (Japanese Patent Laid-Open No. 1-17 / 1990)
In 5076), it is necessary to make the area divided first not too small so that the correction operation is not complicated. For that purpose, the color quantization needs to be moderately coarse. However, if the color quantization is made coarse, the shape of the region obtained by the region division and the true object outline shape are likely to be inconsistent, and the accuracy of the automatically extracted outline shape is lost.

On the other hand, in the second prior art, recalculation is performed by giving a more accurate input to a region where the result of automatic extraction is not sufficient, which is advantageous in terms of contour accuracy. However, the second conventional technique has the following problem.

"Picture mask making method" (Japanese Patent Laid-Open No. 1-1)
55476), “Cut-out mask making method” (Japanese Unexamined Patent Application Publication No.
The operator correction in -68763) operates to correct the contour shape by partially changing the input of the background color or the foreground color and recalculating the contour extraction in that part, as described above. However, the color input operation is
Speaking of the chroma key device, it is equivalent to the operation of adjusting the key level while monitoring the monitor screen. Since this type of operation does not directly input information on the contour shape, it is difficult for the operator to intuitively perform correct input, and several trials and errors are required to obtain a desired contour shape.

On the other hand, "Image contour detection method" (Japanese Patent Laid-Open No. 4-152481) and "Intelligent Scissors for I"
In the “mage Composition” (hereinafter, a conventional technique of inputting a relay point), an operator's correction operation is to operate a relay point on a contour, and an operation is performed so that a contour shape related to the corrected relay point is recalculated. In this method, since an input regarding a contour shape is directly provided, it is relatively easy for an operator to intuitively perform a correct input.

[0017]

However, since the criterion for evaluating whether or not the obtained curve passes on the edge is left to the subjective evaluation of the user, the following problems occur.

An example of an image of a boy holding a bat shown in FIG. 21A will be described. In FIG. 21A,
When the points p1 and p2 are placed, it becomes controversial about which edge the user expects the curve to pass through.

If the user's purpose is "to obtain a contour surrounding the entire boy", the contour curve expected by the user is shown in FIG.
(B) is as follows.

On the other hand, if the user's purpose is “to change the color of the boy's uniform and want to obtain a contour surrounding only the uniform”, the contour curve expected by the user is as shown in FIG. 21 (C).

Which edge a user expects a contour curve to pass through often varies depending on the subjectivity and purpose of the user, and is not uniquely determined. For this reason, it can be fully assumed that even if the result of a certain automatic calculation algorithm that operates normally is correct, it may not meet the user's intention.

That is, as shown in FIG. 22, the data processing unit 100 requires the output result to be subjectively evaluated by the user.
On the other hand, since a single output result by a single algorithm is used, there is a problem that it is not always possible to meet a user's desire that may change depending on the subjectivity and purpose.

The present invention has been made in view of the above circumstances, and presents a plurality of subjectively evaluable candidate data to a user for data processing in which the output result requires subjective evaluation by the user. It is an object of the present invention to provide a contour extraction device and method capable of causing the contour extraction.

[0024]

According to the present invention, there is provided a contour extracting apparatus for extracting a contour of an object from an image, the method comprising extracting a representative point on an edge of the object. Update section extracting means for obtaining a set of representative points sandwiching the update section from the coordinates and the initial closed curve; and a set of representative points obtained by the update section extracting means and a color projection axis candidate in the update section from the image. A plurality of color projection axis calculating means for converting the plurality of color projection axis candidates into data that can be subjectively evaluated by the operator, presenting the data to the operator, and determining the color projection axis by auxiliary input by the operator; Contour curve generating means for generating a contour curve in an update section from the image by using the color projection axis calculated by the color projection axis calculation means and the set of representative points determined by the update section extraction means; Generated by the generation means The contour curve in the update interval synthesized closed curve has, and a contour closed curve synthesizing means for generating a new contour closed curve.

According to a second aspect of the present invention, there is provided a contour extracting method for extracting a contour of an object from an image, the coordinate of a representative point on an edge of the object. And an update interval extraction step of obtaining a set of representative points sandwiching the update interval from the initial closed curve, and obtaining a plurality of color projection axis candidates in the update interval from the set of representative points obtained in the update interval extraction step and the image. Converting the plurality of color projection axis candidates into data that can be subjectively evaluated by the operator, presenting the data to the operator, and determining the color projection axis by auxiliary input by the operator; A contour curve generating step of generating a contour curve in an update section from the image using the color projection axis determined in the projection axis calculation step and the set of representative points determined by the update section extracting means; Generated by The contour curve in the new segment synthesized closed curve, and a contour closed curve synthesizing step of generating a new contour closed curve.

That is, an algorithm with changed parameters or a plurality of results generated by a plurality of algorithms is converted into data that can be subjectively evaluated, presented to the user as a candidate, and made to be selected by the user, thereby allowing the user to expect. The result can be obtained.

Therefore, for data processing, such as a key generation device, in which output results require subjective evaluation by the user, a plurality of results generated by a plurality of algorithms and a plurality of parameters can be subjectively evaluated. By converting the data into simple data, presenting it to the user as a candidate, and causing the user to select the data, it is possible to obtain the result expected by the user.

[0028]

Embodiments of the present invention will be described below with reference to the drawings. This embodiment is a key generation device shown in FIG. 1, which generates a soft key based on a contour area extracted using the contour extraction device according to the present invention.

From the closed contour curve obtained by the closed contour curve generation section 21, the both end closed curve generation section 22 generates a both end closed curve,
The soft key generation unit 23 generates a key image from the closed curve at both ends.

The contour closed curve generating unit 21 is a specific example of the contour extracting apparatus according to the present invention. The closed contour generating unit 21 receives an image, a closed curve of an initial shape, and coordinates of a representative point on an edge, and generates the closed curve by a plurality of algorithms and a plurality of parameters. The plurality of results are converted into data that can be subjectively evaluated, presented to the user as candidates, selected by the user, and a closed contour curve according to the user's desire is generated.

The principle of operation of the closed contour generating section 21 will be described with reference to FIG. Conceptually, it includes a data processing unit 10, a data conversion unit 11, a display device 12, an operation device 13, and an output data selection unit 14.

The data processing unit 10 processes the input data
A plurality of output data candidate lists are presented by applying a plurality of algorithms and parameters. The data conversion unit 11 generates a candidate list of data that can be subjectively evaluated by humans from the candidate list. This candidate list is displayed on the display device 12.
Displayed above.

When the user selects a desired candidate using the operating device 13 while looking at the display device 12, the number of the selected candidate is sent to the output data selecting unit 14, and the output data selecting unit 14 The output data as desired is sent to the display device 12 again. As a result, the user can obtain an expected result.

The contour closed curve generator 2 to which the present invention is applied
The details of the function 1, ie, a plurality of results generated by a plurality of algorithms and a plurality of parameters, are converted into data that can be subjectively evaluated, presented to the user as candidates, and made to be selected by the user, to meet the user's wishes. The details of the function of generating a contour closed curve will be described later,
First, a new technique that can be applied when the contour closed curve generating unit 21 generates one closed contour curve will be described.

FIG. 3 shows a detailed configuration of the contour closed curve generator 21. The contour closed curve generator 21 includes an update section extractor 31, an optimal color projection axis calculator 32, a contour curve generator 33, and an update section synthesizer 34.

When the representative point is designated by the user, the update section extraction section 31 extracts the update section of the curve, and obtains the representative points 1 and 2 which sandwich the update section without excess or deficiency. More specifically, the user may use a mouse or tablet (not shown).
When a representative point is specified on the edge of the image using the dimensional pointing device as input means, the coordinates of the representative point on the edge are supplied to the update section extraction unit 31. In addition, for example, a closed curve such as a circle, a square, or a polygon is also provided to the update section extraction unit 31 as an initial shape. Using the coordinates of the representative points on the edge and the initial shape, the update section extraction unit 31 obtains representative points 1 and 2 that sandwich the update section without excess or deficiency.

In the optimum color projection axis calculation unit 32, the representative points 1,
2 and the image, a color space projection axis that maximizes the contrast around the edge in the update section is determined. The details of the optimum color projection axis calculation unit 30 will be described later.

The contour curve generator 33 generates a new contour curve in the update section from the image, the representative points 1 and 2, and the color projection axis. The details of the contour curve generation unit 33 will also be described later.

The update section synthesizing section 34 synthesizes the contour curve of the updated section with a closed curve and generates a new contour closed curve. While the update occurs due to the movement, addition, or deletion of the representative point by the user, the contour closed curve is also updated.

FIG. 4 shows a specific example of the contour closed curve generated by the update section synthesizing section 34. That is, the contour closed curve is shown in FIG.
As shown in (1), it is composed of representative points on the edge for which the user directly specifies coordinates, and spline curves passing through the edges and passing through them.

FIG. 5 is a detailed diagram of the optimum color projection axis calculator 32. The optimum color projection axis calculation unit 32 calculates a neighborhood optimum color projection axis in the vicinity area of the representative point 1 and the representative point 2, and calculates an optimum color projection axis calculation unit 41 near the representative point and an optimum color projection axis calculation unit near the representative point 41, and a color projection axis synthesizing section 42 for synthesizing the color projection axis using the color projection axis 1 and the color projection axis 2 from the two calculation sections 41, 41.

In the optimum color projection axis calculation unit 41 near the representative point,
A neighborhood optimum color projection axis that maximizes the contrast around the edge included in the neighborhood area of the representative point is obtained. Assume that a representative point 1 (p0) and a representative point 2 (p1) are designated as shown in FIG. At this time, the representative point color projection axis calculation unit 41 calculates the color projection axis (1) B0 at which the contrast is maximum among the edges included in the vicinity area A1 of the representative point p0. Further, the color projection axis calculation unit 41 near the representative point
Is an edge included in the area A2 near the representative point p1,
The color projection axis (2) B1 at which the contrast becomes maximum is obtained by calculation. The detailed configuration of the color projection axis calculation unit 41 near the representative point will be described later.

The color projection axis synthesizing section 42 synthesizes and updates the neighborhood optimum color projection axis (1) B0 and the neighborhood optimum color projection axis (2) B1 which are separately obtained in the vicinity areas of the representative points 1 and 2. One color projection axis that maximizes the contrast of the edge of the section is determined. The combining method here may use, for example, the average vector of the color projection axes as the combined axis. However, in this case, when the vectors of the two color projection axes form an obtuse angle, it should be noted that the description will be made with reference to FIG. 7 below.

As shown in FIG. 7A, the color projection axis vectors obtained at the representative points on the above-mentioned edges at both ends of the update section are defined as v0 and v1. At this time, assuming that a composite vector c of v0 and v1 defined by the following equation is a color projection axis of the section, this c is substantially perpendicular to both v0 and v1, and the color projection axis of the section is Not suitable as an axis.

C = v0 + v1 This is because v0 and v1 form an obtuse angle. In the case of an obtuse angle, that is, when the inner product v0 · v1 is negative, FIG.
As shown in (B), for one of the vectors, for example, v0 'and v1 obtained by inverting v0, a combined vector c' is obtained as in the following equation, and is used as a color projection axis.

V0 '=-v0 c' = v0 '+ v1 As a result, an axis nearly horizontal to both v0 and v1 can be obtained.

Next, although the order is reversed, the detailed configuration of the above-described representative point vicinity color projection axis calculation unit 41 will be described with reference to FIG. The optimal color projection axis calculation unit 41 near the representative point includes a straight edge detection unit 51 and a color information acquisition unit 52 around the edge.
And a color projection axis calculation unit 53.

For example, the straight-line edge detection unit 51 in the optimum color projection axis calculating unit 41 near the representative point for calculating the color projection axis near the representative point 1 calculates the image, the representative point 1 and the adjacent point (neighboring representative point). Based on the point 2), the representative point 1 (p
A linear edge component included in the neighborhood A0 of (0) is detected, and an equation of the straight line is obtained. When a plurality of straight edges are included in the image of the vicinity area A0, for example, the one having the strongest edge is adopted.

The optimum color projection axis calculator 41 near the representative point for calculating the color projection axis near the representative point 2 is as follows.
The straight-line edge detection unit 51 calculates a linear edge component included in the vicinity area A1 of the representative point 2 (p1) shown in FIG. 6 based on the image, the representative point 2, and the adjacent point (neighboring representative point 1). Is detected, and the equation of the straight line is obtained. Here, if a plurality of straight edges are included in the image of the neighboring area A1, for example, the one with the strongest edge is adopted.

More specifically, the operation principle and the detailed configuration of the straight edge detection unit 51 will be described below with reference to FIGS.

FIG. 9 shows a representative point (anchor point) p0.
Shows a slight deviation from the edge of the image. Since the representative point is often placed off the edge as described above, in order to improve the operability, it is necessary to make the linear edge detection unit 51 estimate the position and direction of the edge even in such a case.

Further, in FIG. 9, the edge near the representative point p0 is not uniquely determined, and a plurality of edge candidates such as E1 and E2 may exist. For this reason, the straight edge detection unit 51
Is not originally one, but is naturally a list of straight lines ordered by priority.

Next, in a local area near the representative point, attention is paid to the fact that edges are often linear.
An estimation method for estimating the position and direction of an edge using the Hough transform will be described with reference to FIG.

As shown in FIG. 10A, when a linear edge exists near the representative point (anchor point), the gradient direction of the pixel on the edge is perpendicular to the edge. Assuming that the angle in the gradient direction is θ and the signed distance between the edge and the anchor point is ρ, all the pixels existing on the same edge have θ and ρ which are close to each other. It concentrates on a local area as shown in FIG.

By utilizing this, a two-dimensional histogram in the θ, ρ space is created, and a linear part is detected using the frequency.

FIG. 11 shows the detailed configuration of the straight edge detecting section 51. First, the nearby area cutout unit 81 generates a partial image obtained by cutting out a local area near an anchor point.

The monochrome image generator 82 generates a monochrome image projected on a certain appropriate color projection axis, for example, a luminance axis. Here, since it is used as a rough estimate, it does not need to be the optimal axis.

The gradient image generation unit 83 generates a gradient image by applying a Sobel filter or the like to the monochrome image.

The ρ · θ two-dimensional histogram generator 84 calculates ρ · θ for all the pixels of the gradient image to generate a histogram. Assuming that the anchor point is the origin and the gradient vector at a certain pixel P (x, y) is (gx, gy), ρ, θ is ρ = − (gx · x + gy · y) / sqrt (gx ＾ 2 +
gy ＾ 2) θ = arctan (gy / gx). Here, gx> 0, and sqrt represents a square root.

The priority generation unit 85 assigns a priority using the frequency of the histogram, the value of ρ, the direction to the adjacent anchor point, and the like.

The linear equation coefficient calculating section 86 calculates the coefficient of the linear equation based on the value of θ · ρ. Ax is a straight line equation
If + bx + c = 0, then a = cos θ b = sin θ c = −ρ.

As described above, the straight edge detecting section 51
A straight line equation can be obtained, and the obtained straight line equation is sent to the edge surrounding color information acquisition unit 52. As described with reference to FIG. 9, when the edge near the representative point p0 is not uniquely determined and a plurality of edge candidates may exist,
The output of the straight line edge detection unit 51 is not one, but it is natural to output a list of straight lines ordered according to the priority.

Next, the edge surrounding color information acquiring section 52 shown in FIG.
Then, the periphery of the linear equation (edge) detected by the edge detection unit 51 is sampled, and a histogram of the color distribution classified into one side and the other side of the edge is generated. Specifically, there is a method of acquiring a color within a region of a diameter and several pixels at a position at a certain fixed distance from a straight line. FIG. 1 shows the operation of the edge surrounding color information acquisition unit 52 by this method.
2 will be described.

As shown in FIG. 12, it is assumed that a straight line L approximating the edge is placed on the edge of the image. At this time, a circle having a radius r having a center Q0 and a center Q1 is placed at a position passing through a point P on L and perpendicularly to the straight line by a distance h, and samples colors in this area. This allows
In the vicinity of the point P, color information around the edge can be obtained.

Referring back to FIG. 8, the color projection axis calculation unit 53 calculates the color distribution histogram of the edge surrounding color information acquisition unit 52 from the color distribution histogram.
Find a color projection axis that maximizes edge contrast.
For example, there is a method in which a vector connecting the centers of gravity of distributions on one side and the other side of an edge in a color space is used as an axis.

The operation of calculating the color projection axis in the color projection axis calculation section 53 will be described below with reference to FIG.

In general, the color distribution around the edge is shown in FIG.
It is known to form a banana extending in the direction of the luminance axis as shown in FIG.

At this time, assuming that the axis Axis1 in the luminance axis direction is the color projection axis as shown in FIG. 13A, the color distribution Cf of the foreground area is projected in the range of Wf1. On the other hand, the color distribution Cb of the background area is projected in the range of Wb1. In this,
The color change in each area of Cf and Cb is large,
The color change between Cb is small. Since the color change in the edge region of the image is a color change between Cf and Cb, Axis1 is not the color projection axis to be obtained.

As shown in FIG. 13B, when the axis Axis2 substantially perpendicular to the luminance axis direction is set as the color projection axis, Cf becomes Wf
2. Since Cb is projected onto Wb2, the color change in each area of Cf and Cb is small, and the color change between Cf and Cb is large, so that it is close to the desired color projection axis.

Hereinafter, this is formulated. Assuming that the color projection axis vector is a, the sum of the class sum of squares matrices in each of the color distributions Cf and Cb is Qw, and the class sum of squares matrix between the color distributions Cf and Cb is Qb, the class dispersion of Cf and Cb is aTQw
The interclass dispersion between a, Cf and Cb is aTQba. Here, aT represents transposition of a.

The color projection axis calculation unit 53 obtains the axis a for increasing the ratio of the interclass dispersion to the intraclass dispersion as much as possible.

The average of Cf is F ^* , the average of Cb is B ^* , Qw
The above a is represented by a = Iw (F ^* -B ^* ), where Iw is an inverse matrix of Iw. This is used as a color projection axis.

The above is a detailed description of a specific example of the optimum color projection axis calculation unit 32 in FIG. Note that the optimum color projection axis calculation unit 32 may obtain the optimum color projection axis by another method.
FIG. 14 shows the principle operation for obtaining the optimum color projection axis by another method.

Assume that an anchor point p0 and an adjacent anchor point p1 are located on the edge of the image as shown in FIG. At this time, since the edge may be largely off the line segment connecting p0 and p1, the edge is placed at equal intervals, for example, at intervals of L / 4 with respect to the line segment length L on the line segment p0p1. For each of the points q0, q1, and q2, a search is made for an area having the largest color change in the range of the length H on a line segment perpendicular to p0p1.

Assuming that the color at a certain point r is C (r), the color change vector is dC / dr and the magnitude is | dC
/ Dr |. The value r at which this value becomes the maximum value is obtained.

As a result of the search, as shown in FIG.
It is assumed that r1 and r2 have been obtained. At this time, r0, r
The color change vector dC / dr is determined for each of the color difference vectors 1 and r2, and the average value is set as the optimal color projection axis.

The color change vector may be obtained as follows. In the case of r0, on a straight line perpendicular to the line segment p0p1, a point s having r0 as a center and separated from each other by a distance width h.
For 00 and s01, colors C (s00) and C (s01)
And the difference vector ΔC = C (s01) −C (s
00) is substituted for the color change vector. The same applies to r1 and r2.

Next, details of the contour curve generator 33 in FIG. 3 will be described with reference to FIG. The contour curve generation unit 33 includes a high contrast monochrome image generation unit 6
1, a gradient vector field generation unit 62, a minimum cost path search unit 63, and a spline curve generation unit 64.

High-contrast monochrome image generator 61
Then, a monochrome image is generated by projecting the color at each pixel of the image and the input color projection axis.

The gradient vector field generator 62 generates an XY two-dimensional gradient vector field by applying a gradient detection operator to a monochrome image.

The minimum cost path search unit 63 determines a part having a strong edge in the gradient vector field as a low cost part, and obtains a discrete pixel string representing a path that minimizes the accumulated cost.

The spline curve generator 64 generates a smooth spline curve from the pixel sequence.

The above is the details of the closed contour generation unit 21 in FIG. 1 for generating the closed contour using a new technique. The key generation device using the closed contour generating unit 21 described above (the configuration is the same as the key generating device shown in FIG. 1) is as shown in FIG. 4 generated by the closed contour generating unit 21. A contour closed curve, a both-end closed curve generation unit 22 for generating a both-end closed curve from an image, and the both-end closed curve generation unit 22
And a soft key generation unit 23 for generating a key image from the closed-end curve generated in step (1).

The closed contour generating section 22 generates a closed-end curve from the closed contour. Specifically, the present applicant generates a closed-ended curve using the “contour extraction method” disclosed in Japanese Patent Application Laid-Open No. 10-164436.

This “contour extraction method” determines a contour area for the contour of the object, calculates a gradient vector in the contour area, and, based on the gradient vector in the contour area, moves along the edge of the edge along the edge of the contour area. And a plurality of boundary positions on the outside are extracted, and boundary coordinate information representing the respective boundary positions is generated for each of the inside and outside of the edge, and two boundary curves approximating each shape represented by each boundary coordinate information are obtained. It is generated as a closed curve at both ends of the object.

Further, the contour closed curve generation unit 22 is formed by the two ends of the shape which reflect the width / offset value by the curve deformation technique disclosed in the specification and drawings of Japanese Patent Application No. 10-268687 filed by the present applicant. A closed curve may be generated.

This curve transformation technique converts two curves representing the inner and outer boundary positions in the outline of an object included in an image into a series of points, generates a series of points for each curve, and The coordinates of each point are changed by linking a series of points on the curve, and two curves, that is, closed contours at both ends are reconstructed from the coordinates of each point whose coordinates have been changed.

The soft key generation unit 23 generates a three-dimensional surface to which the α axis of the key signal is added based on the both-end curve generated by the both-end closed-curve generation unit 22, and that the region between the both-end curves has an α value of Generate a soft key image that changes smoothly.

However, in the key generation device using the contour closed curve generating section 21 for generating one closed contour curve described above, as described with reference to FIG. Since there is only one type of output for one type of input, once the edge representative points at both ends are determined, the contour curve in that section is uniquely determined. If the unique result does not match the user's subjectivity, the only option is to change the position, add, or delete the edge representative point.

For example, taking the color projection axis calculation unit 41 near the representative point shown in FIG. 8 as an example, the straight line obtained by the straight line edge detection unit 51 has only one strongest edge. Originally, as described in FIGS. 9 and 11, a plurality of evaluation values are obtained.

Therefore, the contour closed curve generation unit 2 shown in FIG.
1 is changed from the configuration of FIG. 3 to the contour closed curve generation unit 21 ′ of the configuration shown in FIG. 16 based on the principle described in FIG. 2, and a plurality of candidates are presented to the user to assist the selection of the user. To be used as an important input.

As shown in FIG. 16, the contour closed curve generation unit 201 'includes an update section extraction unit 31, a contour curve generation unit 33, and an update section synthesis unit 34 having the same configuration as that shown in FIG. , A candidate selection function addition / change portion surrounded by a broken line.

The candidate selection function addition and change part includes a color projection axis candidate calculation unit 32 'for calculating a plurality of color projection axes, and a contour curve for generating a contour curve candidate list corresponding to the plurality of color projection axis lists. It includes a generation unit 33 and a candidate list selection unit 71 for displaying the contour curve candidate list generated by the contour curve generation unit 33 on the display device 12 as shown in FIG. The candidate list selection unit 71 converts the color projection axis selected by the user into the contour curve generation unit 33.
Send to

FIG. 17 shows a detailed configuration of the color projection axis candidate calculation unit 32 '. The color projection axis candidate calculation unit 32 ′ calculates the representative point 1
A representative point neighboring optimum projection axis candidate calculating unit 41 ′ and a representative point neighboring optimum projecting axis candidate calculating unit 41 ′ for calculating a plurality of candidates for the nearest optimum color projection axis of the representative point 2; It comprises a color projection axis synthesis candidate generation unit 42 'for generating a color projection axis synthesis candidate using the color projection axis candidate list 1 and the color projection axis candidate list 2 from 41'. If there are n candidates in the list 1 of color projection axis candidates from the two calculation units 41 ′ and 41 ′ and there are m candidates in the list 2 of color projection axis candidates, the color projection axis synthesis candidate generation unit 42 ′ generates a list of n × m candidates.

Optimum projection axis candidate calculation unit 41 ′ near a representative point
Outputs a list of a plurality of color projection axis candidates by a linear equation.

FIG. 18 shows a detailed configuration of the above-mentioned optimal projection axis candidate calculating unit 41 'near the representative point. The representative point neighborhood optimal projection axis candidate calculation unit 41 ′ includes a straight edge candidate detection unit 51 ′.
And the color information acquisition unit 52 around the edge and the color projection axis calculation unit 5
3

For example, a candidate color projection axis candidate calculating unit 41 'for calculating a candidate color projection axis near the representative point 1 is shown.
The internal straight edge candidate detection unit 51 ′ is included in the vicinity area A0 of the representative point 1 (p0) illustrated in FIG. 6 based on the image, the representative point 1, and the adjacent point (adjacent representative point 2). A plurality of linear edge components are detected, a straight line equation is obtained, and a list of a plurality of straight line equation candidates is generated.

Further, a candidate for a color projection axis in the vicinity of the representative point 2, which is obtained by calculation, calculates a candidate for a color projection axis in the vicinity of the representative point adjacent color projection axis.
And a plurality of linear edge components included in the neighboring area A1 of the representative point 2 (p1) shown in FIG. 6 based on and the adjacent point (neighboring representative point 1), an equation of the straight line is obtained, Generate a list of candidate linear equations.

The color information acquisition section 52 around the edge and the color projection axis calculation section 53 also output a list of a plurality of candidates.

For this reason, the color projection axis candidate calculation section 32 ′ in the contour closed curve generation section 21 ′ shown in FIG. 16 transmits the color projection axis candidate list to the contour curve generation section 33 and the candidate list selection section 70. Can be sent.

The contour curve generator 33 has the same configuration as that shown in FIG. 15, but generates a plurality of contour curve candidate lists corresponding to the color projection axis candidate lists and displays them on a display device. That is, the contour curve generation unit 33 converts the plurality of color projection axis candidates into data that can be subjectively evaluated by the user, and displays the plurality of contour curves on the display device 12 shown in FIG. . Alternatively, a plurality of contour curves may be displayed in an overlapping manner, or may be displayed at once using a plurality of windows.

The user can select a desired contour curve from the plurality of contour curves displayed on the display device 12. The user's selection result is input to the candidate list selection unit 71. The candidate list selection unit 71 selects a color projection axis that forms a contour curve on the display device 12 selected by the user from the candidate list, and sends the color projection axis to the contour curve generation unit 33 that originally forms the contour closed curve generation unit 21 ′. Supply.

The contour curve generating section 33 includes the candidate list selecting section 7
The contour curve is calculated based on the color projection axis selected in step 1 and supplied to the update section synthesis unit 34.

As described above, the update section synthesizing unit 34
The contour curve of the updated section is combined with a closed curve to generate a new contour closed curve. Moving representative points by user
While the update occurs due to addition or deletion, the contour closed curve is also updated.

The outline closed curve generator 21 'has been described above. The contour closed curve generation unit 21 'constitutes the key generation device shown in FIG.
Used instead of

The operation of the key generation device in this case will be described with reference to FIG.

After the representative points p1 and p2 on the edge are designated by the user, the paths c1 and c2 are used as candidate contours in FIG.
The information is presented on the display device 12 as shown in FIG. FIG.
9 (B) shows a state in which the mouse cursor has been moved to the vicinity of c1 because the user selects c1. When the user performs a click operation, FIG.
As shown in (C), c1 is determined.

In the existing method, when the expected contour curve cannot be obtained, there is no other method than to change, add or delete the position of the edge representative point. On the other hand, in the above-described key generation device whose operation has been described with reference to FIG. 19, the operation for obtaining the expected contour curve is a one-step operation of clicking once on the desired contour. It can be seen that the operation for obtaining the desired contour curve can be reduced.

Further, in the above-described key generation device, the contour extraction device of the present invention is implemented by hardware in the form of a contour closed curve generation unit 21 '.
However, the contour extraction method of the present invention may be realized by a software program and processed by, for example, a computer system as shown in FIG.

This computer system comprises an arithmetic processing unit 1, a program memory 2, a data memory 3, a frame memory 4, an image display device 5, an input device 6, and an external storage device 7 connected to a bus 8.

The program memory 2 stores the above-described contour extraction method as a software program. The arithmetic processing unit 1 sequentially retrieves the software program from the program memory 2 and executes it. Data memory 3
Stores data that is being processed by the arithmetic processing unit 1.

The frame memory 4 stores the image data obtained by the arithmetic processing unit 1 executing the software program and using the data memory 3 for image display. The image data stored in the frame memory 4 is displayed on the image display device 5 and can be viewed by the user. The image display device 5 can perform multi-window display, and can simultaneously display a plurality of images.

As the input device 6, a mouse or a keyboard is used, for example, and data is input by the user. The external storage device 7 stores image data and the like. The bus 8 connects the above-described units, and moves program codes and data to and from each other.

The image data is stored in the data memory 3. When the user inputs the coordinates of two points indicating the section for which the contour curve is to be obtained through the input device 6, the arithmetic processing device 1
In step (1), a candidate list of contour curves is obtained, imaged, and output to the image display device 5 through the frame memory 4.

On the basis of the candidates displayed on the image display device 5,
When the user inputs a contour curve that meets his / her expectation through the input device 6, the contour curve that best meets the user's expectation is stored in the data memory 3. The image data and the contour curve stored in the data memory 3 are formed into an image according to a user's instruction, and output to the image display device 5 through the frame memory 4. In the color projection axis candidate calculation unit 32 'in the contour closed curve generation unit 21' shown in FIG. 16, a specific example of calculating a plurality of candidates with different parameters while using the same algorithm has been described. For example, as an algorithm, FIGS.
In this method, the parameters are changed by independently using the method of finding the color projection axis in the region near the representative point shown by using No. 3 and the method of equally dividing the straight line connecting the representative points shown in FIG.

The color projection axis candidate calculation section 32 'may calculate a plurality of color projection axis candidates by a plurality of algorithms including the above two algorithms, and present them to the user.

Further, the present invention is not limited to the case of edge detection, and can be applied to all data processing that can output a plurality of candidates and convert them into data that can be subjectively evaluated.

[0118]

According to the present invention, a contour extraction device capable of presenting and selecting a plurality of candidate data that can be subjectively evaluated to a user for data processing in which the output result requires subjective evaluation by the user. And methods can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram of a key generation device according to an embodiment of the present invention.

FIG. 2 is a block diagram for explaining an operation principle of a contour closed curve generation unit as a specific example of the contour extraction device of the present invention applied to the key generation device.

FIG. 3 is a block diagram showing a configuration of the contour closed curve generation unit when one contour closed curve is generated.

FIG. 4 is a diagram showing a specific example of an outline closed curve generated by the outline closed curve generation unit shown in FIG. 3;

FIG. 5 is a block diagram showing a detailed configuration of an optimum color projection axis calculation unit that forms the contour closed curve generation unit shown in FIG. 3;

FIG. 6 is a diagram for explaining the operation principle of the optimum color projection axis calculation unit near the representative point, which constitutes the optimum color projection axis calculation unit shown in FIG. 5;

FIG. 7 is a diagram for explaining the principle of operation of a color projection axis synthesizing unit constituting the optimum color projection axis calculation unit shown in FIG. 5;

8 is a block diagram showing a configuration of an optimum color projection axis calculation unit near a representative point, which constitutes the optimum color projection axis calculation unit shown in FIG. 5;

FIG. 9 is a diagram for explaining the operation principle of the straight-line edge detection unit constituting the optimal color projection axis calculation unit near the representative point shown in FIG. 8;

FIG. 10 is a diagram for explaining the principle of operation of an estimation method for estimating the position and direction of an edge by a straight-line edge detector, which constitutes the optimum color projection axis calculator near the representative point shown in FIG.

FIG. 11 is a block diagram showing a configuration of a straight-line edge detection unit constituting the optimum color projection axis calculation unit near the representative point shown in FIG. 8;

FIG. 12 is a diagram for explaining the operation principle of a color information acquisition unit around an edge, which constitutes the optimal color projection axis calculation unit near the representative point shown in FIG. 8;

FIG. 13 is a diagram for explaining the operation principle of the color projection axis calculation unit constituting the optimum color projection axis calculation unit near the representative point shown in FIG. 8;

FIG. 14 is a diagram for explaining another principle of operation of the optimum color projection axis calculation unit constituting the contour closed curve generation unit shown in FIG. 3;

FIG. 15 is a block diagram illustrating a configuration of a contour curve generation unit included in the contour closed curve generation unit illustrated in FIG. 3;

FIG. 16 is a block diagram illustrating a configuration of a contour closed curve generation unit that presents a plurality of contour curve candidates to be selected by a user.

FIG. 17 is a block diagram of an optimum color projection axis candidate calculation unit that constitutes the closed contour curve generation unit shown in FIG. 16;

18 is a block diagram of a candidate color projection axis candidate calculating unit near a representative point, which constitutes the color projection axis candidate calculating unit shown in FIG. 17;

FIG. 19 is a diagram for explaining the operation of the key generation device having the closed contour generation unit shown in FIG. 16;

FIG. 20 is a block diagram of a computer system that uses the contour extraction method of the present invention as a software program.

FIG. 21 is a diagram for explaining a problem caused by a conventional contour extraction method.

FIG. 22 is a diagram generalizing the problem of the conventional contour extraction method.

[Explanation of symbols]

21 'contour curve generator, 31 update section extractor, 3
2 ′ optimal color projection axis candidate calculation unit, 33 contour curve generation unit, 34 update section synthesis unit, 41 ′ representative point neighborhood optimum color projection axis candidate calculation unit, 42 ′ color projection axis synthesis candidate generation unit,
51 'straight edge candidate detection unit, 52 color information acquisition unit around the edge, 53 color projection axis calculation unit

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Takushi Totsuka 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5C023 AA40 BA02 CA01 CA08 DA01 5L096 AA02 AA06 DA02 EA35 EA43 FA03 FA06 FA13 FA15 FA33 FA35 FA38 FA60 GA38 JA01

Claims (24)

[Claims] 1. A contour extraction device for extracting a contour of an object from an image, wherein an update section extracting means for obtaining a set of representative points sandwiching the update section from coordinates of a representative point on an edge of the object and an initial closed curve. And obtaining a plurality of color projection axis candidates in the update section from the set of representative points obtained by the update section extraction means and the image, and converting the plurality of color projection axis candidates into data that can be subjectively evaluated by the operator. Color projection axis calculation means for determining the color projection axis by the auxiliary input by the operator, and the color projection axis obtained by the color projection axis calculation means and the update section extraction means A contour curve generating means for generating a contour curve in the update section from the set of representative points obtained from the image; Generate a closed curve Contour extraction apparatus characterized by comprising a contour closed curve synthesizing means. 2. The color projection axis calculation means obtains a plurality of color projection axis candidates by applying an algorithm with changed parameters or a plurality of algorithms, and allows the operator to subjectively evaluate the plurality of color projection axis candidates. 2. The contour extracting apparatus according to claim 1, wherein the contour extracting apparatus converts the data into data and presents the data to an operator, and determines a color projection axis by auxiliary input by the operator. 3. The color projection axis calculation means for calculating a plurality of color projection axis candidates in an update section from the set of representative points obtained by the update section extraction means and the image, An operator looks at a plurality of contour curves generated by the color projection axis calculation means and generates a contour curve including each of the plurality of color projection axis candidates obtained by the color projection axis calculation means. 3. The contour extraction device according to claim 2, further comprising: a color projection axis selection unit that selects a color projection axis included in the contour curve selected by the subjective evaluation. 4. The color projection axis candidate calculating means, comprising: calculating a plurality of optimum color projection axis candidates in each of the neighboring areas of the set of representative points; Color projection axis synthesis candidate means for synthesizing a color projection axis candidate that maximizes the contrast around the edge in the update interval using the optimum color projection axis candidate calculated by the intra-region optimum color projection axis candidate calculation means. 4. The contour extracting device according to claim 3, wherein: 5. The near-region optimum color projection axis candidate calculating means, the straight-line edge detecting means for detecting a plurality of linear edge component candidates based on the image and the representative point in each of the near regions, A color information acquisition unit that acquires color information around a plurality of linear edge component candidates detected by the straight line edge detection unit; and a plurality of color projection axis candidates from the color information acquired by the color information acquisition unit. 5. The contour extracting apparatus according to claim 4, further comprising: a color projection axis candidate calculating means for calculating. 6. The straight-line edge detecting means includes: a near-region extracting means for extracting a partial image of a nearby area of a representative point from the image and the representative point; and a monochrome image of the partial image cut by the near-area extracting means. A monochrome image generating unit, a gradient image generating unit that generates a gradient image from the monochrome image generated by the monochrome image generating unit, and a gradient direction angle θ from the gradient image generated by the gradient image generating unit. A two-dimensional histogram generating means for generating a two-dimensional histogram of a signed distance ρ between the edge and the representative point; an angle θ generated by the two-dimensional histogram generating means;
Priority generating means for prioritizing the signed distance ρ, and the angles θ, which are prioritized by the priority generating means,
6. The contour extracting apparatus according to claim 5, further comprising a straight line equation coefficient calculating means for calculating a coefficient of a straight line equation of a straight edge based on the value of the signed distance ρ. 7. The color information obtaining means samples the periphery of the linear edge component detected by the straight edge detection means, and generates a histogram of color distribution classified into one side and the other side of the edge. 6. The contour extraction device according to claim 5, wherein: 8. The color information acquisition means, when a straight line approximating an edge is placed on an edge of an image, passes through a point on the straight line, and is positioned on both sides separated by a predetermined distance in a direction perpendicular to the straight line. 8. A method according to claim 7, further comprising: placing two circles having the same radius on a circle, and sampling colors in an area of the circle.
The contour extraction device as described. 9. The color projection axis candidate calculating means converts a vector connecting the centroids of the distributions on one side and the other side of the edge in the color space of the color distribution information acquired by the color information acquiring means to the color projection axis candidate. The contour extracting device according to claim 3, wherein the calculation is performed as follows. 10. The color projection axis candidate calculating means calculates a ratio between the intra-class variance and the inter-class variance of the distribution of the color distribution information obtained by the color information obtaining means on one side and the other side of the edge in the color space. 4. The contour extracting apparatus according to claim 3, wherein an axis that maximizes the distance is calculated as a color projection axis candidate. 11. The color projection axis candidate calculating means, for each of a plurality of equally spaced points on a line connecting a set of representative points placed on the edge of the image, 4. The contour according to claim 3, wherein an area having the largest color change is searched on a vertical line segment drawn on both sides of the line segment by a predetermined length in a vertical direction, and the color projection axis is calculated by calculation. Extraction device. 12. The color projection axis candidate calculating means obtains a color change vector on the vertical line segment, obtains a color change maximum point having a maximum size, and calculates the color change maximum point from the average value of the color change maximum points. 12. The contour extraction device according to claim 11, wherein the color projection axis is obtained by calculation. 13. A contour extraction method for extracting a contour of an object from an image, wherein an update section for obtaining a set of representative points sandwiching the update section from coordinates of a representative point on an edge of the object and an initial closed curve. An extraction step, a plurality of color projection axis candidates in the update section are obtained from the set of representative points obtained in the update section extraction step and the image, and the plurality of color projection axis candidates are data that can be subjectively evaluated by an operator. And presents it to the operator. The color projection axis calculation step of determining the color projection axis by the auxiliary input by the operator, the color projection axis obtained in the color projection axis calculation step and the update section extracting means A contour curve generating step of generating a contour curve in the update section from the image using the set of representative points obtained in the step; and combining the contour curve in the update section generated in the contour curve generating step into a closed curve, Contour closed curve Contour extraction method characterized by comprising the product contour closed curve synthetic steps. 14. The color projection axis calculation step obtains a plurality of color projection axis candidates by applying an algorithm with changed parameters or a plurality of algorithms, and allows the operator to subjectively evaluate the plurality of color projection axis candidates. 14. The contour extraction method according to claim 13, wherein the color projection axis is converted into data and presented to the operator, and the color projection axis is determined by an auxiliary input by the operator. 15. The color projection axis calculation step, comprising: calculating a plurality of color projection axis candidates in an update section from the set of representative points obtained in the update section extraction step and the image; An outline curve generating step of generating an outline curve including each of the plurality of color projection axis candidates obtained in the color projection axis calculation step, and an operator viewing the plurality of outline curves generated in the outline curve generation step. 14. The contour extraction method according to claim 13, further comprising a color projection axis selection step of selecting a color projection axis included in the contour curve selected by the subjective evaluation. 16. The color projection axis candidate calculating step includes: calculating a plurality of optimum color projection axis candidates in each of the neighboring areas of the set of representative points; A color projection axis synthesis candidate step of synthesizing a color projection axis candidate that maximizes the contrast around an edge in the update section using the optimal color projection axis candidate calculated in the intra-region optimal color projection axis candidate calculation step. 16. The contour extracting method according to claim 15, wherein: 17. The near-region optimal color projection axis candidate calculating step includes: detecting a plurality of linear edge component candidates based on the image and the representative point in each of the neighboring regions; A color information acquisition step of acquiring color information around a plurality of linear edge component candidates detected in the straight edge detection step; and a plurality of color projection axis candidates from the color information acquired in the color information acquisition step. 17. The contour extraction method according to claim 16, further comprising a step of calculating a candidate color projection axis. 18. The color information acquiring step includes sampling a periphery of the linear edge component detected in the linear edge detecting step, and generating a histogram of a color distribution classified into one side and the other side of the edge. Claim 1 characterized by the following:
7. The contour extraction method according to 7. 19. The color information obtaining step includes the steps of: when a straight line approximating an edge is placed on an edge of an image, passing through a point on the straight line, and being separated from the straight line by a predetermined distance in the vertical direction. 19. A contour extraction method according to claim 18, wherein two circles having the same radius are placed in the circle, and colors in the area of the circle are sampled. 20. The color projection axis candidate calculating step includes the steps of connecting the centroids of the distributions on one side and the other side of the edge in the color space of the color distribution information obtained in the color information obtaining step to the color projection axis candidate. 18. The contour extraction method according to claim 17, wherein the calculation is performed as follows. 21. The color projection axis candidate calculating step, wherein the ratio between the intra-class variance and the inter-class variance of the distribution on one side and the other side of the edge in the color space of the color distribution information acquired in the color information acquiring step. 18. The contour extraction method according to claim 17, wherein an axis that maximizes the following is calculated as a color projection axis candidate. 22. The color projection axis calculation step includes the steps of: computing a plurality of equally spaced points on a line connecting a set of representative points placed on an edge of the image; 16. The contour extraction according to claim 15, wherein an area having the largest color change is searched for on a vertical line segment drawn on both sides of the line segment in a predetermined length in a predetermined direction, and the color projection axis is calculated by calculation. Method. 23. The color projection axis candidate calculating step obtains a color change vector on the vertical line segment, obtains a color change maximum point having a maximum size, and calculates the color change maximum point from an average value of the color change maximum points. The method according to claim 15, wherein the color projection axis is obtained by calculation. 24. A program recording medium which records a contour extraction program for extracting a contour of an object from an image, wherein an update section is inserted from coordinates of a representative point on an edge of the object and an initial closed curve. An update section extraction step of obtaining a set of representative points, and a plurality of color projection axis candidates in the update section are obtained from the set of representative points and the image obtained in the update section extraction step, and the plurality of color projection axis candidates are A color projection axis calculation step of converting the data into data that can be subjectively evaluated by the operator, presenting the data to the operator, and determining a color projection axis by an auxiliary input by the operator; and a color obtained in the color projection axis calculation step. A contour curve generating step of generating a contour curve in the update section from the image using the projection axis and the set of representative points obtained by the update section extracting means; and a contour in the update section generated in the contour curve generating step. Program recording medium by combining a line closed curve, characterized in that it records a program and a contour closed curve synthesizing step of generating a new contour closed curve.