JPH0773317A - Image recognizing device - Google Patents

Abstract

PURPOSE:To provide an image recognizing device which can take optimum matching with the model of an object given in advance even when a contour part is separately extracted by the influence of noise or the shape of the object is affected by deformation or the like. CONSTITUTION:This device is provided with a model storage means 13 for storing the model feature of the object as the set of contour components, image pick up means 11 for picking up the object and transmitting a video signal, image picking up and storage means 12 for converting the video signal into a digital image and storing this digital image, and image processing means 14 for extracting a straight line component or a curved line component corresponding to the contour from the storage image, dividing the straight line component or the curved line component into groups while considering the connection relation of the straight line component or curved line component, transforming it into the shape of an image feature corresponding to the model feature and taking the optimum matching between the model feature and the image feature.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition apparatus for recognizing an object photographed in an image such as clothes and humans.

[0002]

2. Description of the Related Art Conventionally, with respect to an image recognition apparatus, an object has been extracted from an image and its shape has been recognized by a method described in the following documents, for example. Reference 1: RMHaralick and LG Shapiro, Computer and
Robot Vision, Addison Wesley, 1993, pp.458-465. Reference 2: NMNasrabadi and Y.Liu, “Stereo vision
correspondence usinga multichannel graph matching
technique, "Image and Vision Computing, Vol.7, No.
4, 1989, pp.237-245. Reference 3: R. Horaud and T. Skordas, “Stereo correspon
dence through feature grouping and maximal cliqu
e, "IEEE Transactions on Pattern Analysis and Mach
ine Intelligence, Vol.11, No.11, 1989, pp.1168-118
0. Reference 4: Akira Amano, Yoshiyuki Sakaguchi, Michihiko Mino, Katsuo Ikeda "Snake using sample contour model", IEICE Transactions D-II, Vol.J76-D-II, No.6, pp.1168-1176, June
1993.

[0003]

The method disclosed in Reference 1 is based on a mathematical theory of how to search for an optimal matching between a model of a given object and image features extracted from an image. As stated. in this way,
Calculate the degree of difference and similarity between the relationship graph formed by the model feature group and the relationship graph formed by the image feature group, thereby calculating how similar the model feature group and the image feature group are, The most similar one is the optimum correspondence. This method is considered only when the extracted feature component matches the image feature as it is. Therefore, it is impossible to directly match the model feature and the image feature when the image feature is extracted from the image in a divided form due to the influence of noise or the like.

A method for solving a matching problem regarding a practical image containing a lot of noise components is disclosed in Reference 2.
And 3 are shown. The methods presented in these references are stereo stereo that can handle noisy images.
It shows the matching method. When a straight line or a curved line forming the contour component of the object is divided due to the influence of noise or the like, the divided components are appropriately grouped, and a relational graph associating the correspondence between the left and right is created. Then, it reduces to the problem of searching for the maximum clique in the match graph created from the relation graph. The above problem is limited to the case where the left and right images are similar as a stereo image. However, if the object is represented by an extractive model and an object whose shape is deformable is to be handled, the model features and We could not think of matching between image features.

The Snake's method shown in Reference 4 provides a method for extracting the contour component of an object from an image. It introduces a certain energy function corresponding to the model of the object and finds the contour that minimizes the energy function. This method has the feature that contour features can be extracted even if the image features are deformed, and it is effective when the background is not complicated and the initial contour model is given sufficiently, but the background becomes complicated. Or
It has the drawback that it is difficult to use when the initial contour is not provided in sufficient detail.

The image recognition apparatus of the present invention has been made in view of such a problem, and its purpose is to extract the contour component by dividing it due to the influence of noise or to extract the shape of the object. An object of the present invention is to provide an image recognition device capable of optimal matching with a model of an object given in advance, even if affected by deformation or the like.

In addition, when there are disturbances such as noise and deformation and there are a plurality of models, it is possible to recognize an object included in an image and efficiently classify the objects into one of the models. Another object of the present invention is to provide an image recognition device that can be used.

[0008]

In order to achieve the above object, the image recognition apparatus of the present invention comprises a model storage means for storing model features of an object as a set of contour components, and an image of the object photographed. An image capturing means for transmitting a signal, an image capturing and storing means for converting the video signal into a digital image and storing the digital image, and a straight line component or a curved line component corresponding to the contour from the stored image. Extracting, grouping the straight line component and the curved line component in consideration of the connection relation of the straight line component and the curved line component, and converting them into the form of the image feature that can correspond to the above model feature, And an image processing means for optimally matching the features.

[0009]

That is, in the image recognition apparatus of the present invention,
First, the model features of the object are stored as a set of contour components. Next, the object is photographed and converted into a digital image, and the converted digital image is stored. Then, a straight line component or a curved line component that corresponds to the contour is extracted from the stored image, and the straight line component or the curved line component is grouped in consideration of the connection relation of the straight line component or the curved line component. The image feature is converted into the form of the image feature that can correspond to, and optimal matching is performed between the model feature and the image feature.

Therefore, (1) even if the contour component is divided and extracted due to the influence of noise, or the shape of the object is affected by deformation or the like, optimum matching with a given model of the object is taken. Is possible, and
(2) Even when there are disturbances such as noise and deformation, and there are multiple object models, it is possible to accurately recognize the objects included in the image and to efficiently identify one of the models. Can be classified.

[0011]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is a basic configuration diagram of the first embodiment. The image recognition apparatus in this embodiment comprises at least an image capturing means 11, an image storage means 12, a model storage means 13, and an image processing means 14.

The image photographing means 11 is a means for photographing an object to be recognized by an image pickup device such as a television camera and transmitting a video signal 15 which is an output signal thereof. The image storage means 12 is means for digitally converting the video signal 15 sent from the image capturing means 11 into a digital image signal and storing the image signal as an image 16. This memory location may be a frame memory,
It may be a memory arranged in the computer.

The model storage means 13 is a database for storing models of one or a plurality of objects. The model may be a two-dimensional model or a three-dimensional model. In general, straight line components and curved line components are considered as contour components that characterize an object, and they are defined as model features.

For example, in the case of the garment shown in FIG. 2, the model features of the garment are represented by the straight line or curved line components 101, ..., 113 that form the contour of the garment. Then, the attributes and relationships of the model features themselves are calculated in advance, and the overall features of the model are structurally described as an abstract model.

Then, the features that make up the model (for example,
Curve components and straight line components) are named model primitives. The attributes are the length and direction of the model primitive,
A curvature, a two-dimensional position, etc. can be considered, and as a relationship, the distance between the center of gravity of the model primitives, the angle formed between the model primitives, the two-dimensional positional relationship between the model primitives (upper and lower relationships, left-right relationship) Shall be represented.

An example of this relationship is shown in FIG.
In FIG. 3, there are two straight line components or curved line components 20.
1, 202 indicates the attribute and the relationship. As the attributes of the model primitive 201, the length 203 of the model primitive 201, the model primitive 201
The direction 204 of the model primitive 201 and the position 205 of the center of gravity of the model primitive 201 are considered. As the relationship between the model primitives 201 and 202, the distance 208 between the center of gravity, the direction 209 of the straight line connecting the center of gravity, and the angle 21 between the model primitives
Consider 0 as a relationship. In addition, the vertical relation between the primitives and the left-right relation may be expressed as a logical relation such as 0, 1.

These attributes and relationships are calculated in advance for the salient model primitives that make up the model and stored in the model storage means 13 as a model of the object. In the present embodiment, such a method of describing an object will be referred to as a structure description.

A more mathematical definition of this structural description is as follows. The structural description C is represented by C = (P, R) (1). Here, P is a set of primitives, and R is a set representing the relationship between the primitives.
The set R of this relation is a set representing the relation of N terms between primitives.

The set P of primitives can be written in the form P = {p ₁ , p ₂ , ..., P _n } (2). Where p _i represents the i th primitive, which is

[0020]

[Equation 1] It is represented by. Here, A represents a set of attribute names, and V represents the value of the attribute. For example, if a line segment is taken as an example of p _i , A is the length of the line segment and the slope of the line segment.
It is composed of orientation θ, centroid centroid of line segment, etc., and A = {length, orientation, centroid} (4) can be considered. Here, in Expression (3), a symbol that is not an equal sign is used to represent the relationship between p _i and A × V, because the value may not be defined by the primitive depending on the attribute. .

Now, the relation set R represents the N-term relation of N primitives (N = 1, 2, ...). Therefore, R can be written in the form of R = {R ₁ , R ₂ , ..., R _k } (5), and each R _k is R _k = (NR _k , r _k ) (6) where NR _k is a representation of the set of quantities (or quantity indicated by the relation) which represents the intensity of the name and its relationship k th relationship, r _k denotes the M _k term relationship M _k-number of primitives. That is,

[0022]

[Equation 2]

Considering the case of FIG. 3, for the primitive 201 (p ₁ ) and the primitive 202 (p ₂ ), A = {length, orientation, centroid}, NR1,
The name of the relationship between NR2 and NR3 is c-distance, c-angle, pa
ngle. Here, c-distance is the distance between the middle points of the primitives, c-angle is the angle formed by the line segment connecting the middle points of the primitives with the x axis, and p-angle is the angle formed between the primitives. This state is shown in FIG. Then, each quantity defined above can be calculated as follows:

P ₁ = {(length 20.7), (orientation 117.0), (centroid (10.6,12.7))} p ₂ = {(length 13.2), (orientation 67.8), (centroid (40.5,10.9))} R ₁ = {(c-distance 36.5), {p ₁ , p ₂ }} R ₂ = {(c-angle 3.4), {p ₁ , p ₂ }} R ₃ = {(p-angle 49.2), {p ₁ , p ₂ }} (8)

The image processing means 14 (1) extracts a straight line component or a curved line component constituting the contour of the object from the image, and (2) a connection relation between the extracted straight line components or curved line components. In consideration of the above, the straight line component and the curved line component are grouped and converted into the form of the image feature that can correspond to the model feature of the object model, and (3) the above image feature is the same as the case of the model feature. And (4) search for the optimum matching between the model features described above and the image features, and what model the object captured in the image corresponds to. It is a means to determine whether to do.

Next, the operation of this embodiment will be described. In the present embodiment, as an example of the processing of recognizing the clothing image, the contour corresponding to the object model given in advance is extracted from the image including the clothing photographed as the image recognition target. I will give you an explanation. Of course, the method described in the present embodiment is not limited to only the classification of the clothing patterns, and the target object included in the actually photographed image such as the classification of the coat of arms and the classification / analysis of the design image can be obtained. It is obvious that the present invention can be applied to a process of associating and classifying a model, and a process of recognizing and classifying a human image, a human face, and the like from an image.

First, in classifying the clothing patterns given in this embodiment, the model storing means 13 has one or a plurality of clothing patterns as shown in FIG. For example, in Figure 4, as a model of clothing, "double suit",
"Single suit", "tent dress" etc. are included. In this embodiment, when the natural image shown in FIG. 6A is given, it is necessary to recognize whether or not there is a contour portion corresponding to the “double suit” shown in FIG. Think The model storage unit 13 sends the attributes and relationships of the model features 401, 402, ..., 415 as shown in FIG. 5 to the image processing unit 14 for this task.

In the image processing means 14, the model storage means 1
.., 415, the salient model features that can be extracted from the image are first selected. For example, the length of a model feature is adopted as the selection criterion, a model feature whose length exceeds a certain threshold is selected, and the attributes and relationships of such salient model features are stored. In FIG. 5, 402, 404, 405, 406, 407, 40
8,410,412,413,414,415 are selected as salient model features and their attributes and relationships are stored.

The image photographing means 11 generates an image including the double suit as the object, and sends the image signal 15 to the image storage means 12. The image storage means 12 receives the video signal 15, digitally converts the video signal 15, and stores the digital signal. The image is, for example, as shown in FIG. In the case of this image, in addition to clothes, a human face, a background object, characters included in the upper end, and the like are mixed. One of the problems given to the image processing means 14 is to detect a portion corresponding to the salient model feature in FIG. 5 from this image.

The image processing means 14 obtains the digital image stored in the image storage means 12, and first, from among them,
A smooth curve component corresponding to a salient model feature is extracted. FIG. 6B shows this state. As this method, the method represented by the curve component extraction device of Japanese Patent Application No. 05-030732 can be used for extraction. Of course, the method of extracting the curve component is not limited to the above method. However, when a deformable object such as clothes is photographed by a TV camera or the like, a curved line component corresponding to the contour of the clothes or The straight line component is not necessarily extracted continuously, but the straight line component and the curved line component, which should be originally one, are divided and extracted. These problems are caused by the above-mentioned literatures 2 and 3.
Also listed in.

However, since the model feature in the object model given in FIG. 5 is not considered until it is divided, the model feature and the extracted contour component are in a one-to-one correspondence as they are. It turns out to be difficult to take. In order to deal with such a problem, the image processing means 14 described in the present embodiment adopts a special method to solve the problem of dividing the contour component. This method will be described below.

FIG. 7 shows a method according to the present embodiment. A method for extracting an optimum contour component corresponding to a model feature given in advance even if the extracted contour component is fragmented is shown. It is shown. The overall flow will be briefly described with reference to FIG. 7, and then detailed description will be given.

Given the image 505, the contour components as shown in FIG. 6 (b) are first extracted. Now, the contour component thus extracted will be named a fragment. Then, the fragment 50 thus extracted
An image feature capable of considering the correspondence with the model feature (model primitive) in the object model is generated from 6 (these image features are named as image primitives), and then configured by the image primitive 507. The description is calculated, and the model storage means 1 is stored in advance.
The structure description of the model and the structure description of the image primitive stored in 3 are compared, the optimum correspondence between the model primitive and the image primitive is searched, and finally the fragment corresponding to the model is selected.

The above flow will be described in detail. A method of generating the image primitive 507 from the fragment 506 which is the contour component extracted in FIG. 7 will be described. Figure 8 now
It is assumed that there are two fragments 601 and 602 as shown in FIG. At this time, both of the case where two fragments are combined to form one image primitive and the case where two fragments separately form an image primitive are considered. Which of the first case and the second case is correct cannot be known without actually considering the upper model primitive. Therefore, at the stage of generating this image primitive, it is important to leave both possibilities as possible candidates if both are possible. Then, depending on how to express the possibility, consider the following method for this.

As shown in FIG. 8, two fragments 6
Check the connection relationship between 01 and 602. Fragment 601
Consider tangents 603 and 604 at the adjacent endpoints of fragment 602. Angle 6 between tangent line 603 and tangent line 604
08 (θ), the distance 606 (d ₁₂ ) from the fragment 601 to the tangent 604, and the tangent 6 from the fragment 603.
Consider the distance 607 (d ₂₁ ) to 04 and evaluate with these whether fragments 601 and 602 can constitute an image primitive. Basically, the smaller the angle 608 (θ), the distance 606 (d ₁₂ ) and the distance 607 (d ₂₁ ) is,
It can be seen that the two fragments 601 and 602 are highly likely to form one primitive. Specifically, d = min (d ₁₂ , d ₂₁ ) (9) gives the minimum value between d ₁₂ and d ₂₁ , and the following scale dist _p = f ₁ (θ) + f ₂ (d ) Consider (10). Here, various functional forms of f ₁ ( ^* ) and f ₂ ( ^* ) can be considered. For example, regarding θ and d, it is 0 when it is in a certain range, and otherwise
A logical function that becomes inf (hereinafter, inf represents an infinite size) may be considered. That is, with certain constants θ ₀ and d ₀ as threshold values, f ₁ (θ) = 0 for | θ | < θ ₀ f ₁ (θ) = inf for | θ |> θ ₀ (11) f ₂ (d ) = 0 for d < d ₀ f ₂ (d) = inf for d> d _{0 Further} , the following linear function or non-linear function may be considered.

F _i (x) = α _i x + β _i (11 ′) where i = 1, 2, and to determine the coefficients such as α _i , β _i (i = 1, 2), the actual Such a situation can be estimated from the sample image by performing statistical processing or the like. However, the function value shall be defined to take a positive or zero value.

Further, in a practical process, when θ, d is larger than a certain threshold value, two fragments 60 are included.
It may be assumed that 1, 602 cannot generate one primitive. By using this method, the number of candidates for primitive generation can be reduced.

In the above discussion, the case of two fragments has been assumed, but three or more fragments may generate one primitive. In this case,
It can be realized by expanding the case of the above two fragments. Specifically, for example, dist _p when n fragments form a primitive
I will consider. In this case, dist _p similar to that in Expression (10) is calculated between two adjacent primitives i and i + 1. Now let dist _p between fragments i and i + 1 be
If we write dist _p (i), dist _p between n fragments 1, 2, ..., N is dist _p = dist _p (1) + dist _p (2) + ... + dist _p (n-1) It can be defined by ≧ 0 (12).

In this way, for example, as shown in FIG.
Given fragments 701, 702, 703,
From the practical combination of these fragments, if they are considered to generate a primitive,
Primitives 704, 705, 706, 707, 70
Six image primitives of 8,709 are generated. In this way, the image primitives that can be generated from the fragments arranged substantially in parallel are registered.

In the next step 503, the structure description similar to that performed with the model primitive is performed for the image primitive 507 thus generated. That is, the structural description as shown in FIG. 3 is performed on the generated image primitive. However, since the image primitive 507 generated in step 502 is generated from the fragment 506, the description of the structure of the image primitive is performed by adding two attributes indicating the fragments forming the primitive.

In the image primitive 709 shown in FIG. 9, the fragments 701, 702, 70 constituting the image primitive 709 are included.
Set {701, 702, 703} consisting of 3 is fragment
Dist _p, which is a measure for evaluating the generation of that primitive, is registered as a value for an attribute named
Registered as a value named st _frag . For example, for the primitive 709 (q _j ), the attribute expression of the primitive is q _j = {(length 30.7), (orientation 127.0), (centroid (14.6,12.2)), (fragment {701,702,703}), (dist _frag 10.7)} It is expressed in the form (13).

In the next step 504, step 503 is executed.
The optimum correspondence between the model primitive and the image primitive is searched for by using the structure description created in (3) and the structure description of the model stored in the model storage means 13. To explain this method, FIG. 10 is used. FIG. 10A shows
Model primitives 801 (p ₁ ), 802 (p ₂ ),
803 (p ₃ ) and 804 (p ₄ ) are shown in FIG.
Is the extracted fragments 811 (f ₁ ), 812
(F ₂ ), 813 (f ₃ ), 814 (f ₄ ), 815
(F ₅ ), 816 (f ₆ ), 817 (f ₇ ), 818
Represents (f ₈ ).

FIG. 10C shows image primitives 821 (q ₁ ) and 822 generated from these fragments.
(Q ₂ ), 823 (q ₃ ), 824 (q ₄ ), 825
(Q ₅ ), 826 (q ₆ ), 827 (q ₇ ), 828
It represents (q ₈ ), 829 (q ₉ ), and 830 (q ₁₀ ). Here, the primitive q ₁ is from the fragment f ₁ , the primitive q ₂ is from the fragment f _2, and the primitive q is
₃ is generated from the fragment f _3, and the primitive q ₄ is generated from the fragments f ₂ and f ₃ .

Similarly, q ₆ is from f ₅ , q ₇ is from f ₆ ,
From q ₈ is f _7, q ₉ from f _8, q ₁₀ are those produced from f ₇ and f _8. Note that the primitives q ₄ and q ₁₀ are generated from multiple fragments. Structural description is performed on the primitive generated in this way.

The problem to be solved in this step is to find the optimum image primitive q for each model primitive p _i .
You will be asked for _j . In solving this problem, we must consider p _i (i = 1, 2, ...) And q _j
Since (j = 1, 2, ...) Is expressed by the structure description as described above, the optimum combination must be searched for in the sense that this structure description optimally corresponds. .

That is, considering various mappings that define the correspondence from the model primitives p _i to the image primitives q _j , a mapping that optimizes a "certain scale" that defines the evaluation of the correspondences in the mappings is considered. It is important to search. Since both the model primitive and the image primitive are expressed by the structure description, the structure description between the model primitives M = (P, R) and the corresponding structure description between the image primitives I = (can be used to evaluate the mapping. It is necessary to evaluate conformity or incompatibility (difference) with Q, S). To realize this, consider the following operation. Mapping from a set of model primitives to a set of image primitives h:

[0047]

[Equation 3] Consider the set Q _{h of} image primitives bounded by this mapping h and the set S _h of relationships. Here, Q _h represents a set in which P is mapped by the mapping h, and S _h represents a set in which R is mapped by the mapping h. Then, two structure descriptions M = (P, R) and I _h = (Q _h ,
Consider evaluating the mapping h by evaluating the difference in S _h ). The scale for this evaluation is DIST (M,
I _h ) and decompose this measure as follows.

DIST (M, I _h ) = DIST (P, Q _h ) + DIST (R, S _h ) (≧ 0) (15) where DIST (P, Q _h ) is the model primitive set P and the image primitive set. It is a scale (hereinafter referred to as a total attribute scale) for evaluating the difference seen between the attributes of Q _h ,
DIST (R, S _h ) is a scale (hereinafter, referred to as a comprehensive relation scale) for evaluating the difference between the model primitive set P and the image primitive set S _h .

(1) Calculation of DIST (P, Q _h ) The total attribute scale is calculated by comparing the attributes of the model primitive p _{i and} the attributes of the corresponding image primitive q _j by the mapping h. The comprehensive attribute scale represents how well the attributes of all model primitives are represented by the attributes of the image primitive.
Before considering this comprehensive attribute scale, the basic individual attribute scale is defined. This individual attribute scale is a scale for measuring the difference in attributes between one model primitive and one image primitive. For example, consider a straight line component as a model primitive. One of its attributes, orientat
Regarding ion θ, it can be considered that it is most likely that both the model primitive θ model and the image primitive θ image have similar values.

As described above, by comparing the attribute value of the model primitive and the attribute value of the image primitive and calculating the difference between them, it is possible to calculate how well the attribute of the model primitive is represented by the image primitive. It does. Specifically, the attribute (attribute) of each model primitive p _i name, value model) and the corresponding attributes of the image primitives q _j (attribute name,
value image), a scale di indicating the difference between attribute values
st (attribute name) is defined. For example, orientat
As a scale dist (orientation) for ion θ, dist (orientation) = 0 for │θ _model −θ _image │ < θ ₁ dist (orientation) = inf for │θ _model −θ _image using a threshold θ _1. A logical scale such as |> θ ₁ (16) may be considered, or a linear / non-linear scale as shown below may be used.

Dist (orientation) = A (| θ _model −θ _image |) dist (orientation) = B (1-exp (− | θ _model −θ _image |)) (17) Here, A and B are constants. And θ _model and θ _image represent the attribute values of the model primitive p _i and the image primitive q _j corresponding to the attribute orientation. This θ ₁ , A, B
Is a value that can be determined by experiments and experience, and dist ( ^* )
Shall be defined to take a positive or zero value.

In this way, for each attribute value, after calculating the scale indicating the difference, one model primitive p
The measure dist _ij between all attributes between _i and one image primitive q _i is defined as the sum of the measures for each attribute. Ie

[0053]

[Equation 4] Is calculated, and this is called an individual attribute scale.

As described above, when dist _ij is calculated for each image primitive corresponding to each model primitive, the scale for all model primitives is calculated, and the scale is defined as DIST (P, Q _h ). That is, the comprehensive attribute scale is defined as the sum of individual attribute scales. In particular,

[0055]

[Equation 5]

(2) Calculation of DIST (R, S _h ). This comprehensive relation scale is the relation R _m between model primitives.
It is determined whether (m = 1, 2, ...) Is established without a contradiction with the relationship S _k (k = 1, 2, ...) Between corresponding image primitives. To evaluate this consistency, we define a measure of the inconsistency between one relationship R _m between model primitives and one relationship S _k between corresponding image primitives. Specifically, the amount representing the difference between the relations represented by R _m and S _k is represented by rel _mk and is used as the contradiction scale of this relation (individual relation scale).

For example, the relation p-angle shown in FIG.
Is used as the relationship between the model primitives p ₁ and p ₂ in FIG. 10A, and the relationship is expressed by R _m , and the p-
Suppose the value of angle is Ψ _m . On the other hand, the corresponding image primitives q ₁ and q ₄ have the corresponding relation S
_Suppose it is _m and the value of its p-angle is Ψ _k . Then, rel _mk uses, for example, a threshold Ψ ₁ as a logical measure, and rel _mk = 0 for | Ψ _m −Ψ _k | < Ψ ₁ rel _mk = inf for | Ψ _m −Ψ _k | > Ψ ₁ (20) Alternatively, it can be expressed in the form of rel _mk = C | Ψ _m −Ψ _k | + D (20 ′) using an appropriate linear / non-linear function (where Ψ ₁ , C, D is a constant and rel ( ^* ) is defined to have a positive or zero value). In this way, the individual relationship measure rel _mk can be calculated between the relations, comprehensive relationship measure DIST is a measure of the differences for all relations (R, S _h) can be calculated as the sum. That is,

[0058]

[Equation 6] Can be calculated by

As described above, two scales: comprehensive attribute scale DIST (P, Q _h ) and comprehensive relation scale DIS.
A method for evaluating the mapping h by using T (R, S _h ) has been shown. This method does not include an evaluation for generating image primitives from fragments. To be more specific, it can be seen that the term evaluating the validity of the image primitive selected by the mapping h existing in the image is omitted. As mentioned earlier, this is one attribute of the image primitive dist
_frag represented it. Now the image primitive q _j
Let 's _call the dist _frag of dist _{frag (j)} . The measure DIST _Qh to evaluate the generation of the entire image primitive is
As the sum of dist _frag of each image primitive,

[0060]

[Equation 7] Can be written.

DIST (M, I) which has previously defined the term of the scale DIST _Qh for evaluating the existence of this image primitive.
while adding to _h), it is defined as follows a measure DIST _h to assess the overall mapping h.

[0062] _{DIST h = DIST (M, I} h) + DIST Qh = DIST (P, Q h) + DIST (R, S h) + In DIST _Qh (23) embodiment, the mapping of that of the measure DIST _h h I will call it a scale. With this DIST _h ,
The mapping h can be evaluated.

The above discussion shows how to evaluate the mapping h, but how h
It doesn't say what to ask for. Hereinafter, how to obtain the mapping h while using the mapping scale DIST _h will be described. DIST _h

[0064]

[Equation 8] Rewrite Here, it is defined as d _ij = dist _ij + dist _{frag (j)} (≧ 0) (25). This _dij will be called a primitive attribute scale. Further, r _ij is a calculation of a difference measure between a set A _i including only the following relations among the relation set {R _m } and the corresponding relation set {S _k }. . This A _i is a set of relationships consisting of B = a subset of model primitives {p ₁ , p ₂ , ..., P _i } (27) C = a subset of model primitives {p ₁ , p ₂ , ..., p _When a set of relations (28) consisting of _i-1 } is given, it is expressed as A _i = B−C. That is, A _i can be added to the set of relations by newly adding the model primitive p _i to the subset of model primitives {p ₁ , p ₂ , ..., P _i-1 }. Represents a subset of. Regarding the _four model primitives p ₁ , p ₂ , p ₃ and p ₄ shown in FIG. 10A, a single relation p-an
Taking gle as an example, consider the set of relations A _i as follows. First, the relations R ₁ , R ₂ , ..., R ₆ are represented by R ₁ = {(p-angle Ψ ₁₂ ), {p ₁ , p ₂ }}, R ₂ = {(p-angle Ψ ₁₃ ), {p ₁ , p ₃ }}, R ₃ = {(p-angle Ψ ₁₄ ), {p ₁ , p ₄ }}, R ₄ = {(p-angle Ψ ₂₃ ), {p ₂ , p ₃ }}, R ₅ = Let {(p-angle Ψ ₂₄ ), {p ₂ , p ₄ }}, R ₆ = {(p-angle Ψ ₃₄ ), {p ₃ , p ₄ }}. At this time, A _i can be obtained as follows.

A ₁ = φ, A ₂ = {R ₁ }, A ₃ = {R ₂ , R ₄ }, A ₄ = {R ₃ , R ₅ , R ₆ } Furthermore, dr _ij = d _ij + r _ij ( 30) and dr _ij are defined. The definition of r _ij and dr _ij as described above is for sequentially calculating DIST _h for i of h: p _i → q _j . The first term in equation (30) is the model primitive p _i and the image primitive q _j.
Represents the term regarding attributes and the term regarding image primitive generation (individual primitive attribute scale).
The term is a scale (individual relation scale) representing the relation between the model primitives and the image primitives. Therefore, the mapping h to be obtained is this DIST _h
Is the minimum.

To create the mapping h, (1) For each model primitive p _i , d _ij <thre
The image primitive q _j that becomes the hold is selected and listed. For example, regarding the model primitive and the image primitive shown in FIG. 10, p ₁ : {(q ₁ , d ₁₁ )} p ₂ : {(q ₂ , d ₂₂ ), (q ₃ , d ₂₃ ), (q ₄ , _{d 24), (q 5,} d 25)} p 3: {(q 6, d 36), (q 7, d 37)} p 4: {(q 8, d 48), (q 9, d 49 ), (Q ₁₀ , d ₄₁₀ )} (31) (2) The image primitives corresponding to the respective model primitives are arranged in ascending order by the individual primitive attribute scale _dij . In addition, as in (1), the number of image primitives that may correspond to the listed model primitives is not limited to one. Further, for all model primitives, the corresponding image primitives cannot always be extracted from the image, so that the image primitives that truly correspond do not necessarily exist in this list. Therefore, at the end of each list, the symbol ni indicating that it cannot correspond to any image primitive
Include l. The result of performing this operation on the primitive of FIG. 10 is p _{1 ::} ((q ₁ , d ₁₁ ), (nil, d _1nil )} p _{2 ::} ((q ₅ , d ₂₅ ), (q ₄ , _{d 24), (q 3,} d 23), (q 2, d 22), (nil, d 2nil)} p 3: {(q 7, d 37), (q 6, d 36), (nil, d _3nil )} p ₄ : {(q ₁₀ , d ₄₁₀ ), (q ₈ , d ₄₈ ), (q ₉ , d ₄₉ ), (nil, d _4nil )} (32). Here, d _inil represents the individual primitive attribute scale when there is no image primitive corresponding to the model primitive p _i . (3) For the list of model primitives and image primitives created in (2), find the combination that minimizes the value of equation (24). As this method, considering the model primitives p _i sequentially in combination with the image primitive candidates q _j included in the above list,
DIS _ij + r _ij can be calculated by the combination
In addition to T _h , the combination of (p _i , q _j ) that finally minimizes DIST _h may be searched. At this time, DIST for one solution found so far
Branch and bound method (pruning method) by comparing with _h etc.
Thus, the efficiency of the search can be improved.

FIG. 11 is a diagram for explaining a method for searching for an optimum combination according to this method. In FIG.
The optimal search starts at the Start position. First, the image primitive q ₁ located at the beginning of the list of model primitives p ₁ is selected, and dr ₁₁ which is the partial sum of DIST _h is calculated. Then q _{5 at} the head of the list of model primitives p ₂ is selected and is the partial sum of DIST _h.
dr ₁₁ + dr ₂₅ is calculated. Further model primitive p
Q _{7 at} the head of the list of ₃ is selected, and the partial sum of DIST _h , dr ₁₁ + dr ₂₅ + dr _37, is calculated. Then, q _{10 at} the head of the list of the model primitive p ₄ is selected, and dr ₁₁ + dr ₂₅ + dr ₃₇ + corresponding to the entire DIST _h is selected.
dr ₄₁₀ is calculated. At this point, the mapping h is {(p
₁ , q ₁ ), (p ₂ , q ₅ ), (p ₃ , q ₇ ), (p
₄ , q ₁₀ )} is obtained. Now, let's write this mapping as h ^* . This mapping h ^* is not always optimal.

[0068] However, if if there is another in optimum h of this mapping, DIST _h this mapping h ^* of DIST _h corresponding to the mapping h ^*
Must be a smaller value. Utilizing this fact, the search for the optimum solution will be further carried out. That is, the partial sum of DIST _h is calculated in the process of searching, and the partial sum becomes a value larger than DIST _h ^{* of} h ^* which is the best solution obtained so far. Occasionally, the search based on the branch of the solution is canceled, backtracked once, and moved to the search of another branch.

[0069] In FIG. 11, when Yuku repeatedly such backtracking, as better solutions than h ^* of _{^{DIST h * {(p 1,}} q 1), (p 2, q 5), (p 3,
Since q ₆ ), (p ₄ , q ₁₀ )} is obtained, the optimal solution h ^* obtained so far is given by this {(p ₁ , q ₁ ),
While updating with (p ₂ , q ₅ ), (p ₃ , q ₆ ), (p ₄ , q ₁₀ )}, DIST _h ^* also dr ₁₁ + dr ₂₅ + dr ₃₇ +
Update to dr ₄₁₀ .

If this operation is further continued, this DIST
Solutions that give smaller values than _h ^* {(p ₁ , q ₁ ), (p
₂ , q ₄ ), (p ₃ , q ₇ ), (p ₄ , q ₁₀ )} are obtained. Update the mapping h ^* again with this solution. This operation is continued for the solution space shown in FIG. 11 until no solution gives a value smaller than DIST _h ^* obtained so far, and finally the mapping h ^* is optimized. Will be output as a solution.
For example, in the case of FIG. 11, the optimum combination of (p _i , q _j ) obtained by such a method is (p ₁ ,
q ₁ ), (p ₂ , q ₄ ), (p ₃ , q ₇ ), (p ₄ , q
₁₀ )

The above is summarized as follows. Procedure 1 Procedure SOLUTION () 1. Let the initial value of the solution S be the empty set.

S: = φ; 2. DIST _{h is set} to 0. DIST _h : = 0; Let P be the set of model primitives.

P: = {p ₁ , p ₂ , p ₃ , ...}; status value is not yet. status: = not yet; 5. Let the optimal solution S ^* be the empty set.

S ^* : = φ; 6. The mapping scale DIST _h ^* of the optimum solution is set to 0. DIST _h ^* : = 0; 7. Let C be a set of C ₁ , C ₂ , C ₃ , ....

C = {C ₁ , C ₂ , C ₃ , ...}; Here, it is assumed that each C _i is arranged in the order of smaller _dij . 8. Procedure MATCH (P, C, S, DIST _h , S
^* , DIST _h ^* , status). 9. Report the optimal solutions S ^* , DIST _h ^* . 10. End Procedure 2 Procedure MATCH (P, C, S, DI
ST _h , S ^* , DIST _h ^* , status) 1. If S ^* is not an empty set and DIST _h > DIS
If T _h ^* , return from this procedure MATCH. 2. If P is an empty set, (1) If S ^* is an empty set, then a new optimal solution has been found, and the following (a), (b), (c),
The process of (d) is performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH.

(2) If S ^* is not an empty set, then DIST _h
<DIST _h ^* , I found a better solution, so
The following processes (a), (b), (c) and (d) are performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH. 3. Select the first element p _i of P and remove p _i from P.

3. P: = P- {p _i }; Unless C _i is an empty set, the following (1), (2),
(3), (4), and (5) are repeated.

[0082]: (1) the beginning of the elements of C _i and q _j, C _i
Remove q _j from. (Q _j = nil may be used.) C _i : = C _i − {(q _j , d _ij )}; (2) The value obtained by adding (p _i , q _j ) to S is S. Substitute in new.

S new: = S + {(p _i , q _j )} (3) After calculating r _ij , calculating dr _ij = d _ij + r _ij ,
Add dr _ij to DIST _h and add DIST _{h Let's say new} .

DIST _{h new} = DIST _h + d _ij + r _ij ; status new to not substitute yet. (4) MATCH (P, C, S recursively new, DIS
_{Th new} ; S ^* , DIST _h ^* , status run new). 5. This procedure returns from MATCH.

A method of searching for an optimal solution using the following variables in the above procedures 1 and 2 will be shown. Let the set of model primitives be P = {p ₁ , p ₂ , p ₃ ,
}, C is a set of sets of image primitives and their attribute scales that may correspond to each model primitive p _i.
i = {(q _i1 , d _i1 ), (q _i2 , d _i2 ), ...}, C _i
Let C be a set having (i = 1, 2, 3, ...) As its constituent elements. The solution representing the mapping h is S, the mapping scale of S is DIST _h , the solution representing the optimum solution is S ^* , and the mapping scale of S ^* is DIST _h ^* . Here, the solution S is S = {(p
₁ , q ₁ ), (p ₂ , q ₂ ), ...} As a set of combinations of model primitives and image primitives. In addition, a sign indicating whether a solution was found
status. Here, the value of status is found if at least one solution has been found, and not if not yet found. Let's say yet. At this time, the solution search can be represented by the above two procedures.

Of these, procedure 1 is the main procedure for performing the entire solution search. First, in step 1, the initial value of the solution S is set to the empty set. In step 2, the initial value of the mapping measure DIST _h corresponding to the solution S is set to zero. In step 3, the set of model primitives is set to P. In step 4, the value of the sign status for notifying the result of this procedure is not Set to yet, indicating that the solution has not yet been found in the initial state.

In steps 5 and 6, the initialization regarding the optimum solution is performed. In step 7, the image primitives that may correspond to the model primitive p _i are selected in the manner described above, and the model primitive p _i is selected.
_i candidate image primitives are stored in C _i relates. Where C _i is the image primitive q _j and
The corresponding attribute measure d _{ij of} (p _i , q _j ) is calculated and C _i
= {(Q ₁ , d _i1 ), (q ₂ , d _i2 ), ...} is stored, and model primitive p ₁ ,
_{p 2, ..., p i,} ... all _{C 1, C 2, ...,} C i, ... is as a set, are stored in C.

The procedure called MATCH shown in step 8 is a recursive procedure for searching for an optimal solution. This procedure is based on the above P, C, S, DIST _h , S.
^The parameters are ^* , DIST _h ^* , and status, and the optimal solution S ^* and the corresponding DIST _h ^* and the status of the solution
status is output as a set. The details will be described below using Procedure 2. In step 9, the solution obtained in step 8 is reported, and in step 10, this procedure SOLUTION ends.

The above procedure 2 represents the recursive procedure MATCH for searching the optimum solution. When this step is called, as a parameter, a set P of model primitives not yet used at present, a set C of image primitives that may correspond to the model primitive,
The partial sum S of the search up to the present time and the partial sum DIST _h of the corresponding mapping measures, S ^* which is the optimal solution among the solutions found so far and the corresponding mapping measure DIST _h ^* , and The status is used to indicate whether a solution has been found.

In step 1, it is checked whether at least a solution has been found up to the present time, and the mapping measure DIST _h ^* of the optimum solution S ^* found so far is found ^.
And the partial sum DIST _h of the mapping measures of the partial solution that is currently performing the search. If DIST _h
If already exceeds the value of DIST _h ^* , it can be seen that the solution with this partial solution cannot be the optimal solution, so stop searching further and backtrack once. , Return from this procedure.

In step 2, it is confirmed whether or not the search for all model primitives has been completed up to the present time.
If all the model primitives are searched, the current partial solution S can be an optimal solution, and therefore, the current solution S or the solution S ^* found so far is checked to see which is a better solution. This step is step 2 (1). If no solution has been found so far, the solution found at this time is registered as the optimum solution up to the present. This operation is performed in step 2 (2).

In step 3, P is updated by selecting the top element p _i from the set of model primitives and removing the element p _i from P. With this operation, each time the procedure is called, the number of elements of the model primitive checked by the procedure MATCH decreases, so it can be seen that the recursive procedure ends in a finite time.

In step 4, the set C _i of candidate image primitives for the model primitive p _i selected in step 3 is checked. In C _i , since the image primitive attribute scales d _ij are arranged in ascending order, it can be said that the most promising image primitive is arranged at the head of C _i . Step 4
In (1), this most promising image primitive q _j is selected and the element for that q _j is removed from C _i . As a result, the number of elements of C _i decreases each time Step 4 is repeated, and it can be seen that MACTH, which is a recursive procedure, ends in a finite time.

In the next step 4 (2), a set of model primitive p _i and image primitive q _j is added to S, which is the partial decomposition up to now, and S is added to S. Defined as a subdivision of a new name new.

In step 4 (3), the model primitive
Bu p_i And image primitive q_j Relational measure r related to_ij
Is calculated. This relation scale r_ijTo calculate
And the model primitives in the partial solution S searched in
Needless to say that the relationship with image primitives is taken into account
Absent. R calculated in this way_ijAnd calculated in advance
D_ijIs the partial sum up to now DIST_h Is added to
Result of DIST_{h  new} Stored in. Further next
Status for step 4 (4) the value of new is not yet
Is set to.

In step 4 (4), the procedure M is recursively performed.
ATCH is executed. Note, however, that when this procedure is called, its arguments have changed. Finally, in step 5, the procedure MATCH is returned and the processing is terminated.

By adopting the above method, the mapping h ^* for determining the optimum combination (p _i , q _j ) can be obtained. According to this method, even if the fragment is fragmented and extracted in the image, it becomes possible to correspond to the optimal model, and stable recognition can be achieved as compared with the conventional method. it can.

In the following, a second embodiment for thresholding the relation scale r _ij will be described. In the first embodiment, when the image primitives corresponding to the model primitives are searched for as shown in FIG. 11, the corresponding scale r for each relation is calculated.
_{A method} of calculating _ij and adding it to DIST _h in equation (24) to find the optimum combination was shown. However, when the correspondence measure r _ij for the relationship is calculated, the value may be extremely large. This means that the relationship established between the model primitives is very unlikely to be established among the image primitives.
In such a case, even though the solution is being searched for, the intermediate solution currently being searched is already unrealizable, and the search for the intermediate solution is not continued, but a backtrack shifts to the search for another solution. Means that you can. In order to realize such a method, a certain threshold value threshold is used to determine r _ij in advance, and only a combination of model primitives and image primitives satisfying r _ij ≤threshold (33) is searched for. You can go. That is, procedure 2
This can be realized by changing (procedure MATCH of the first embodiment) to procedure 3 below.

Procedure 3 Procedure MATCH (P,
C, S, DIST _h , S ^* , DIST _h ^* , status) 1. If S ^* is not an empty set and DIST _h > DIS
If T _h ^* , return from this procedure MATCH. 2. If P is an empty set, (1) If S ^* is an empty set, then a new optimal solution has been found, and the following (a), (b), (c),
The process of (d) is performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH.

(2) If S ^* is not an empty set, then DIST _h
<DIST _h ^* , I found a better solution, so
The following processes (a), (b), (c) and (d) are performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH. 3. Select the first element p _i of P and remove p _i from P.

3. P: = P- {p _i }; Unless C _i is an empty set, the following (1), (2),
(3), (4), and (5) are repeated.

[0106]: (1) the beginning of the elements of C _i and q _j, C _i
Remove q _j from. (It may be q _j = nil.) C _i : = C _i − {(q _j , d _ij )}; Calculate r _ij associated with (p _i , q _j ).

If r _ij is larger than a threshold value threshold, the process returns to the beginning of step 4. (2) dr _ij = a d _ij + r _ij is calculated, by adding the dr _ij to DIST _h DIST _{h Let's say new} .

DIST _{h new} = DIST _h + d _ij + r _ij ; status new to not substitute yet. (3) The sum of S and (p _i , q _j ) is S Substitute in new.

S new: = S + {(p _i , q _j )} (4) MATCH (P, C, S recursively new, DIS
_{Th new} ; S ^* , DIST _h ^* , status execute new). 5. This procedure returns from MATCH.

In the above procedure 3, it is the step (1) of step 4 that is changed. The model primitives p ₁ , p ₂ , ..., P _i-1 that have been associated so far
And the corresponding image primitives (q ₍₁₎ , q ₍₂₎ ,
... q _(i-1) ) and the model primitive p selected this time
pairs of _i and the image primitives _{q j (p i, q j} ) based on the terms r _ij involved in related measures (p _i, q _j) is calculated. If this r _ij is larger than a certain threshold threshold, this (p _i , q _j ) is the set of the model primitive and the image primitive (p ₁ , q ₍₁₎ ), ...
Since it is inconsistent with (p _i-1 , q _(i-1) ), it can be seen that q _i cannot correspond to the model primitive p _i after all. Therefore, in this case, it is possible to return to the beginning of step 4 and correspond to another image primitive.

As described above, based on the second embodiment, the search for a solution that seems to be extremely impossible can be stopped in the middle of the solution search, so the search for the optimum solution can be speeded up. You can

The third embodiment for realizing the high speed pruning by utilizing the lower bound will be described below. The second embodiment is
The present invention was devised to speed up the search for the optimum solution, but the third embodiment described below can also speed up the search for the optimum solution for this embodiment. As shown in FIG. 11 and Procedure 2, in this embodiment, the search for the optimum solution is realized by the Depth-first search in consideration of pruning. As the threshold value for the pruning, DIST _{h of} the optimum solution found up to that point was used.

However, as expressed by the equation (32),
When solution candidates are given as a list in advance, a faster optimal solution search can be realized by effectively using this list and the threshold value.

Total scale DIST _h considered in the first embodiment
Is evaluated by terms of its partial sums DP _i and DC _i . DIST _h = DP _i + DC _i (34) where

[0115]

[Equation 9] In other words, DP _i is a partial sum of the DIST _h from the model primitive p ₁ of the mapping h to p _i, DC _i
Are model primitives p _{i + 1 to} p of the mapping h
_It represents the partial sum of DIST _h up to _n .

Now, in order to search for the optimum solution, the above-mentioned procedure 2 is used to search the model primitives p ₁ to p _i , and the remaining model primitives p _{i + 1} to p _n. Suppose Then, for the halfway solution of h up to this point, DI calculated in procedure 2
The partial sum of ST _h is DP _i . In such a situation, in the first embodiment, the optimal solution h found so far is
^* DIST _h ^* by the Toko of DP _i is compared, this middle solution h is evaluated can be a optimal solution. Specifically, if DIST _h ^* > DP _i , the search for a solution regarding this intermediate solution is continued, but if DIST _h ^* ≦ DP _i , this intermediate solution is discarded and backtracking to another intermediate solution is performed. It was supposed to be done.

However, the efficiency of this comparison can be improved by utilizing the list represented by equation (32). That is, in equation (35), if the model primitive p
_If the lower bound (or minimum value) of DC _i is known for various mappings h from _{i + 1} to the model primitive p _n , that is,

[0118]

[Equation 10] Is known, DP _i + DC _i ^inf is already DIS
It can be seen that this intermediate solution cannot be an optimum solution when it takes a value larger than T _h ^* . If this lower bound of DC _k is used, the solution that cannot be the optimum solution can be discarded at an earlier stage of the search, so that the optimum solution search can be speeded up.

The method of obtaining one of DC _i ^inf is as follows.
4) from the attribute measure at the beginning of the list of candidate image primitives from model primitive p _{i + 1} to model primitive p _n from d _kj (k = i + 1,
..., the sum of n) may be used. For example, now q
_{Let (i + 1)} , q _{(i + 2)} , ..., Q _(n) be the image primitives at the beginning of the list, and d _{i + 1, (i + 1)} , d
_{If i + 2, (i + 2)} , ..., d _{n, (n)} are the attribute measures,

[0120]

[Equation 11] Can be calculated by At this time, the procedures of SOLUTION and MATCH corresponding to the third embodiment are the following procedures 4 and 5.

Procedure 4 Procedure SOLUTION
() 1. Let the initial value of the solution S be the empty set. S: = φ; 2. DIST _{h is set} to 0.

DIST _h : = 0; Let P be the set of model primitives. _3. P: = {p ₁ , p ₂ , p ₃ , ...}; status value is not yet.

Status: = not yet; 5. Let the optimal solution S ^* be the empty set. S ^* : = φ; 6. The mapping scale DIST _h ^* of the optimum solution is set to 0.

DIST _h ^* : = 0; 7. Let C be a set of C ₁ , C ₂ , C ₃ , .... C = {C ₁ , C ₂ , C ₃ , ...}; Here, it is assumed that each C _i is arranged in the order of smaller _dij .

Calculate the DC _i ^inf using the image primitive at the beginning of each C _i . 8. Procedure MATCH (P, C, S, DIST _h , S
^* , DIST _h ^* , status). 9. Report the optimal solutions S ^* , DIST _h ^* . 10. End.

Procedure 5 Procedure MATCH (P,
C, S, DIST _h , S ^* , DIST _h ^* , status) 1. If S ^* is not the empty set and the first model primitive of P is P _i .

If DIST _h + DC _i ^inf > DIST _h
If ^* , return from this procedure MATCH. 2. If P is an empty set, (1) If S ^* is an empty set, then a new optimal solution has been found, and the following (a), (b), (c),
The process of (d) is performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH.

(2) If S ^* is not an empty set, then DIST _h
<DIST _h ^* , I found a better solution, so
The following processes (a), (b), (c) and (d) are performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH. 3. Select the first element p _i of P and remove p _i from P.

3. P: = P- {p _i }; Unless C _i is an empty set, the following (1), (2),
(3), (4), and (5) are repeated.

[0134]: (1) the beginning of the elements of C _i and q _j, C _i
Remove q _j from. (Q _j = nil may be used.) C _i : = C _i − {(q _j , d _ij )}; (2) The value obtained by adding (p _i , q _j ) to S is S. Substitute in new.

S new: = S + {(p _i , q _j )} (3) After calculating r _ij , calculating dr _ij = d _ij + r _ij ,
Add dr _ij to DIST _h and add DIST _{h Let's say new} .

DIST _{h new} = DIST _h + d _ij + r _ij ; status new to not substitute yet. (4) MATCH (P, C, S recursively new, DIS
_{Th new} ; S ^* , DIST _h ^* , status run new). 5. This procedure returns from MATCH.

The difference from the first embodiment is that an item for calculating DC _i ^inf is added in step 7 of procedure 4, and DIST _h ^{* in} step 1 of procedure 5 ^.
And DIST _h (DP _i ) + DC _i ^inf .

By adopting the method as in the third embodiment, the intermediate solution which cannot be the optimum solution can be discarded sooner, so that the faster optimum solution search can be realized.

The fourth embodiment for executing the rejection of nil mapping will be described below. In the first embodiment, even if the image contour component (fragment) corresponding to the model primitive is divided, the image primitive is generated from the possible combination of fragments, and the optimum matching between the model primitive and the image primitive is expressed by the formula (24). ) It asks by evaluating by the scale expressed by. This formula (2
As a term included in 4), a nil symbol is added to the end of the image primitive candidates that may correspond to each model primitive so that the characteristics as a model primitive can be matched without being displayed in the image. did.

For a particular object, it may not be considered that correspondence has been achieved unless all of the model primitives that describe the model of the object appear in the image. For example, the model primitives p ₁ , p ₂ , p appearing in FIG.
For all ₃ and p ₄ , the corresponding image primitives must be searchable.

In order to deal with such a case, the effects are realized by making the following improvements to the first embodiment. (1) The nil component added in equation (32) is removed. (2) Regarding the setting of the scale of the attribute or the relation included in the equation (24), all the model primitives corresponding to nil are defined as zero. That is, the individual primitive attribute scale d _ij = 0 regarding the attribute, the individual relation scale re
_Let l _mk = 0.

By making the above improvements, this embodiment can be realized even when all model primitives have to be extracted from the image.
The fifth embodiment for executing the search for the optimum solution by nil counting will be described below.

In the first, third, or fourth embodiment, even if there is no image primitive corresponding to the model primitive, the image primitive represented by nil is added to search for the optimum solution. I showed you how to do it. The above embodiment is realized by calculating the attribute scale d _inil and the relation scale r _ij corresponding to nil.

Meanwhile, reference: A. Kosaka and ACKak, “Fast
vision-guided mobile robotnavigation using model
based reasoning and prediction of uncertainties, "
Computer Vision, Graphics, and Image Processing-Im
age Understanding, Vol.56, No.3, Nobember, pp.271-3
29, 1992, when finding the image feature corresponding to the model feature, the number of model features and image features that can be dealt with is the maximum, and the criterion that the solution that optimizes a certain evaluation function is the optimal solution The optimality is judged. That is,
This optimality means that the number of model features mapped to nil is the minimum, and the solution that optimizes a certain evaluation function is the optimal solution. According to the same document, the criterion of such optimality is developed because the value of d _inil as described above is difficult to evaluate experimentally.
Of course, the model features described in the above literature are
It is not something that can be handled by a model of an abstract object such as the model primitive considered in this proposal. In addition, the image features that are dealt with in the above documents correspond to the fragments of the present proposal, and there is a limit that only those that can be dealt with even if they are divided are dealt with.

The fifth embodiment shows that even in the optimum solution search handled by the present proposal, it is possible to realize the optimality as handled by the above document. In order to adopt such optimality, (1) nil for all model primitives p _i
The scale d _inil for the attribute corresponding to is set to zero.

(2) Regarding the relation R _m defined between model primitives, the relation S _k between image primitives is not taken into consideration when at least one of the corresponding image primitives is nil. That is, the individual relation scale rel _mk between R _m and S _{k for} the relation scale is set to zero.

(3) Under the above conditions (1) and (2), the following criteria are used as the optimal solution criteria. 1) The optimal solution has the largest number of corresponding model primitives. (That is, the number of nil mappings of model primitives is the minimum.

2) Among the solutions satisfying the condition 1), the solution having the smallest evaluation function DIST _h is the optimum solution. The following procedure 6 represents the SOLUTION of the fifth embodiment, and procedure 7 represents the procedure MATCH of the fifth embodiment.

Procedure 6 Procedure SOLUTION
() 1. Let the initial value of the solution S be the empty set. S: = φ; 2. DIST _{h is set} to 0.

DIST _h : = 0; #nil is set to 0. # nil = 0; 3. Let P be the set of model primitives.

_3. P: = {p ₁ , p ₂ , p ₃ , ...}; status value is not Let's say yet. status: = not yet ； 5. Let the optimal solution S ^* be the empty set.

S ^* : = φ; 6. The mapping scale DIST _h ^* of the optimum solution is set to 0. DIST _h ^* : = 0; The number of nil of optimal solution # nil ^{* is set} to 0.

# Nil ^* = 0; 7. Let C be a set of C ₁ , C ₂ , C ₃ , .... C = {C ₁ , C ₂ , C ₃ , ...}; Here, it is assumed that each C _i is arranged in the order of smaller _dij . 8. Procedure MATCH (P, C, S, DIST _h , #ni
l, S ^* , DIST _h ^* , #nil ^* , status). 9. Report the optimal solutions S ^* , DIST _h ^* . 10. End Procedure 7 Procedure MATCH (P, C, S, DI
ST _h , #nil, S ^* , DIST _h ^* , #nil ^* , status) 1. If S ^* is not the empty set and #nil># nil ^* , return from this procedure. If S ^* is not an empty set, # nil = # nil ^* , and DIST _h > DIST _h
If ^* , return this procedure MATCH. 2. If P is an empty set, (1) If S ^* is an empty set, then a new optimal solution has been found, and the following (a), (b), (c),
The process of (d) is performed.

(A) Substitute S for S ^* :
S ^* : = S; (b) Substitute DIST _h for DIST _h ^* : DIST
_h ^* : = DIST _h ; #nil ^* = # nil; (c) Substitute found for status.

(D) Return from this procedure MATCH. (2) If S ^* is not an empty set and (#nil <# nil ^* or (# nil = # nil ^* and DIST _h <DIST _h ^* )), a better solution was found, so (A),
Processes (b), (c) and (d) are performed.

(A) Substitute S for S ^* :
S ^* : = S; (b) Substitute DIST _h for DIST _h ^* : DIST
_h ^* : = DIST _h ; # nil ^* :
= # Nil; (c) Substitute found for status.

(D) Return from this procedure MATCH. 3. Select the first element p _i of P and remove p _i from P. P: = P- { _pi }; 4. Unless C _i is an empty set, the following (1), (2),
Repeat (3) and (4).

(1) Let the leading element of C _{i be} q _j . (2) If q _j is nil, (a) remove (nil, 0) from C _i : C: = C _i −
{(Nil, 0)}; (b) the S (p _{i, nil)} a plus S Substitute in new: S new: = S {(p _i , nil)}; (c) DIST _h new: = DIST _h , #nil new:
= # Nil + 1.

[0159] If unless q _j is nil, 'remove from _{C i (q j, d ij} ): C i: C i - {(q j, d ij)}; (b (a)') S To (p _i , q _j ) is added to S Substitute in new: S new: = S + {(p i, q j)}; (c ') After calculating r _ij, calculate the _{dr ij = d ij + r ij} , by adding dr _ij to DIST _h DIST _{h new} to: DIST _{h new} = DIST _h + d _ij + r _ij ; new: = #nil.

Status new = not Let's say yet. (3) MATCH (P, C, S recursively new, DIS
_{Th new} , #nil; S ^* , DIST _h ^* , #nil ^* , stat
us run new). 5. This procedure returns from MATCH.

In procedure 6, in SOLUTION, two more parameters are used as compared with the case of the first embodiment.
Use #nil and #nil ^* . #nil is ni among the model primitives included in the partial mapping h that has been obtained so far.
Indicates the number of items corresponding to l. On the other hand, # nil ^* represents the number of model primitives included in the optimal solution h ^* obtained so far and corresponding to nil. These two parameters are used to determine the optimality of the solution.

In steps 2 and 6 of procedure 6, both #nil and # nil ^* are set to zero, and these parameters are included, and the recursive procedure MAT in step 8 is included.
CH is called.

The procedure 7 represents the procedure MATCH. In the step 1, the intermediate solution S currently being searched is found.
Is inferior to the optimal solutions found so far, it means that the procedure returns. On the other hand, in step 2, if it is found that there is one solution in the middle of the search currently being performed and it is better than the optimal solution found so far, the optimal solution S ^* is updated with the solution found this time. Represents a step. Step 3
Is the same as in the first embodiment.

In step 4, the intermediate solution is further searched, but for the model primitive and the image primitive to be combined next, if the image primitive q _j is nil, (p _i , nil) is added to S. At the same time, a counter #nil that counts the number of nil Let us say that the value of new is # nil + 1 and the number of nil has increased by one. On the other hand, if q
_{If j} is not nil, the search for the optimum solution proceeds while performing the same processing as in the first embodiment. afterwards,
In step 4 (3), MATCH is called recursively. And in step 4 (4), if step 4
Only if a better solution is found in (3), S ^* ,
Update DIST _h ^* and #nil ^* .

As described above, by providing the special counters #nil and # nil ^* for nil, the model primitive and the image that optimize the maximum evaluation function of the corresponding model primitive can be obtained. A combination of primitives can be sought. By adopting such a method, it becomes possible to obtain a model primitive corresponding to a fragment divided in an image, and a more stable image recognition device can be provided.

Further, although the fifth embodiment has been described based on the optimum solution searching method of the first embodiment, the processing can be performed by combining the speed-up means as in the second and third embodiments. Is also clear.

The sixth embodiment employing the essential model primitive will be described below. The method for searching the image primitive corresponding to the model primitive has been considered from the first embodiment to the fifth embodiment, but depending on the object model given in advance, the case where the model primitives forming the model are weighted, etc. There is.

For example, in FIG. 12, model primitives 901 (p ₁ ), 902 (p ₂ ), 903 (p
₃ ), 904 (p ₄ ), 905, 906, 907, 90
Consider a case where there are a total of 12 model primitives of 8,909,910,911,912. At this time, in order for this model to exist, at least model primitives 901 (p ₁ ), 902 (p ₂ ), and 903 (p
₃ ) and 904 (p ₄ ) are indispensable. For example, considering the recognition of women's clothing, the existence of a skirt is one of the evidences. Consider a case in which the presence of some model primitives is essential or important, as in this case. In order to solve such a problem, it can be realized by improving the embodiment described so far.

As in the above example, the skirt is constructed 9
01 (p ₁ ), 902 (p ₂ ), 903 (p ₃ ), 90
Consider that a model primitive such as 4 (p ₄ ) is an essential component for the target model. At this time, such a model primitive will be called an essential primitive. When finding the best correspondence between model primitives and image primitives, if no correspondence is found for any of these mandatory primitives (when mapped to nil)
Handles this case by adding the condition that the solution cannot be the optimal solution. Such indispensable primitives are binary logically generated as in equations (11), (16), and (20), the scale between attributes and the scale between relations are considered, or as in the second embodiment. This is useful when thresholding relational measures.

The procedures 8 and 9 described below are the procedures SOL when the essential model primitives are taken into consideration.
It is a representation of UNIT and MATCH. Procedure 8 Procedure SOLUTION () 1. Let the initial value of the solution S be the empty set.

S: = φ; 2. DIST _{h is set} to 0. DIST _h : = 0; Let P be the set of model primitives. Align the model primitives so that the set of essential model primitives is P ₁ and the other model primitives are P ₂ ,
Create P from P ₁ and P ₂ .

P ₁ : = {p ₁ , p ₂ , p ₃ , ..., P _m }; P ₂ : = {p _{m + 1} , p _{m + 2} , ..., P _n }; P: = {p ₁ , P ₂ , p ₃ , ..., P _n }; status value is not Let's say yet.

Status: = not yet ； 5. Let the optimal solution S ^* be the empty set. S ^* : = f; 6. The mapping scale DIST _h ^* of the optimum solution is set to 0.

DIST _h ^* : = 0; 7. Let C be a set of C ₁ , C ₂ , C ₃ , .... C = {C ₁ , C ₂ , C ₃ , ...}; Here, it is assumed that each C _i is arranged in the order of smaller _dij . 8. Procedure MATCH (P, C, S, DIST _h , S
^* , DIST _h ^* , status). 9. Report the optimal solutions S ^* , DIST _h ^* . 10. End Procedure 9 Procedure MATCH (P, C, S, DI
ST _h , S ^* , DIST _h ^* , status) 1. If S ^* is not an empty set and DIST _h > DIS
If T _h ^* , return from this procedure MATCH. 2. If P is an empty set, (1) If S ^* is an empty set, then a new optimal solution has been found, and the following (a), (b), (c),
The process of (d) is performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH.

(2) If S ^* is not an empty set, then DIST _h
<DIST _h ^* , I found a better solution, so
The following processes (a), (b), (c) and (d) are performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST _h for DIST _h ^* .

DIST _h ^* : = DIST _h ; (c) Substitute found for status. (D) Return from this procedure MATCH. 3. Select the first element p _i of P and remove p _i from P.

P: = P- {p _i }; 4. Unless C _i is an empty set, the following (1), (2),
(3), (4), and (5) are repeated.

[0181]: (1) the beginning of the elements of C _i and q _j, C _i
Remove q _j from. C _i : = C _i − {(q _j ,
d _ij )}; If i ≦ m, (a) If q _j is nil, return from this procedure.

(B) If q _j is not nil, r _ij <th
If reshold, return to the beginning of step 4. (2) The sum of S and (p _i , q _j ) is S Substitute in new.

S _{new: = S {(p i} , q j)} (3) dr ij = a d _ij + r _ij is calculated, by adding the dr _ij to DIST _h DIST _{h Let's say new} .

DIST _{h new} = DIST _h + d _ij + r _ij ; status new to not substitute yet. (4) MATCH (P, C, S recursively new, DIS
_{Th new} ; S ^* , DIST _h ^* , status run new). 5. This procedure returns from MATCH.

In procedure 8, procedure SOLUTIO
The difference in N from the second embodiment lies in step 3. That is, after the set P ₂ other models primitives and set P ₁ of the required model primitives are created, as required model primitives are placed in front, a set of the set P of model primitives are generated. This is because the essential model primitives {p ₁ , p ₂ , ..., P _m } are procedures M.
This is to be processed first in ATCH.

The procedure 9 shows the procedure MATCH. In this procedure, the threshold value processing for r _ij described in the second embodiment is performed so that high-speed selection can be performed when an image primitive corresponds to an essential model primitive.

The steps 1 to 3 of the procedure MATCH are
It is similar to the procedure MATCH in the second embodiment. In step 4 (1), first, the top element of the set C _i storing the image primitive candidates is selected and the element is removed from C _i . Then, if the model primitive p _i currently under consideration is an essential model primitive, special processing is performed. That is, if there is no corresponding image primitive (if nil corresponds)
Since the required model primitive does not exist in the image, the procedure immediately returns from this procedure. On the other hand, the model primitives p ₁ and p _1
2 , ..., p _i-1 and the corresponding image primitive (q
₍₁₎ , q ₍₂₎ , ... Q _(i-1) ) and a pair (p _i , p _i , _i ) of the model primitive p _i and the image primitive d _j selected this time.
Based on q _j), terms r _ij involved in related measures (p _i, q _j) is calculated. Threshold value thresh with this r _ij
If it is larger than old, this (p _i , q _j ) is the set (p ₁ , q) of the model primitive and the image primitive so far.
₍₁₎ ), ..., (p _i-1 , q _(i-1) ), it is understood that q _j cannot correspond to the model primitive p _i after all. Therefore, in this case, it is possible to return to the beginning of step 4 and correspond to another image primitive.

By adopting the method as shown in Procedure 9, it is first evaluated whether or not there is an image primitive corresponding to the essential model primitive, so that the object corresponding to the model in the image is evaluated. You can immediately check if there is something. Therefore, the search for the optimum solution can be speeded up.

The seventh embodiment for realizing the grouping of model primitives will be described below. Objects such as clothes are made up of several parts such as sleeves, collars, and body parts, and these parts themselves are important elements that make up a model of clothes. On the other hand, in the first to sixth embodiments, since the curve component and the straight line component are considered as the model primitive, it is not possible to consider the recognition of the object while handling such parts.
In this embodiment, a method for recognizing an object while taking into consideration the parts constituting the object is proposed within the basic framework of this embodiment.

In the first to sixth embodiments, the curved line and straight line components are handled as model primitives, but it can be considered that such model primitives constitute a part as a group. On the other hand, such parts can be considered as the above-mentioned parts as a group of model primitives forming the object.

FIG. 13 groups the model primitives of the clothes displayed in FIG. 12 into three groups 9.
The case of division into 21, 922 and 923 is shown. In the case of this example, it is composed of a group 1 which is a part of the upper clothes, a group 2 which is a part of the lower clothes and a group 3 which is a part of the left and right sleeves, and by the existence of all three groups,
This clothing model holds.

The seventh embodiment will explain how to search for the optimum correspondence with the pixel primitives that determine the existence of these groups of clothes. In the seventh embodiment,
We will describe how to find the optimal correspondence with the image primitives that determine the existence of these groups of clothes.

Basically, the model primitives are hierarchized into groups, and the calculation of the evaluation function DIST _h is simplified by considering the attribute scale / relationship scale within a group and the attribute scale / relationship scale between groups. The goal is to speed up the search for the optimal solution.

As shown in FIG. 13, a group of model primitives constitutes one model feature, and the group itself may be considered as "upper model primitive". The upper model primitives that consist of this group are named group model primitives. At this time, the attributes of the group model primitives themselves and the relationships between the group model primitives can be considered.

For example, regarding the attributes of the three group model primitives as shown in FIG. 13, consider various geometric features such as the barycentric coordinates (x, y) of the group model primitive, its shape, and the perimeter. You can On the other hand, regarding the relationship between the group model primitives, the distance and direction between the centers of gravity can be considered.

Similarly, regarding the image primitives, when a group image primitive which may possibly correspond to the group model primitive is generated, the attribute and relationship of the group image primitive can be considered. What can be seen from the above is that the equivalent of the evaluation scale between the model primitive and the image primitive shown in Expression (24) can be extended to the image primitive corresponding to the grouped model primitive. ing.

To illustrate the above method, assume that there are now N groups and each group _t consists of N _t model primitives. Now, the model primitives of the group t will be written as p ₁ (t), p ₂ (t), ..., P _Nt (t). Furthermore, since the mapping h can be expressed in the form of a partial mapping within each group t, the partial mapping within the group h will be written as h (t). That is, h (t): p _i → q _j (t) for i = 1,2, ..., N _t , t = 1,2, ..., N (38).

First, consider the attribute scale and the relation scale within the group t. DIST shown in equation (18)
_Since it is equivalent to expressing _h by considering only the image primitive corresponding to the model primitive in the group t, the mapping scale in the group can be considered as follows.

[0199]

[Equation 12]

The above calculation can be performed in the same manner as in the first embodiment. On the other hand, the partial mapping h (t) for the group t defines image primitives as a group. If the group model primitive and the group image primitive defined by the partial mapping h (t) are written as p ^G (t) and q ^G (t), the group model primitive mapping {p ^G (t), t =
, 1, ..., N} and the model image primitive mapping {q ^G (t), t = 1, 2, ..., N} can define the relationships R ^G _m and S ^G _k . Then, in the same form as the equation (21), the comprehensive relation scale DIST ^G
_h (R ^G , S ^G _h ) can be derived. Therefore,
Mapping measure D for h taking grouping into account
IST ^G _h

[0201]

[Equation 13] Can be written. REL ^G _h here is group 1
Is a relational scale of the groups to which the relations in the difference set obtained by subtracting the set of relations established from the group 1 to the group t-1 from the set of relations established from the group t to the group t contribute. This derivation is equivalent to the method used in the individual relation scale of the first embodiment. Further, in this formula (40), the reason for the attribute of the group is not taken into consideration because the attribute measure is implicitly included in the attribute measure and the relation measure in the group.

The following procedure 1 shows the method for obtaining the optimum correspondence between the model primitive and the image primitive using the mapping scale shown in the equation (40) as the evaluation criterion.
0 and procedure 11.

Procedure 10 Procedure SOLUTIO
N () 1. Let the initial value of the solution S be the empty set. S: = φ; 2. DIST ^G _{h is set} to 0.

DIST ^G _h : = 0; Group the model primitives and create group model primitives p ^G (1), p ^G (2), ..., P ^G (N). Furthermore, P = {p ^G (1), p ^G (2), ..., P ^G
(N)}. 4. Each group model primitive p ^G (t) (t = 1,
2, ..., N), a candidate for a group image primitive that can correspond thereto is obtained. Then, it is obtained in the form of C (t) = {C ₁ (t), C ₂ (t), C ₃ (t), ...}. Here, C _i (t) is represented by a set of the i-th group image primitive candidate for the group model primitive P (t) and DIST _h (t) of Expression (39). Furthermore, C _i (t) is DIST _h (t)
The values are sorted in ascending order. And C =
Let {C (1), C (2), ..., C (N)}. 5. status value is not Let's say yet.

Status: = not yet ； 6. Let the optimal solution S ^* be the empty set. S ^* : = φ; 7. The mapping scale DIST _h ^* of the optimum solution is set to 0.

DIST ^G _h ^* : = 0; Procedure MATCH (P, C, S, DIST ^G _h , S
^* , DIST ^G _h ^* , status). 9. The optimal solutions S ^* and DIST ^G _h ^* are reported. 10. End Procedure 11 Procedure MATCH (P, C, S, D
IST ^G _h , S ^* , DIST ^G _h ^* , status) 1. If S ^* is not an empty set and DIST ^G _h > DI
If ST ^G _h ^* , return from this procedure MATCH. 2. If P is an empty set, (1) If S ^* is an empty set, then a new optimal solution has been found, and the following (a), (b), (c),
The process of (d) is performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST ^G _h ^* for DIST ^G _h ^* .

DIST ^G _h ^* : = DIST ^G _h ; (c) Substitute found for status. (D) Return from this procedure MATCH.

(2) If S ^* is not an empty set, then DIST ^G
_{If h} <DIST ^G _h ^* , a better solution was found, so the following processes (a), (b), (c), and (d) are performed.

(A) Substitute S for S ^* . S ^* : = S; (b) Substitute DIST ^G _h ^* for DIST ^G _h ^* .

DIST _h ^* : = DIST ^G _h ; (c) Substitute found for status. (D) Return from this procedure MATCH. 3. Select the element p ^G (t) at the beginning of P and select from P to p ^G (t)
Get rid of.

3. P: = P- {p ^G (t)}; Unless C (t) is an empty set, the following (1),
(2), (3), (4) and (5) are repeated.

(1) Let q ^G _{j be} the first element of C (t),
Remove q ^G _j from C (t). C (t): = C (t)-{(q ^G _j , DIST ^G _h (t)
)}; If q ^G _j is nil, this procedure returns.

(2) The value obtained by adding (p ^G (t), q ^G _j ) to S is S Substitute in new. S new: = S + {(p ^G (t), q ^G _j )}; (3) DIST ^G _h (t) + REL ^G _h (t) is calculated, and D
Add to IST ^G _h and DIST ^G _{h Let's say new} .

DIST ^G _{h new} ： = DIST ^G _h ＋ DIST ^G _h (t) ＋ REL ^G _h (t) ； status new to not substitute yet.

(4) MATCH (P, C, S recursively
new, DIST ^G _{h new} ； S ^* , DIST ^G _h ^* , st
atus run new). 5. This procedure returns from MATCH.

In procedure 11, the solution is initialized in steps 1 and 2, and an empty set is set in the solution S, and DIST ^G _h is also initialized to 0. In step 3, the model primitives are grouped,
Group model primitives p ^G (1), p ^G (2), ...
p ^G (N) is generated. Furthermore, P = {p ^G (1), p ^G
(2), ..., P ^G (N)}.

In step 4, for each group model primitive p ^G (t) (t = 1, 2, ..., N),
A group image primitive candidate that can deal with it is obtained.

Then, it is obtained in the form of C (t) = {C ₁ (t), C ₂ (t), C ₃ (t), ...}. Here, C _i (t) is represented by a set of the i-th group image primitive candidate for the group model primitive P (t) and DIST _h (t) in Expression (39). Furthermore, C _i (t) is DIST _h (t)
The values are sorted in ascending order. And C =
Let {C (1), C (2), ..., C (N)}. Such C
To obtain (i), the procedure MATC shown in procedure 2
A method similar to H may be adopted. However, instead of simply storing only one optimal solution S ^* and DIST _h ^* ,
The value of DIST _h or stored as all those below a certain threshold list on, by or storing the solution S number was determined with a higher if all list to determine from the value of DIST _h , C (t), a candidate list of group image primitives can be created.

From step 5 to step 7, the solution is initialized to execute the procedure MATCH in step 8. In step 8, the procedure MATCH is executed in order to find the optimal solution considering the groups. Then, in step 9, the optimum solution is reported, and all processing is terminated.

The procedure 11 represents the procedure MATCH when the group is considered. In step 1, the measure DIST ^G of the optimal solution S ^{* found} so far
_h ^* and the partial sum of the scale of the solution currently being searched are compared. If the solution currently being searched for is inferior to S ^* , the search for that solution is stopped, and backtracking is performed so as to search for another solution.

In step 2, if a better solution than the current time is found, the current solution is registered as the optimum solution. In step 3, the head element p ^G (t) of the group model primitive is selected, and a search for a solution for this group model primitive is performed in the subsequent steps.

In step 4, a process for checking the possibility of an optimum solution is performed for each group image primitive candidate that may correspond to the group model primitive. First, step (1) Then
The most promising candidate q ^G. _j at the moment is selected. If it is nil, abort the search for this solution and backtrack to another solution. Otherwise, the process proceeds to step (2), and DIST ^G _h represented by equation (40)
Is calculated, and the procedure MATCH is recursively executed in step (4).

As described above, the model primitives are grouped in the seventh embodiment. When the object becomes complicated and the number of model primitives increases, it is difficult to calculate the matching evaluation scale shown in Expression (24), but by grouping, the evaluation scale within the group and the inter-group evaluation scale can be calculated. Since it becomes possible to calculate the evaluation scale independently, there is an effect that the search for the optimum solution can be performed at high speed.

Next, an eighth embodiment for carrying out the processing when a large number of models exist will be described. In the embodiment described so far, the method of evaluating the image primitive corresponding to the model of one object and searching for the optimum image primitive and fragment has been described. In many cases, one or more models are stored in the model storage means 13, and it is possible to evaluate which model corresponds to an object in a given image and where in the image the image is captured. To do is also an important issue. The eighth embodiment shows a method for evaluating which model the object photographed in the image best fits when many such models are stored in the model storage unit 13. .

To clarify the problem, the model storage means 1
It is assumed that 3 models store M models. In general, if multiple models are stored, it is empirically possible to define a priori probabilities of which models may be present in the image. This prior probability corresponding to the model m will be written as prob (m). Of course, if the prior probability is not known, it is possible to assign equal probabilities such as prob (m) = 1 / M.

As described in the first to seventh embodiments, after the fragment is extracted from the image and the image primitive is generated from the fragment, it corresponds to one model stored in the model storage means 13. It turns out that we can search for the optimal image primitive to do.
The optimum correspondence S ^* (m) of the image primitives with respect to the model primitives that form the model m currently stored in the model storage means 13 and its matching scale are DIST ^*.
_{Let h} (m) be all models 1, 2, ..., M, ...,
When M is taken into consideration, the model m that optimally corresponds to an image primitive is to find m and S ^* (m) that optimize DIST _h ^* (m).

The following procedure 12 is the procedure BEST showing this method. It shows a MODEL. Procedure 12 Procedure BEST MODEL () 1. The models stored in the model storage unit 13 are arranged in descending order of prior prob (m). The aligned models are sequentially model 1, model 2, ..., Model m ,.
Let it be model M. 2. A fragment is extracted from an image, an image primitive is generated from the fragment, and a structure description about the image primitive is performed. 3. Set the scale corresponding to the optimal model to DIST _h ^* = inf. 4. Let m = 1. 5. Find the optimal correspondence between the model primitives that make up the model m and the image primitives generated in step 2. The optimal solution is S ^* (m), and the optimal measure is DIST
_{Let h} ^* (m). Regarding this optimal search, the matching measure DIST ^* of the optimum solution up to that point is used, and a solution that generates a matching measure having a value lower than DIST _h ^* during the search interrupts the search midway. When a solution superior to the previous optimal solution can be found, m ^* : = m, S ^* : = S ^* (m), DIST _h ^* : = DI
The optimal solution is updated by ST _h ^* (m). 6. If m = M, go to step 7. Otherwise, set m: = m + 1 and return to step 5. 7. Report the model m ^* corresponding to the optimal solution and the image primitives that make up the optimal solution. 8. End In step 1, the models stored in the model storage means 13 are arranged in descending order of prior prob (m). The aligned models are referred to as model 1, model 2, ..., Model m ,. By this processing, a model having a higher possibility of being processed is processed earlier in time.

In step 2, a fragment is extracted from the image, an image primitive is generated from the fragment, and a structural description regarding the image primitive is performed. This method is similar to the method described in the first embodiment.

In step 3, the scale corresponding to the optimum model is set to DIST _h ^* = inf, and the optimum solution is initialized. In step 4, m representing the index of the model
Is set to 1. This will ensure that the most promising models are processed first.

At step 5, the optimum correspondence between the model primitives forming the model m and the image primitives generated at step 2 is obtained. The optimal solution is S ^* (m)
, Its optimal scale is DIST _h ^* (m). Regarding this optimal search, the matching measure DI of the optimal solutions up to that point
A solution that uses ST _h ^* and generates a matching measure having a value lower than DIST _h ^* during the search is interrupted during the search. When a solution superior to the previous optimal solution can be found, m ^* : = m, S ^* : = S ^* (m), DIST _h ^* : = DI
The optimal solution is updated by ST _h ^* (m). In this way, in step 5, when searching for the correspondence with the optimum image primitive in each model m, the information of the optimum solution obtained up to that point is effectively used, and unnecessary search processing is omitted.

In step 6, m is compared with the total number M of models in order to judge whether the processing has been completed for all models. If m = M, step 7
Proceed to. Otherwise, set m: = m + 1 and return to step 5.

In step 7, the model m ^* corresponding to the optimum solution and the image primitives forming the optimum solution are reported, and in step 8, the processing ends. According to the eighth embodiment described above, even if a plurality of models are stored in the model storage unit 13, it is possible to search for a model that is most suitable for the object captured in the image. Further, in the middle of the process, the time required for the search can be reduced by effectively using the value of DIST _h ^* of the optimum solution obtained so far.

A ninth embodiment will be described below in which processing is performed after grouping a large number of models. In the method described in the eighth embodiment, the processing method when the model storage unit 13 includes a plurality of models has been described. When an enormous number of models are stored in the model storage unit 13, the models may be similar to each other, and it is desired to perform higher speed processing. The method described in the ninth embodiment is effective when the model storage means 13 has a huge number of models in this way.

Basically, it is realized by grouping model primitives as described in the seventh embodiment. FIG. 14 illustrates a method of grouping model primitives that make up a model when there are many models. In the figure, two models 1301 and 1302 are considered to simplify the description. These models 1301 and 1302 are model primitive 1
001, 1002, ..., 1013, 1101, 110
2, ..., 1107. Of these, the model primitives 1001, 1002, 1003, 1104 form the group 1, and the model primitives 1003, 10
04,1005,1006,1007 is group 2
Model primitives 1101, 1102, 1103, 1
104, 1105, 1106, 1107, 1003 group 3, and model primitives 1008, 10
09, 1010, 1011, 1012, 1013, 13
01, 1302 can be defined as a common part (group).

Therefore, by organizing the set of model primitives stored in the model storing means 13 in advance, the parts constituting various models are classified, and they are expressed as a group of model primitive primitives. .

The method of searching for the optimum model and the optimum image primitive when using the above grouping is the procedure BEST shown in the following procedure 13. MODE
It is shown in L.

Procedure 13 Procedure BEST MO
DEL () 1. The models stored in the model storage unit 13 are arranged in a cross manner to generate group model primitives,
Find these structural descriptions. 2. For each model m, a flag status (m) indicating the status of the model m is prepared, and its initial value is set to 0. 3. Extract a fragment from the image and generate an image primitive from the fragment. 4. List the candidate group image primitives that can correspond to each group model primitive. 5. Mark group model primitives that have no candidate group image primitives. A model m whose components are the marked group model primitives.
The status flag status (m) of 1 is set to 1. 6. Models whose status flag status (m) is 0 are sorted in descending order of prior prob (m). The models thus obtained are referred to as model 1, model 2, ..., Model M again. 7. The parameter m representing the model index is set to 1.

The matching scale that takes into account the optimal model group is DIST ^G _h ^*, and its initial value is inf
And Let m ^{* be} the index that represents the optimal model, and S ^* be the optimal correspondence of the model image primitive for the model m ^* . 8. For model m aligned by step 6,
The optimum correspondence between the group model primitives constituting it and the group model primitives is obtained by using the evaluation scale of the equation (40). When obtaining the optimum correspondence by this evaluation scale, DIST ^G _h ^* corresponding to the optimum solution found so far is used as a threshold value. That is, if the value of the middle DIST ^G _h the solution search is DIST ^G
When it becomes larger than the value of _h ^* , the search for the intermediate solution is stopped and backtracking to another solution. If DI
When a solution superior to ST ^G _h ^* is found, m ^* : = m, S ^* : = S, DIST ^G _h ^* : = DIST
^The optimal solution is updated as ^G _h ；. 9. Compare m with the total number M of models.

If m = M, the process proceeds to step 10. Otherwise, set m: = m + 1 and return to step 8. 10. Report the model m ^* corresponding to the optimal solution and the group image primitives that make up the optimal solution. 11. Conclusion That is, in step 1, the group model primitives are g1, g2, ..., Gt ,.

At step 2, the set of group model primitives constituting each model m is set to G 1, G 2 ,.
Let m and GM. For each model m, prepare a flag status (m) indicating the status of the model m, and set its initial value to 0.
And

In step 3, a fragment is extracted from the image and an image primitive is generated from the fragment. In step 4, the group image primitive candidates that can correspond to each group model primitive gt (t = 1, 2, ..., N) are listed.

In step 5, a group model primitive having no group image primitive candidate is marked. Set the status flag status (m) of model m whose constituent elements are the marked group model primitives to 1
And Since it can be seen that the model whose flag is 1 cannot correspond to the image primitive, simplification of the processing can be achieved.

In step 6, the models whose status flag status (m) is 0 are sorted in descending order of prior prob (m). The models thus obtained are referred to as model 1, model 2, ..., Model M again.

At step 7, the parameter m representing the model index is set to 1. Let DIST ^G _h ^* be the matching measure that takes into account the optimal group of models and its initial value be inf. Further, let m ^{* be} the index representing the optimum model, and S ^* be the optimum correspondence of the model image primitive with respect to the model m ^* .

In step 8, for the models m arranged in step 6, the optimum correspondence between the group model primitives forming the model m and the group model primitives is obtained by using the evaluation scale of the equation (40). When obtaining the optimum correspondence by this evaluation scale, DIST ^G _h ^* corresponding to the optimum solution found so far is used as a threshold value. That is, if DI
When the value of ST ^G _h becomes larger than the value of DIST ^G _h ^* , the search for the midway solution is stopped and backtracking to another solution is performed. If a solution superior to DIST ^G _h ^* is found, then m ^* : = m, S ^* : = S, DIST ^G _h ^* : = DIST
^The optimal solution is updated as ^G _h ；. Thus, in step 8,
When searching for the correspondence with the optimum group image primitive in each model m, the information of the optimum solution obtained up to that point is effectively used, and unnecessary search processing is omitted.

In step 9, m is compared with the total number M of models in order to determine whether the processing has been completed for all the models. If m = M, step 1
Go to 0. Otherwise, set m: = m + 1 and return to step 8.

At step 10, the model m ^* corresponding to the optimum solution and the group image primitives forming the optimum solution are reported, and at step 11, the processing is ended. Therefore, this embodiment has the following effects. Even when a huge number of models are stored in the model storage unit 13, or even when the models have similar parts, the optimum solution is searched at high speed and the optimum model is stored in the model storage unit 13. You can choose. It should be noted that each structure of this embodiment can of course be modified or changed in various ways.

In this embodiment, the example of the clothes image has been mainly described, but it is obvious that the method of this embodiment is applied to other images. In other words, with respect to an object for which a relatively abstract level model is obtained, even if the object is subject to some deformation or the contour components are extracted in a discontinuous form, they are allowed. can do. For example, the present invention can be applied to analysis of human facial expressions and extraction of human images.

[0250]

As described above in detail, according to the present invention, a primitive is generated from a contour component extracted from an image to search an optimum correspondence with a model stored in advance. It is possible to stably and accurately recognize the object even from an image taken when the object to be deformed is photographed under a noisy photographing condition.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of the present invention.

FIG. 2 is a diagram showing a clothing image of the first embodiment.

FIG. 3 is a diagram for explaining a structure description of a model primitive.

FIG. 4 is a diagram showing a model of clothes.

FIG. 5 is a diagram showing model primitives forming a double suit.

FIG. 6 is a diagram showing an example of an actual image of a double suit and a curve component (fragment) extracted from this image.

FIG. 7 is a diagram showing a flowchart of the present invention.

FIG. 8 is a diagram showing a relationship between fragments.

FIG. 9 is a diagram illustrating generation of a primitive.

FIG. 10 is a diagram showing a model primitive, a fragment, and an image primitive.

FIG. 11 is a diagram illustrating a method of searching for an optimum solution.

FIG. 12 is a diagram showing a model for explaining a sixth embodiment.

FIG. 13 is a diagram showing grouped model primitives for explaining a seventh embodiment.

FIG. 14 is a diagram illustrating grouping of model primitives according to the ninth embodiment.

[Explanation of Codes] 11 ... Image photographing means, 12 ... Image storage means, 13 ... Model storage means, 14 ... Image processing means, 15 ... Video signal, 1
6 ... Image, 17 ... Described model.

Claims (1)

[Claims] 1. A model storing means for storing model characteristics of an object as a set of contour components, an image photographing means for photographing an object and transmitting a video signal, and converting the video signal into a digital image. An image capturing / storing means for storing a digital image and a straight line component or a curved line component corresponding to a contour are extracted from the stored image, and the straight line component or the curved line component is considered in consideration of the connection relation of the straight line component or the curved line component. Image processing means for grouping the curve components and converting them into a form of image features capable of corresponding to the model features, and performing an optimum matching between the model features and the image features. Image recognition device.