JPH08249353A - Method and device for image retrieval - Google Patents

Abstract

PURPOSE: To provide an image retrieving method and device which enable efficient retrieving operation even in the case in which an example image just matching with features of a desired image to be retrieved can not be obtained, or when image features are displayed with an example image representing only a basic concept and then corresponding images are sequentially retrieved by expanding a design concept while a slight variation is added to the example image. CONSTITUTION: An example image menu including >=2 example images G1 and G6 which are different in feature from each other is displayed, and 1 or >=2 example images which resemble the desired image and/or example images which do not resemble it are specified among example images included in the displayed example image menu, and retrieval conditions for retrieving the desired image are generated on the basis of respective feature quantities of the said specified example images and under the generated retrieval conditions, 1 or >=2 images which resemble the desire image are selected out of the images stored in an image storage means, and then displayed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image search device for an exemplary image input method condition for presenting an exemplary image from a large number of images prepared in advance and searching for it, and is particularly suitable as an exemplary image. The present invention relates to an image search device capable of efficiently searching for a desired image even when it is difficult to obtain a desired item.

[0002]

2. Description of the Related Art For example, when designing clothes, curtains, wallpaper, embroidery, etc., a new design is created by selecting a design concept close to the desired design concept from existing design samples and adding appropriate modifications to it. Creating is often done among designers. In order to improve the efficiency of such design work, it is necessary to find a design close to the desired design concept from the design sample as quickly as possible. However, as the number of symbol samples becomes huge, there is a limit to relying on such a symbol sample searching operation manually.

Therefore, conventionally, in order to mechanize such a pattern sample search operation, various image search apparatuses using computer technology have been proposed. In such an image retrieval apparatus, each of the above-mentioned pattern samples is converted into image data through an image scanner or the like, and then the CD-RO.
It is stored in a mass storage medium such as M. At the same time, various image processing such as density value conversion, noise elimination, blur restoration, contour detection enhancement, and connected portion extraction is applied to each image data, and then based on the image processing result. Then, various feature extraction operations are performed, and various image feature amounts such as the number of boundary line pixels, the number of holes, the degree of curve, the number of colors, the color distribution, the contrast, the boundary line pixel distribution, and the color distribution (hereinafter, this will be described later). “Physical feature amount” for comparison with the “sensitive feature amount”). Now, suppose k images are y ₁ , y
₂ , y ₃ , ... Y _k , and the physical characteristics of the n types are x ₁ ,
If x ₂ , x ₃ , ... X _n are used, each image is an n-dimensional vector (x ₁₁ , x ₂₁ , x ₃₁ , ... x _n1 ), (x ₁₂ , x ₂₂ ,
x ₃₂ , ... X _n2 ), ... (X _1k , x _2k , x _3k , ... X _nk ). Therefore, an image y _p that matches the desired design concept is represented as a vector (x _1p , x _2p , x _3p , ... X _np ) using the above-mentioned physical quantity, and this and each of the images y ₁ and y described above are represented. _By finding the distance from the vector corresponding to ₂ , y ₃ , ..., Y _k and selecting the one having the shortest distance among them, it is possible to mechanically retrieve the image most similar to the desired image. Is.

Techniques for improving this type of image retrieval device
As a document, JP-A-5-6437 (the title of the invention is
Title: "Image feature extraction device, image feature matching device and image
Search device ”) or Information CG / CAD Study Group 199
August 4th, Vol. 94, No. 72, p. 43-48
(Paper title: "Kansei characteristics and image characteristics of design images")
It has been known. In these documents, image search
It is possible to use human emotional expressions
It That is, the objects used in the image search device described above.
The physical feature amount is the result of image processing of the given image data.
By applying a predetermined feature extraction operation to the fruits,
While having the advantage of being mechanically and uniquely extracted, its table
Currently, for example, the number of border pixels, the number of holes, curvature, color
Number, color distribution, contrast, border pixel distribution, color distribution, etc.
To express any design concept like
It lacks appropriateness. So this improved image search
The equipment is warm / cold, masculine / feminine, safe
Constant / unstable, symmetric / asymmetric, simplicity / complexity, etc.
Humans (designers) make impressions of arbitrary images like
Focusing on the emotional characteristics used to identify,
Statistical correlation between the physical features and the physical features described above.
Sensitivity features using this correlation
It is possible to search images by quantity. If you explain it conceptually,
The emotional feature amount of an arbitrary image is set to w₁, W₂, W₃,…
w_m, Its physical feature x₁, X₂, X₃, ... x_nWhen
By definition, each affective feature quantity w₁, W₂, W₃, ... w_m
Are n-dimensional vectors (a₁₁・ X₁, A₁₂・ X
₂, A₁₃・ X₃, ... a_1n・ X_n), (A _{twenty one}・ X₁, A_{twenty two}
・ X₂, A_{twenty three}・ X₃, ... a_2n・ X_n), (A₃₁・ X₁,
a₃₂・ X ₂, A₃₃・ X₃, ... a_3n・ X_n), ... (a_m1・
x₁, A_m2・ X₂, A_m3・ X₃, ... a_mn・ X_n) And represented
Be done. Therefore, it is possible to convert physical features into emotional features.
Weighting value group (a₁₁, A₁₂, A₁₃, ... a_1n),
(A_{twenty one}, A_{twenty two}, A_{twenty three}, ... a_2n), (A₃₁, A₃₂, A₃₃,
... a_3n), ... (a_m1, A_m2, A_m3, ... a_mn) Designer
In advance using statistical methods through questionnaire surveys
If you ask for it, from any image data, its affective features
Can be mechanically extracted. Therefore, the image search
In the process, the sensitivity of the input image and the sense of the registered image
By calculating the distance from the sexual feature, the physical feature
Compared with the case of using,
Image search is performed.

[0005]

By the way, in order to retrieve a desired image in this type of image retrieval apparatus, it is necessary to present the characteristics of the image to be retrieved to the apparatus side by some method. If this image feature presentation is performed by numerically inputting all the physical features or the affective features such as linearity [0.9], sharpness / softness [0.8], etc., it will be complicated. Absent. Therefore, conventionally, in this type of image retrieval apparatus, an exemplary image in which the characteristics of an image desired to be retrieved is exemplarily expressed using a diagram is often used. That is, the input exemplary image is mechanically converted into a physical feature amount and / or a sensible feature amount through the image processing process described above, and the feature amount and the feature amount of each registered image are combined. The distances are compared and one or more images similar to the input image are finally retrieved.

[0006] However, in the conventional image search device of the example image input method, such complicated image input operation does not require complicated numerical value input operation, but on the other hand, it exactly matches the feature of the image desired to be searched. If an exemplary image cannot be obtained, it is difficult to find an image having desired characteristics, which is rather inconvenient.

[0007] Initially, the image features are presented by an exemplary image showing only the basic concept, and then the design concept is inflated while adding slight variations such as color change and pattern change to the exemplary image. In case of sequentially searching for corresponding images, coloring and pattern changes must be actually added to the exemplary image each time,
However, there were some problems such as poor usability.

The present invention has been made in view of the above problems, and an object thereof is to obtain an exemplary image that does not exactly match the characteristics of an image desired to be searched, or to use the basic concept. Even if the image features are presented by an exemplary image that shows only the image, and then the design concept is expanded and the corresponding images are sequentially searched while adding a slight variation to the exemplary image, these search operations are performed. An object of the present invention is to provide an image search device capable of efficiently performing the above.

[0009]

The invention according to claim 1 of the present application includes an image storing step of storing a plurality of images with a plurality of types of characteristic amounts attached to each of the plurality of images, and two or more different features. An exemplary image menu displaying step of displaying an exemplary image menu including exemplary images, and an exemplary image similar to and / or dissimilar to the desired image among the exemplary images included in the displayed exemplary image menu. 1 or 2
The similarity specifying step specified above, a search condition generating step of generating a search condition for searching the desired image based on each feature amount of the example image for which the similarity is specified, and the generated search condition On the basis of the above, 1 or 2 images similar to the desired image are selected from the images stored in the image storage means.
It is characterized by comprising an image extraction step of extracting the above and an image display step of displaying the extracted one or more images.

In the invention according to claim 2 of this application, in the invention according to claim 1, when both the similar exemplary image and the dissimilar exemplary image are designated in the search condition generating step. , On the feature coordinates defined by the plurality of types of feature amounts, a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions is a predetermined distance in the similar designated exemplary image direction. It is characterized in that the search condition is generated based on the assumption that the desired image exists near the extended position.

In the invention according to claim 3 of this application, in the invention according to claim 1, in the search condition generating step, when only a similar exemplary image is designated, the plurality of types of feature amounts Based on the premise that the desired image exists in the vicinity of a position where a straight line connecting the average position of the similar designated exemplary image positions and the origin is extended by a predetermined distance in the similar designated exemplary image direction on the feature coordinates defined by It is characterized in that search conditions are generated.

According to a fourth aspect of this application, in the invention according to the first aspect, the search condition generating step is characterized in that when only dissimilar exemplary images are designated, A search is performed based on the assumption that the desired image exists in the vicinity of a position where a straight line connecting the average position of the dissimilar designated exemplary image positions and the origin is extended by a predetermined distance in the direction of the origin on the characteristic coordinates defined by the quantity. It is characterized by generating a condition.

According to a fifth aspect of this application, in the invention according to the first aspect, among the one or more images extracted and displayed, an image similar to a desired image, and And / or a similarity re-designating step for re-designating one or more dissimilar images.

The invention according to claim 6 of the present application is an example including an image storage means for storing a plurality of images with a plurality of types of characteristic amounts respectively stored therein, and two or more exemplary images having mutually different features. One or more exemplary images similar to and / or not similar to the desired image among the exemplary images included in the displayed exemplary image menu and the exemplary image menu displaying means for displaying the exemplary image menu are designated. Based on possible similarity designating means and each feature amount of the exemplary image designated as similarity,
Search condition generating means for generating a search condition for searching the desired image, and one or more images similar to the desired image from the images stored in the image storage means based on the generated search condition. It is characterized by comprising image extracting means for extracting and image displaying means for displaying the one or more extracted images.

In the invention according to claim 7 of this application, in the invention according to claim 6, when both the similar exemplary image and the dissimilar exemplary image are designated by the search condition generating means. , On the feature coordinates defined by the plurality of types of feature amounts, a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions is a predetermined distance in the similar designated exemplary image direction. It is characterized in that the search condition is generated based on the assumption that the desired image exists near the extended position.

In the invention according to claim 8 of this application, in the invention according to claim 6, the search condition generating means, when only similar exemplary images are designated, Based on the premise that the desired image exists in the vicinity of a position where a straight line connecting the average position of the similar designated exemplary image positions and the origin is extended by a predetermined distance in the similar designated exemplary image direction on the feature coordinates defined by It is characterized in that search conditions are generated.

In the invention according to claim 9 of this application, in the invention according to claim 6, the search condition generating means is characterized in that, when only dissimilar exemplary images are designated, A search is performed based on the assumption that the desired image exists in the vicinity of a position where a straight line connecting the average position of the dissimilar designated exemplary image positions and the origin is extended by a predetermined distance in the direction of the origin on the characteristic coordinates defined by the quantity. It is characterized by generating a condition.

According to a tenth aspect of the present application, in the invention according to the sixth aspect, an image similar to a desired image among the one or more images extracted and displayed, and / Or 1 or 2 dissimilar images
It is characterized in that it comprises a similarity redesignating means for redesignating.

[0019]

According to the invention described in claim 1 of this application, a plurality of images are stored with a plurality of types of feature values attached to each, and an example image menu including two or more example images having different features is provided. Among the exemplary images displayed and included in the displayed exemplary image menu, one or more exemplary images similar to and / or dissimilar to the desired image are designated, and the similarity is designated. A search rule for searching for the desired image is generated based on each feature amount of the image, and one or more images similar to the desired image from the stored images are generated based on the generated search condition. The extracted one or more images are displayed.

According to the invention of claim 2 of this application, when both the similar exemplary image and the dissimilar exemplary image are designated, on the characteristic coordinates defined by the plurality of types of characteristic amounts. Based on the premise that the desired image exists near a position obtained by extending a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions in the similar designated exemplary image direction by a predetermined distance. A search condition is generated in.

According to the invention of claim 3 of this application, when only the similar exemplary image is designated, the similar designated exemplary image position is located on the characteristic coordinates defined by the plurality of types of characteristic amounts. The search condition is generated on the assumption that the desired image exists near the position where the straight line connecting the average position and the origin is extended by a predetermined distance in the similar designated illustration direction.

According to the invention of claim 4 of this application, when only the dissimilar exemplary image is designated, the dissimilar designated exemplary is displayed on the feature coordinates defined by the plurality of types of feature amounts. The search condition is generated on the assumption that the desired image exists near the position where a straight line connecting the average position of the image positions and the origin is extended by a predetermined distance in the direction of the origin.

According to the invention of claim 5 of the present application, among the one or more images extracted and displayed, the image similar to the desired image and / or the image not similar to the desired image is one. Or, two or more are designated again.

According to the invention of claim 6 of this application, a plurality of images are stored with a plurality of types of characteristic amounts attached to each of them, and an exemplary image menu including two or more exemplary images with different features is provided. An exemplary image that is displayed and is similar to the desired image among the exemplary images included in the displayed exemplary image menu,
And / or one or two or more dissimilar exemplary images are designated, and based on each feature amount of the exemplary images designated as similarity or similar,
A search condition for searching the desired image is generated, and one or more images similar to the desired image are extracted from the stored image based on the generated search condition, and the extracted one or Two or more images are displayed.

According to the invention of claim 7 of this application, when both the similar exemplary image and the dissimilar exemplary image are designated, on the characteristic coordinates defined by the plurality of types of characteristic amounts, Based on the assumption that the desired image exists near the position where a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions is extended by a predetermined distance in the similar designated exemplary image direction. Search conditions are generated.

According to the invention of claim 8 of this application, when only the similar exemplary image is designated, the similar designated exemplary image position is located on the characteristic coordinates defined by the plurality of types of characteristic amounts. The search condition is generated on the assumption that the desired image exists near the position where the straight line connecting the average position and the origin is extended by a predetermined distance in the similar designated illustration direction.

According to the invention of claim 9 of this application, when only the dissimilar exemplary image is designated, the dissimilar designated example is displayed on the feature coordinates defined by the plurality of types of feature amounts. The search condition is generated on the assumption that the desired image exists near the position where a straight line connecting the average position of the image positions and the origin is extended by a predetermined distance in the direction of the origin.

According to the invention of claim 10 of this application, among the one or more images extracted and displayed, one image that is similar to the desired image and / or one that is not similar to the desired image Or, two or more are designated again.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. The hardware configuration of the image search apparatus of this embodiment is shown in FIG. As shown in the figure, the hardware of this image search apparatus includes a central processing unit 1, a display unit 2, an operation unit 3, a printer 4, and an image storage unit 5.
And a personal computer system including the image scanner 6.

As is well known, the central processing unit 1 is mainly composed of an MPU, a RAM, a ROM and the like, and the above-mentioned system elements 2 to 2 are connected via a system bus 7.
8 is to be totally controlled.

The display unit 2 is composed of a CRT display, an LCD, etc., and on the display screen of the display unit 2, as shown in detail later with reference to FIGS. 9 and 10, an exemplary image is displayed. Number of first place displayed as menu (9 in this example)
Or a fixed number (nine in this example) of similar images extracted as a search result are displayed in color.

The operation unit 3 is composed of a keyboard, a mouse, etc., and various instruction inputs to this apparatus are performed using this operation unit 3.

The printer 4 is composed of a high precision printer such as a laser color printer and is used for obtaining a hard copy of the searched image.

The image storage unit 5 is composed of a large-capacity storage device such as a write-once type optical disc and a rewritable type optical disc. The image storage unit 5 will be described later in detail with reference to FIGS. 2 and 11. As described above, a large number of images to be searched are stored with their physical features and / or emotional features attached.

The image scanner 6 is for converting an arbitrary pattern sample 8 displayed in color into image data, and this image scanner 6 is an image as will be described later in detail with reference to FIG. It is used for the registration process, or for inputting the pattern sample 8 as an example image.

Those skilled in the art will readily understand that various operating systems, drawing application software and the like are required to realize the above personal computer system.

Next, FIG. 3 is a control flow chart showing the software configuration of the device of the present invention, and the operation of the device of the present invention will be systematically described below with reference to this flow chart and the explanatory views of FIGS. .

First, the image registration process will be described. As shown in FIG. 11, it is assumed that there are now 1000 images or pattern samples to be registered. In such a case, the present invention first secures 50 of them as sample analysis images, and with respect to this sample analysis image,
The emotional feature amount and the physical feature amount are extracted. The extraction of the kansei features is performed by conducting a questionnaire survey on an appropriate number of designers (for example, 100 people) or general consumers. As a result, the sensitivity features of each image forming the sample analysis image are, for example, 0.8 for sharpness / softness, 0.7 for simple decorativeness, 0.6 for dynamic / staticity, and modern / modern. The classic degree is quantified such as 0.3, and the abstract / realistic degree is 0.9. In this example, the value of each feature scale is
It is represented by each numerical value obtained by dividing 0.0 to 1.0 into 10 steps, but this is merely an example, and any unit system can be adopted as necessary. On the other hand, the extraction of physical features is mechanically performed by computer processing. That is, for each of the images constituting the 50 sample analysis images, first the density value conversion, noise removal,
Various image processes such as blur restoration, contour detection enhancement, and extraction of connected parts are performed, and then various feature extraction operations are performed based on the image process results, and finally, for example, the linearity is 0. .9, contrast degree 0.8, complexity degree 0.
Each physical feature amount is quantified, such as 1.

In this way, if the sensible feature amount and the physical feature amount are extracted from each image forming the sample analysis image, then the mapping rule f that defines the correlation between both the feature amounts. Is created by a statistical method. As a result, the mapping rule f is obtained, for example, sharpness / softness = 0.9, linearity-0.2, complexity.

After that, the mapping rule f calculated above is obtained.
By using, the emotional feature amount is mechanically obtained from the 1000 images to be registered, and the physical feature amount and the affective feature amount thus obtained are respectively associated with the corresponding image and stored in the image database. Configured as
Finally, it is stored in the image storage unit 5. An example of the image database created in this way is shown in FIG. In this example, as shown in FIG.
o. Is attached, and as the feature amount, “soft / hard degree”, “static / dynamic degree”, which are the affective feature amounts,
"Simple / decorative degree", "Modern / classic degree", "Abstract / realistic degree", "Warm / cool degree"
Etc. are adopted. It should be noted that this is merely a simplified example for the sake of explanation, and other items such as the number of colors, the degree of complexity, the number of stitches (in the case of an embroidery pattern), the degree of curvature, the degree of symmetry, etc. may be used as necessary. Any feature scale can be adopted for.

Next, the details of the image search processing, which is the main part of the present invention, will be described with reference to the flow chart of FIG. 3 and the explanatory views of FIGS. In the figure, when the processing is started, an exemplary image menu including two or more exemplary images (nine in this example) having different characteristics is displayed on the screen of, for example, a CRT display device that constitutes the display unit 2. (Step 30
1) After that, the state of waiting for the similarity designation operation is set (step 302 NO).

FIG. 9 shows an example of a menu display used for the exemplary image classifying operation. As shown in the figure, a square window 10 is provided in the center of the screen 9 of the CRT display device.
Sheets of exemplary images G1 to G9 are displayed. Further, a scroll cursor 11 that can move vertically is displayed on the right edge of the window 10. The scroll cursor 11 can be moved up and down by operating the mouse 15, and accordingly, the nine exemplary images G1 to G2 displayed in the window 10 are scrolled as appropriate,
Another exemplary image can be displayed in the window 10. Further, the image designating cursor 14 indicated by an arrow in FIG. 9 is also movable to an arbitrary exemplary image position by operating the mouse 15, and the cursor 14 can be moved to any position of the exemplary images G1 to G9. Move
When a similar designation operation is performed by clicking the mouse 15 or the like in that state, a similar mark 12 indicated by a circle in the drawing is displayed on the similarly designated exemplary image. A dissimilar mark 13 indicated by a cross mark in the drawing is displayed on the similar designated illustration image. In this example, the similar mark 12 is displayed on the exemplary image G1 and the exemplary image G6, and the dissimilar mark 13 is displayed on the exemplary image G4 and the exemplary image G5.

Returning to the flowchart of FIG. 3, when any similarity designation operation is detected for any of the exemplary images G1 to G9 (YES in step 302), the exemplary image is designated as a similar designated exemplary image or a dissimilar designated exemplary image. It is stored, and at the same time, the similar mark 12 or the dissimilar mark 13 is displayed on the exemplary image (step 303). Thereafter, until the predetermined operation for instructing the completion of the similarity specifying operation is performed (NO in step 304), the above processing (step 30)
1 to 303) are repeated. In this state, when a predetermined similarity / non-designation completion operation is confirmed (step 304Y).
ES), and subsequently, the search condition generation processing is executed based on the feature amount of the exemplary image for which similarity is designated and / or dissimilarity is designated (step 305).

Next, a search condition generating process (step 30)
The details of 5) will be described with reference to FIGS. The general idea for generating search conditions will be explained. Now, let us say that the average position of the images designated to be similar on the sensitivity scale coordinates is x ₁ , x ₂ , ... X _n (original point if there is none), and the average position of the images designated to be dissimilar is y ₁ , y ₂ ,. Assuming that y _n (original point if not present), the position of the obtained virtual search image is a ₁ · (x ₁ −y ₁ ) + x ₁ , a ₂ · (x ₂ −y ₂ ) +
x _2, expressed as _{... a n · (x n -y} n) + x n. Here, a _n is a weighting coefficient.
Therefore, by searching for an image close to this position, the desired similar image can be extracted. This can be performed by searching and extracting in order of decreasing Euclidean distance. The above general idea will be described with a specific example.

In FIG. 4, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the warm / cool axis (Y axis), two similar images are designated and dissimilar designations are made. The case where the number of illustrated images is two is shown. In the figure, points P1 and P2 indicated by white circles are image positions designated as similar, point P3 and point P4 indicated by x marks are image positions designated as dissimilar, and point P indicated by a black circle.
5 and a point P6 indicate the average position of the image indicated by a white circle or an X on the affective coordinates, and a point P7 indicated by a double circle indicates the position of the virtual search image to be obtained. In this case, the line segment connecting the average position P5 of the two similar designated image positions P1 and P2 and the average position P6 of the two dissimilar designated image positions P3 and P4 is moved to the average position P5 side on the similar designated image side. The virtual search image position P7 can be obtained by extending the points so that L1: L2 = 2: 1. Therefore, the Euclidean distance between the virtual search image position P7 and each registered image position is obtained, and n pieces selected in order from the closest one can be obtained as similar images. In particular, according to this example, when there are a plurality of similar and / or dissimilar exemplary images, the tendency of the desired image can be taught to the device more accurately. .

In FIG. 5, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the worm / cool axis (Y axis), one similar designated illustration image and a dissimilar designation are shown. The case where the number of illustrated images is 0 is shown. In this case, the line segment connecting the similar designated image position P8 and the origin O is extended to the similar designated image position P8 side, and the virtual search image position P9 can be obtained as a point where L1: L2 = 2: 1. it can. Therefore, the Euclidean distances between the virtual search image position P9 and each registered image position are respectively obtained, and n pieces selected in order from the ones closer to each other can be obtained as similar images. In particular, according to this example, even if there is an exemplary image that is similar to some extent, but there is no exemplary image that is not similar at all, the tendency of the desired image can be accurately taught to the apparatus side. it can.

In FIG. 6, in two-dimensional emotional characteristic coordinates consisting of a hard / soft axis (X axis) and a worm / cool axis (Y axis), 0 similar illustrations are designated and dissimilar designations are made. The case where the number of illustrated images is one is shown. In this case, the line segment connecting the dissimilar designated image position P10 and the origin O is extended to the origin O side, and the virtual search image position P11 can be obtained as a point where L1: L2 = 2: 1. Therefore, the Euclidean distance between the virtual search image position P11 and each registered image position is obtained, and n pieces selected in order from the closest one can be obtained as similar images. In particular, according to this example, the tendency of the desired image can be properly taught to the apparatus side even when there are no similar images but no similar images at all.

In FIG. 7, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the worm / cool axis (Y axis), two similar designated illustration images and a dissimilar designation are shown. The case where the number of illustrated images is one is shown. In this case, a line segment connecting the average position P14 of the two similar designated image positions P12 and P13 and the dissimilar designated image position P15 is extended to the average designated position P14 side on the similar designated image side, and L1: L2.
The virtual search image position P16 can be obtained as a point where = 2: 1. Therefore, this virtual search image position P1
It is possible to obtain Euclidean distances between 6 and each registered image position, and obtain n similar images selected in order from the closest ones as similar images.

In FIG. 8, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the worm / cool axis (Y axis), one similar illustration is designated and a dissimilar designation is made. The case where the number of illustrated images is two is shown. In this case, the line segment connecting the average position P19 of the two dissimilar designated image positions P17 and P18 and the similar designated image position P20 is extended to the similar designated image position P20 side, and L1: L2 = 2: 1.
Then, the virtual search image position P21 can be obtained. Therefore, the Euclidean distance between the virtual search image position P21 and each registered image position is obtained, and n pieces selected in order from the closest one can be obtained as similar images.

Returning to the flowchart of FIG. 3, if the search condition is obtained in the search condition generating process (step 305), then a predetermined number of similar images are stored in the image storage unit 5 according to the search condition. It is extracted from the database (step 306) and the extracted similar image is displayed on the screen of the image display device (step 307).

FIG. 10 shows a screen display example of the search results obtained in this way. As is clear from a comparison between FIG. 9 and FIG. 10, in the exemplary image menu shown in FIG. 9, exemplary images G1 and G6 including a vertical stripe pattern are designated similarly, and an exemplary image including a horizontal stripe pattern is designated. G4, G5
As a result of dissimilarity designation, in the display screen shown in FIG. 10, nine images G1 including a vertical stripe pattern are displayed.
It will be understood that 1 to G19 are displayed in search form.

Returning to the flow chart of FIG. 3, on the apparatus side, as shown in FIG. 10, the search operator displays the nine extracted images G11 to G19 on the screen, and the search operator performs a similarity re-designation operation. The process waits for the operation (NO in step 308, NO in step 309, step 307). In this state, one of the images G11 to G11
When some kind of similarity designation operation is detected for G19 (YES in step 308), the image is stored as a similar image or a dissimilar image, and at the same time, the similar mark 12 or the dissimilar mark 13 is displayed on the image ( Step 310). After that, until a predetermined operation for instructing the completion of the similarity specifying operation is performed (step 311 NO),
The above processing (steps 307, 308, 310) is repeated. In this state, when a predetermined similarity / non-designation completion operation is confirmed (YES in step 311), subsequently, based on the feature scale value of the exemplary image designated as the similarity designation and / or the dissimilar designation, the same as the above-described case. Then, the search condition generation process is re-executed (step 305), the image extraction process (step 306), and the extracted image display process (step 30).
After 7), another 9 images are newly displayed.

According to the image retrieval apparatus of the above-described embodiment, even if there is no example image that exactly matches the desired image in the initially set example image menu, the example image that is similar to some extent and / or If there are similar images that are not very similar, the tendency of the desired image can be indirectly taught to the device side by using them, and by repeating the similarity / non-identification operation, finally Images can be retrieved efficiently from the image database.

In FIGS. 4 to 8, the virtual search image position is obtained by setting L1: L2 = 2: 1 and the similar image is extracted from the vicinity thereof. In addition, for example, in FIG. 2 of Japanese Examined Patent Publication No. 3-194655. As shown, it is also possible to expand a cone having the apex of the position of the designated similar image or the average position thereof to the side of the similar image, and retrieve and extract the similar image from the internal region thereof.

[0055]

According to the invention of claim 1 or claim 6 of the present application, an exemplary image menu including two or more exemplary images having different characteristics is displayed, and the exemplary image menu included in the displayed exemplary image menu is displayed. Among the images, one or more exemplary images that are similar to and / or are not similar to the desired image are designated, and the desired image is displayed based on each feature scale value of the exemplary image that is designated as the similarity. A search condition for searching is generated, and based on the generated search condition, one image similar to the desired image is selected from the images stored in the image storage means.
Or, two or more are extracted, and the extracted one or two
The above image will be displayed. Therefore, even if there is no example image that exactly matches the desired image in the initially set example image menu, as long as there is one or more example images that are somewhat similar and / or not very similar. By using them, the tendency of the desired image can be indirectly taught to the apparatus side, and the desired image can be searched efficiently.

According to the invention described in claim 2 or claim 7 of this application, when both the similar exemplary image and the dissimilar exemplary image are designated, they are defined by the plurality of types of feature scales. On the characteristic coordinates, it is premised that the desired image exists near a position obtained by extending a straight line connecting the average position of the similar exemplary image positions and the average position of the dissimilar exemplary image positions by a predetermined distance in the similar exemplary image direction. The search condition is generated based on the original. Therefore, when there are a plurality of similar and / or dissimilar exemplary images, the tendency of the desired image can be more accurately taught to the apparatus.

According to the invention described in claim 3 or claim 8 of this application, when only the similar exemplary image is designated,
On the feature coordinates defined by the plurality of types of feature scales, the desired image is present in the vicinity of a position obtained by extending a straight line connecting the average position of the similar exemplary image positions and the origin by a predetermined distance in the similar exemplary image direction. The search condition is generated based on the premise. Therefore, even if some similar images exist, but there is no completely dissimilar example image, the tendency of the desired image can be properly taught to the apparatus side.

According to the invention described in claim 4 or claim 9 of this application, when only dissimilar exemplary images are designated, on the feature coordinates defined by the plurality of types of feature scales, The search condition is generated on the assumption that the desired image exists near the position where a straight line connecting the average position of the dissimilar exemplary image positions and the origin is extended by a predetermined distance in the direction of the origin. Therefore, only the illustrations that are not similar at all,
Even when there is no similar exemplary image at all, the tendency of the desired image can be accurately taught to the apparatus side.

According to the invention of claim 5 or 10 of this application, among the one or more images extracted and displayed, an image similar to the desired image, and / or
Alternatively, one or more dissimilar images are re-designated. Therefore, even if the desired image cannot be found by one search, the desired image can be finally searched efficiently while repeating the similarity redesignation operation.

[Brief description of drawings]

FIG. 1 is a block diagram showing a hardware configuration of a device of the present invention.

FIG. 2 is a configuration diagram of an image database applied to the device of the present invention.

FIG. 3 is a flowchart showing details of image search processing.

FIG. 4 is an explanatory diagram of a search condition generation process.

FIG. 5 is an explanatory diagram of a search condition generation process.

FIG. 6 is an explanatory diagram of a search condition generation process.

FIG. 7 is an explanatory diagram of a search condition generation process.

FIG. 8 is an explanatory diagram of a search condition generation process.

FIG. 9 is a diagram showing an example of a screen display during a similarity specifying operation.

FIG. 10 is a diagram showing a screen display example of search results.

FIG. 11 is a diagram showing a mutual relationship among a sensible feature amount, a physical feature amount, and a mapping rule.

[Explanation of symbols]

1 Central Processing Unit 2 Display Section 3 Operation Section 4 Printer 5 Image Storage Section 6 Image Scanner 7 System Bus 8 Design Sample 9 Screen 10 Window (for Image Display) 11 Scroll Cursor (for Image Scroll) 12 Similar Mark 13 Dissimilar Mark 14 cursor for image designation 15 mouse G1 to G9 exemplary images included in the exemplary image menu G11 to G19 searched images P1, P2, P8, P12, P13, P20 exemplary image positions for similar designation P3, P4, P10, P15, P17, P18 Illustrative image position of dissimilarity designation P5, P6, P14, P19 Average position of exemplary image P7, P9, P11, P16, P21 Virtual search image position to be obtained

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] July 10, 1995

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] Full text

[Correction method] Change

[Correction content]

[Document name] Statement

Title: Image retrieval method and device

[Claims]

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image search device for an exemplary image input method condition for presenting an exemplary image from a large number of images prepared in advance and searching for it, and is particularly suitable as an exemplary image. The present invention relates to an image search device capable of efficiently searching for a desired image even when it is difficult to obtain a desired item.

[0002]

2. Description of the Related Art For example, when designing clothes, curtains, wallpaper, embroidery, etc., a new design is created by selecting a design concept close to the desired design concept from existing design samples and adding appropriate modifications to it. Creating is often done among designers. In order to improve the efficiency of such design work, it is necessary to find a design close to the desired design concept from the design sample as quickly as possible. However, as the number of symbol samples becomes huge, there is a limit to relying on such a symbol sample searching operation manually.

Therefore, conventionally, in order to mechanize such a pattern sample search operation, various image search apparatuses using computer technology have been proposed. In such an image retrieval apparatus, each of the above-mentioned pattern samples is converted into image data through an image scanner or the like, and then the CD-RO.
It is stored in a mass storage medium such as M. At the same time, various image processing such as density value conversion, noise elimination, blur restoration, contour detection enhancement, and connected portion extraction is applied to each image data, and then based on the image processing result. Then, various feature extraction operations are performed, and various image feature amounts such as the number of boundary line pixels, the number of holes, the degree of curve, the number of colors, the color distribution, the contrast, the boundary line pixel distribution, and the color distribution (hereinafter, this will be described later). “Physical feature amount” for comparison with the “sensitive feature amount”). Now, suppose k images are y ₁ , y
₂ , y ₃ , ... Y _k , and the physical characteristics of the n types are x ₁ ,
If x ₂ , x ₃ , ... X _n are used, each image is an n-dimensional vector (x ₁₁ , x ₂₁ , x ₃₁ , ... x _n1 ), (x ₁₂ , x ₂₂ ,
x ₃₂ , ... X _n2 ), ... (X _1k , x _2k , x _3k , ... X _nk ). Therefore, an image y _p that matches the desired design concept is represented as a vector (x _1p , x _2p , x _3p , ... X _np ) using the above-mentioned physical quantity, and this and each of the images y ₁ and y described above are represented. _By finding the distance from the vector corresponding to ₂ , y ₃ , ..., Y _k and selecting the one having the shortest distance among them, it is possible to mechanically retrieve the image most similar to the desired image. Is.

Techniques for improving this type of image retrieval device
As a document, JP-A-5-6437 (the title of the invention is
Title: "Image feature extraction device, image feature matching device and image
Search device ”) or Information CG / CAD Study Group 199
August 4th, Vol. 94, No. 72, p. 43-48
(Paper title: "Kansei characteristics and image characteristics of design images")
It has been known. In these documents, image search
It is possible to use human emotional expressions
It That is, the objects used in the image search device described above.
The physical feature amount is the result of image processing of the given image data.
By applying a predetermined feature extraction operation to the fruits,
While having the advantage of being mechanically and uniquely extracted, its table
Currently, for example, the number of border pixels, the number of holes, curvature, color
Number, color distribution, contrast, border pixel distribution, color distribution, etc.
To express any design concept like
It lacks appropriateness. So this improved image search
The equipment is warm / cold, masculine / feminine, safe
Constant / unstable, symmetric / asymmetric, simplicity / complexity, etc.
Humans (designers) make impressions of arbitrary images like
Focusing on the emotional characteristics used to identify,
Statistical correlation between the physical features and the physical features described above.
Sensitivity features using this correlation
It is possible to search images by quantity. If you explain it conceptually,
The emotional feature amount of an arbitrary image is set to w₁, W₂, W₃,…
w_m, Its physical feature x₁, X₂, X₃, ... x_nWhen
By definition, each affective feature quantity w₁, W₂, W₃, ... w_m
Are n-dimensional vectors (a₁₁・ X₁, A₁₂・ X
₂, A₁₃・ X₃, ... a_1n・ X_n), (A _{twenty one}・ X₁, A_{twenty two}
・ X₂, A_{twenty three}・ X₃, ... a_2n・ X_n), (A₃₁・ X₁,
a₃₂・ X ₂, A₃₃・ X₃, ... a_3n・ X_n), ... (a_m1・
x₁, A_m2・ X₂, A_m3・ X₃, ... a_mn・ X_n) And represented
Be done. Therefore, it is possible to convert physical features into emotional features.
Weighting value group (a₁₁, A₁₂, A₁₃, ... a_1n),
(A_{twenty one}, A_{twenty two}, A_{twenty three}, ... a_2n), (A₃₁, A₃₂, A₃₃,
... a_3n), ... (a_m1, A_m2, A_m3, ... a_mn) Designer
In advance using statistical methods through questionnaire surveys
If you ask for it, from any image data, its affective features
Can be mechanically extracted. Therefore, the image search
In the process, the sensitivity of the input image and the sense of the registered image
By calculating the distance from the sexual feature, the physical feature
Compared with the case of using,
Image search is performed.

[0005]

By the way, in order to retrieve a desired image in this type of image retrieval apparatus, it is necessary to present the characteristics of the image to be retrieved to the apparatus side by some method. If this image feature presentation is performed by numerically inputting all the physical features or the affective features such as linearity [0.9], sharpness / softness [0.8], etc., it will be complicated. Absent. Therefore, conventionally, in this type of image retrieval apparatus, an exemplary image in which the characteristics of an image desired to be retrieved is exemplarily expressed using a diagram is often used. That is, the input exemplary image is mechanically converted into a physical feature amount and / or a sensible feature amount through the image processing process described above, and the feature amount and the feature amount of each registered image are combined. The distances are compared and one or more images similar to the input image are finally retrieved.

[0006] However, in the conventional image search device of the example image input method, such complicated image input operation does not require complicated numerical value input operation, but on the other hand, it exactly matches the feature of the image desired to be searched. If an exemplary image cannot be obtained, it is difficult to find an image having desired characteristics, which is rather inconvenient.

[0007] Initially, the image features are presented by an exemplary image showing only the basic concept, and then the design concept is inflated while adding slight variations such as color change and pattern change to the exemplary image. In case of sequentially searching for corresponding images, coloring and pattern changes must be actually added to the exemplary image each time,
However, there were some problems such as poor usability.

The present invention has been made in view of the above problems, and an object thereof is to obtain an exemplary image that does not exactly match the characteristics of an image desired to be searched, or to use the basic concept. Even if the image features are presented by an exemplary image that shows only the image, and then the design concept is expanded and the corresponding images are sequentially searched while adding a slight variation to the exemplary image, these search operations are performed. An object of the present invention is to provide an image search device capable of efficiently performing the above.

[0009]

The invention according to claim 1 of the present application includes an image storing step of storing a plurality of images with a plurality of types of characteristic amounts attached to each of the plurality of images, and two or more different features. An exemplary image menu displaying step of displaying an exemplary image menu including exemplary images, and an exemplary image similar to and / or dissimilar to the desired image among the exemplary images included in the displayed exemplary image menu. 1 or 2
The similarity specifying step specified above, a search condition generating step of generating a search condition for searching the desired image based on each feature amount of the example image for which the similarity is specified, and the generated search condition On the basis of the above, 1 or 2 images similar to the desired image are selected from the images stored in the image storage means.
It is characterized by comprising an image extraction step of extracting the above and an image display step of displaying the extracted one or more images.

In the invention according to claim 2 of this application, in the invention according to claim 1, when both the similar exemplary image and the dissimilar exemplary image are designated in the search condition generating step. , On the feature coordinates defined by the plurality of types of feature amounts, a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions is a predetermined distance in the similar designated exemplary image direction. It is characterized in that the search condition is generated based on the assumption that the desired image exists near the extended position.

In the invention according to claim 3 of this application, in the invention according to claim 1, in the search condition generating step, when only a similar exemplary image is designated, the plurality of types of feature amounts Based on the premise that the desired image exists in the vicinity of a position where a straight line connecting the average position of the similar designated exemplary image positions and the origin is extended by a predetermined distance in the similar designated exemplary image direction on the feature coordinates defined by It is characterized in that search conditions are generated.

According to a fourth aspect of this application, in the invention according to the first aspect, the search condition generating step is characterized in that when only dissimilar exemplary images are designated, A search is performed based on the assumption that the desired image exists in the vicinity of a position where a straight line connecting the average position of the dissimilar designated exemplary image positions and the origin is extended by a predetermined distance in the direction of the origin on the characteristic coordinates defined by the quantity. It is characterized by generating a condition.

According to a fifth aspect of this application, in the invention according to the first aspect, among the one or more images extracted and displayed, an image similar to a desired image, and And / or a similarity re-designating step for re-designating one or more dissimilar images.

The invention according to claim 6 of the present application is an example including an image storage means for storing a plurality of images with a plurality of types of characteristic amounts respectively stored therein, and two or more exemplary images having mutually different features. One or more exemplary images similar to and / or not similar to the desired image among the exemplary images included in the displayed exemplary image menu and the exemplary image menu displaying means for displaying the exemplary image menu are designated. Based on possible similarity designating means and each feature amount of the exemplary image designated as similarity,
Search condition generating means for generating a search condition for searching the desired image, and one or more images similar to the desired image from the images stored in the image storage means based on the generated search condition. It is characterized by comprising image extracting means for extracting and image displaying means for displaying the one or more extracted images.

In the invention according to claim 7 of this application, in the invention according to claim 6, when both the similar exemplary image and the dissimilar exemplary image are designated by the search condition generating means. , On the feature coordinates defined by the plurality of types of feature amounts, a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions is a predetermined distance in the similar designated exemplary image direction. It is characterized in that the search condition is generated based on the assumption that the desired image exists near the extended position.

In the invention according to claim 8 of this application, in the invention according to claim 6, the search condition generating means, when only similar exemplary images are designated, Based on the premise that the desired image exists in the vicinity of a position where a straight line connecting the average position of the similar designated exemplary image positions and the origin is extended by a predetermined distance in the similar designated exemplary image direction on the feature coordinates defined by It is characterized in that search conditions are generated.

In the invention according to claim 9 of this application, in the invention according to claim 6, the search condition generating means is characterized in that, when only dissimilar exemplary images are designated, A search is performed based on the assumption that the desired image exists in the vicinity of a position where a straight line connecting the average position of the dissimilar designated exemplary image positions and the origin is extended by a predetermined distance in the direction of the origin on the characteristic coordinates defined by the quantity. It is characterized by generating a condition.

According to a tenth aspect of the present application, in the invention according to the sixth aspect, an image similar to a desired image among the one or more images extracted and displayed, and / Or 1 or 2 dissimilar images
It is characterized in that it comprises a similarity redesignating means for redesignating.

[0019]

According to the invention described in claim 1 of this application, a plurality of images are stored with a plurality of types of feature values attached to each, and an example image menu including two or more example images having different features is provided. Among the exemplary images displayed and included in the displayed exemplary image menu, one or more exemplary images similar to and / or dissimilar to the desired image are designated, and the similarity is designated. A search rule for searching for the desired image is generated based on each feature amount of the image, and one or more images similar to the desired image from the stored images are generated based on the generated search condition. The extracted one or more images are displayed.

According to the invention of claim 2 of this application, when both the similar exemplary image and the dissimilar exemplary image are designated, on the characteristic coordinates defined by the plurality of types of characteristic amounts. Based on the premise that the desired image exists near a position obtained by extending a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions in the similar designated exemplary image direction by a predetermined distance. A search condition is generated in.

According to the invention of claim 3 of this application, when only the similar exemplary image is designated, the similar designated exemplary image position is located on the characteristic coordinates defined by the plurality of types of characteristic amounts. The search condition is generated on the assumption that the desired image exists near the position where the straight line connecting the average position and the origin is extended by a predetermined distance in the similar designated illustration direction.

According to the invention of claim 4 of this application, when only the dissimilar exemplary image is designated, the dissimilar designated exemplary is displayed on the feature coordinates defined by the plurality of types of feature amounts. The search condition is generated on the assumption that the desired image exists near the position where a straight line connecting the average position of the image positions and the origin is extended by a predetermined distance in the direction of the origin.

According to the invention of claim 5 of the present application, among the one or more images extracted and displayed, the image similar to the desired image and / or the image not similar to the desired image is one. Or, two or more are designated again.

According to the invention of claim 6 of this application, a plurality of images are stored with a plurality of types of characteristic amounts attached to each of them, and an exemplary image menu including two or more exemplary images with different features is provided. An exemplary image that is displayed and is similar to the desired image among the exemplary images included in the displayed exemplary image menu,
And / or one or two or more dissimilar exemplary images are designated, and based on each feature amount of the exemplary images designated as similarity or similar,
A search condition for searching the desired image is generated, and one or more images similar to the desired image are extracted from the stored image based on the generated search condition, and the extracted one or Two or more images are displayed.

According to the invention of claim 7 of this application, when both the similar exemplary image and the dissimilar exemplary image are designated, on the characteristic coordinates defined by the plurality of types of characteristic amounts, Based on the assumption that the desired image exists near the position where a straight line connecting the average position of the similar designated exemplary image positions and the average position of the dissimilar designated exemplary image positions is extended by a predetermined distance in the similar designated exemplary image direction. Search conditions are generated.

According to the invention of claim 8 of this application, when only the similar exemplary image is designated, the similar designated exemplary image position is located on the characteristic coordinates defined by the plurality of types of characteristic amounts. The search condition is generated on the assumption that the desired image exists near the position where the straight line connecting the average position and the origin is extended by a predetermined distance in the similar designated illustration direction.

According to the invention of claim 9 of this application, when only the dissimilar exemplary image is designated, the dissimilar designated example is displayed on the feature coordinates defined by the plurality of types of feature amounts. The search condition is generated on the assumption that the desired image exists near the position where a straight line connecting the average position of the image positions and the origin is extended by a predetermined distance in the direction of the origin.

According to the invention of claim 10 of this application, among the one or more images extracted and displayed, one image that is similar to the desired image and / or one that is not similar to the desired image Or, two or more are designated again.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. The hardware configuration of the image search apparatus of this embodiment is shown in FIG. As shown in the figure, the hardware of this image search apparatus includes a central processing unit 1, a display unit 2, an operation unit 3, a printer 4, and an image storage unit 5.
And a personal computer system including the image scanner 6.

As is well known, the central processing unit 1 is mainly composed of an MPU, a RAM, a ROM and the like, and the above-mentioned system elements 2 to 2 are connected via a system bus 7.
8 is to be totally controlled.

The display unit 2 is composed of a CRT display, an LCD, etc., and on the display screen of the display unit 2, as shown in detail later with reference to FIGS. 9 and 10, an exemplary image is displayed. Number of first place displayed as menu (9 in this example)
Or a fixed number (nine in this example) of similar images extracted as a search result are displayed in color.

The operation unit 3 is composed of a keyboard, a mouse, etc., and various instruction inputs to this apparatus are performed using this operation unit 3.

The printer 4 is composed of a high precision printer such as a laser color printer and is used for obtaining a hard copy of the searched image.

The image storage unit 5 is composed of a large-capacity storage device such as a write-once type optical disc and a rewritable type optical disc. The image storage unit 5 will be described later in detail with reference to FIGS. 2 and 11. As described above, a large number of images to be searched are stored with their physical features and / or emotional features attached.

The image scanner 6 is for converting an arbitrary pattern sample 8 displayed in color into image data, and this image scanner 6 is an image as will be described later in detail with reference to FIG. It is used for the registration process, or for inputting the pattern sample 8 as an example image.

Those skilled in the art will readily understand that various operating systems, drawing application software and the like are required to realize the above personal computer system.

Next, FIG. 3 is a control flow chart showing the software configuration of the device of the present invention, and the operation of the device of the present invention will be systematically described below with reference to this flow chart and the explanatory views of FIGS. .

First, the image registration process will be described. As shown in FIG. 11, it is assumed that there are now 1000 images or pattern samples to be registered. In such a case, the present invention first secures 50 of them as sample analysis images, and with respect to this sample analysis image,
The emotional feature amount and the physical feature amount are extracted. The extraction of the kansei features is performed by conducting a questionnaire survey on an appropriate number of designers (for example, 100 people) or general consumers. As a result, the sensitivity features of each image forming the sample analysis image are, for example, 0.8 for sharpness / softness, 0.7 for simple decorativeness, 0.6 for dynamic / staticity, and modern / modern. The classic degree is quantified such as 0.3, and the abstract / realistic degree is 0.9. In this example, the value of each feature scale is
It is represented by each numerical value obtained by dividing 0.0 to 1.0 into 10 steps, but this is merely an example, and any unit system can be adopted as necessary. On the other hand, the extraction of physical features is mechanically performed by computer processing. That is, for each of the images constituting the 50 sample analysis images, first the density value conversion, noise removal,
Various image processes such as blur restoration, contour detection enhancement, and extraction of connected parts are performed, and then various feature extraction operations are performed based on the image process results, and finally, for example, the linearity is 0. .9, contrast degree 0.8, complexity degree 0.
Each physical feature amount is quantified, such as 1.

In this way, if the sensible feature amount and the physical feature amount are extracted from each image forming the sample analysis image, then the mapping rule f that defines the correlation between both the feature amounts. Is created by a statistical method. As a result, the mapping rule f is obtained, for example, sharpness / softness = 0.9, linearity-0.2, complexity.

After that, the mapping rule f calculated above is obtained.
By using, the emotional feature amount is mechanically obtained from the 1000 images to be registered, and the physical feature amount and the affective feature amount thus obtained are respectively associated with the corresponding image and stored in the image database. Configured as
Finally, it is stored in the image storage unit 5. An example of the image database created in this way is shown in FIG. In this example, as shown in FIG.
o. Is attached, and as the feature amount, “soft / hard degree”, “static / dynamic degree”, which are the affective feature amounts,
"Simple / decorative degree", "Modern / classic degree", "Abstract / realistic degree", "Warm / cool degree"
Etc. are adopted. It should be noted that this is merely a simple example for the sake of explanation, and if necessary, the number of colors, complexity, number of stitches (in the case of embroidery pattern), curvature, symmetry, etc. Any feature scale can be adopted for.

Next, the details of the image search processing, which is the main part of the present invention, will be described with reference to the flow chart of FIG. 3 and the explanatory views of FIGS. In the figure, when the processing is started, an exemplary image menu including two or more exemplary images (nine in this example) having different characteristics is displayed on the screen of, for example, a CRT display device that constitutes the display unit 2. (Step 30
1) After that, the state of waiting for the similarity designation operation is set (step 302 NO).

FIG. 9 shows an example of a menu display used for the exemplary image classifying operation. As shown in the figure, a square window 10 is provided in the center of the screen 9 of the CRT display device.
Sheets of exemplary images G1 to G9 are displayed. Further, a scroll cursor 11 that can move vertically is displayed on the right edge of the window 10. The scroll cursor 11 can be moved up and down by operating the mouse 15, and accordingly, the nine exemplary images G1 to G2 displayed in the window 10 are scrolled as appropriate,
Another exemplary image can be displayed in the window 10. Further, the image designating cursor 14 indicated by an arrow in FIG. 9 can also be moved to an arbitrary exemplary image position by operating the mouse 15, and the cursor 14 can be moved to any of the exemplary images G1 to G9. Move
When a similar designation operation is performed by clicking the mouse 15 or the like in that state, a similar mark 12 indicated by a circle in the drawing is displayed on the similarly designated exemplary image. A dissimilar mark 13 indicated by a cross mark in the drawing is displayed on the similar designated illustration image. In this example, the similar mark 12 is displayed on the exemplary image G1 and the exemplary image G6, and the dissimilar mark 13 is displayed on the exemplary image G4 and the exemplary image G5.

Returning to the flowchart of FIG. 3, when any similarity designation operation is detected for any of the exemplary images G1 to G9 (YES in step 302), the exemplary image is designated as a similar designated exemplary image or a dissimilar designated exemplary image. It is stored, and at the same time, the similar mark 12 or the dissimilar mark 13 is displayed on the exemplary image (step 303). Thereafter, until the predetermined operation for instructing the completion of the similarity specifying operation is performed (NO in step 304), the above processing (step 30)
1 to 303) are repeated. In this state, when a predetermined similarity / non-designation completion operation is confirmed (step 304Y).
ES), and subsequently, the search condition generation processing is executed based on the feature amount of the exemplary image for which similarity is designated and / or dissimilarity is designated (step 305).

Next, a search condition generating process (step 30)
The details of 5) will be described with reference to FIGS. The general idea for generating search conditions will be explained. Now, let us say that the average position of the images designated to be similar on the sensitivity scale coordinates is x ₁ , x ₂ , ... X _n (original point if there is none), and the average position of the images designated to be dissimilar is y ₁ , y ₂ ,. Assuming that y _n (original point if not present), the position of the obtained virtual search image is a ₁ · (x ₁ −y ₁ ) + x ₁ , a ₂ · (x ₂ −y ₂ ) +
x _2, expressed as _{... a n · (x n -y} n) + x n. Here, a _n is a weighting coefficient.
Therefore, by searching for an image close to this position, the desired similar image can be extracted. This can be performed by searching and extracting in order of decreasing Euclidean distance. The above general idea will be described with a specific example.

In FIG. 4, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the warm / cool axis (Y axis), two similar images are designated and dissimilar designations are made. The case where the number of illustrated images is two is shown. In the figure, points P1 and P2 indicated by white circles are image positions designated as similar, point P3 and point P4 indicated by x marks are image positions designated as dissimilar, and point P indicated by a black circle.
5 and a point P6 indicate the average position of the image indicated by a white circle or an X on the affective coordinates, and a point P7 indicated by a double circle indicates the position of the virtual search image to be obtained. In this case, the line segment connecting the average position P5 of the two similar designated image positions P1 and P2 and the average position P6 of the two dissimilar designated image positions P3 and P4 is moved to the average position P5 side on the similar designated image side. The virtual search image position P7 can be obtained by extending the points so that L1: L2 = 2: 1. Therefore, the Euclidean distance between the virtual search image position P7 and each registered image position is obtained, and n pieces selected in order from the closest one can be obtained as similar images. In particular, according to this example, when there are a plurality of similar and / or dissimilar exemplary images, the tendency of the desired image can be taught to the device more accurately. .

In FIG. 5, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the worm / cool axis (Y axis), one similar designated illustration image and a dissimilar designation are shown. The case where the number of illustrated images is 0 is shown. In this case, the line segment connecting the similar designated image position P8 and the origin O is extended to the similar designated image position P8 side, and the virtual search image position P9 can be obtained as a point where L1: L2 = 2: 1. it can. Therefore, the Euclidean distances between the virtual search image position P9 and each registered image position are respectively obtained, and n pieces selected in order from the ones closer to each other can be obtained as similar images. In particular, according to this example, even if there is an exemplary image that is similar to some extent, but there is no exemplary image that is not similar at all, the tendency of the desired image can be accurately taught to the apparatus side. it can.

In FIG. 6, in two-dimensional emotional characteristic coordinates consisting of a hard / soft axis (X axis) and a worm / cool axis (Y axis), 0 similar illustrations are designated and dissimilar designations are made. The case where the number of illustrated images is one is shown. In this case, the line segment connecting the dissimilar designated image position P10 and the origin O is extended to the origin O side, and the virtual search image position P11 can be obtained as a point where L1: L2 = 2: 1. Therefore, the Euclidean distance between the virtual search image position P11 and each registered image position is obtained, and n pieces selected in order from the closest one can be obtained as similar images. In particular, according to this example, the tendency of the desired image can be properly taught to the apparatus side even when there are no similar images but no similar images at all.

In FIG. 7, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the worm / cool axis (Y axis), two similar designated illustration images and a dissimilar designation are shown. The case where the number of illustrated images is one is shown. In this case, a line segment connecting the average position P14 of the two similar designated image positions P12 and P13 and the dissimilar designated image position P15 is extended to the average designated position P14 side on the similar designated image side, and L1: L2.
The virtual search image position P16 can be obtained as a point where = 2: 1. Therefore, this virtual search image position P1
It is possible to obtain Euclidean distances between 6 and each registered image position, and obtain n similar images selected in order from the closest ones as similar images.

In FIG. 8, in the two-dimensional emotional characteristic coordinates consisting of the hard / soft axis (X axis) and the worm / cool axis (Y axis), one similar illustration is designated and a dissimilar designation is made. The case where the number of illustrated images is two is shown. In this case, the line segment connecting the average position P19 of the two dissimilar designated image positions P17 and P18 and the similar designated image position P20 is extended to the similar designated image position P20 side, and L1: L2 = 2: 1.
Then, the virtual search image position P21 can be obtained. Therefore, the Euclidean distance between the virtual search image position P21 and each registered image position is obtained, and n pieces selected in order from the closest one can be obtained as similar images.

Returning to the flowchart of FIG. 3, if the search condition is obtained in the search condition generating process (step 305), then a predetermined number of similar images are stored in the image storage unit 5 according to the search condition. It is extracted from the database (step 306) and the extracted similar image is displayed on the screen of the image display device (step 307).

FIG. 10 shows a screen display example of the search results obtained in this way. As is clear from a comparison between FIG. 9 and FIG. 10, in the exemplary image menu shown in FIG. 9, exemplary images G1 and G6 including a vertical stripe pattern are designated similarly, and an exemplary image including a horizontal stripe pattern is designated. G4, G5
As a result of dissimilarity designation, in the display screen shown in FIG. 10, nine images G1 including a vertical stripe pattern are displayed.
It will be understood that 1 to G19 are displayed in search form.

Returning to the flow chart of FIG. 3, on the apparatus side, as shown in FIG. 10, the search operator displays the nine extracted images G11 to G19 on the screen, and the search operator performs a similarity re-designation operation. The process waits for the operation (NO in step 308, NO in step 309, step 307). In this state, one of the images G11 to G11
When some kind of similarity designation operation is detected for G19 (YES in step 308), the image is stored as a similar image or a dissimilar image, and at the same time, the similar mark 12 or the dissimilar mark 13 is displayed on the image ( Step 310). After that, until a predetermined operation for instructing the completion of the similarity specifying operation is performed (step 311 NO),
The above processing (steps 307, 308, 310) is repeated. In this state, when a predetermined similarity / non-designation completion operation is confirmed (YES in step 311), subsequently, based on the feature scale value of the exemplary image designated as the similarity designation and / or the dissimilar designation, the same as the above-described case. Then, the search condition generation process is re-executed (step 305), the image extraction process (step 306), and the extracted image display process (step 30).
After 7), another 9 images are newly displayed.

According to the image retrieval apparatus of the above-described embodiment, even if there is no example image that exactly matches the desired image in the initially set example image menu, the example image that is similar to some extent and / or If there are similar images that are not very similar, the tendency of the desired image can be indirectly taught to the device side by using them, and by repeating the similarity / non-identification operation, finally Images can be retrieved efficiently from the image database.

In FIGS. 4 to 8, the virtual search image position is obtained by setting L1: L2 = 2: 1 and the similar image is extracted from the vicinity thereof. In addition, for example, in FIG. 2 of Japanese Examined Patent Publication No. 3-194655. As shown, it is also possible to expand a cone having the apex of the position of the designated similar image or the average position thereof to the side of the similar image, and retrieve and extract the similar image from the internal region thereof.

[0055]

According to the invention of claim 1 or claim 6 of the present application, an exemplary image menu including two or more exemplary images having different characteristics is displayed, and the exemplary image menu included in the displayed exemplary image menu is displayed. Among the images, one or more exemplary images that are similar to and / or are not similar to the desired image are designated, and the desired image is displayed based on each feature scale value of the exemplary image that is designated as the similarity. A search condition for searching is generated, and based on the generated search condition, one image similar to the desired image is selected from the images stored in the image storage means.
Or, two or more are extracted, and the extracted one or two
The above image will be displayed. Therefore, even if there is no example image that exactly matches the desired image in the initially set example image menu, as long as there is one or more example images that are somewhat similar and / or not very similar. By using them, the tendency of the desired image can be indirectly taught to the apparatus side, and the desired image can be searched efficiently.

According to the invention described in claim 2 or claim 7 of this application, when both the similar exemplary image and the dissimilar exemplary image are designated, they are defined by the plurality of types of feature scales. On the characteristic coordinates, it is premised that the desired image exists near a position obtained by extending a straight line connecting the average position of the similar exemplary image positions and the average position of the dissimilar exemplary image positions by a predetermined distance in the similar exemplary image direction. The search condition is generated based on the original. Therefore, when there are a plurality of similar and / or dissimilar exemplary images, the tendency of the desired image can be more accurately taught to the apparatus.

According to the invention described in claim 3 or claim 8 of this application, when only the similar exemplary image is designated,
On the feature coordinates defined by the plurality of types of feature scales, the desired image is present in the vicinity of a position obtained by extending a straight line connecting the average position of the similar exemplary image positions and the origin by a predetermined distance in the similar exemplary image direction. The search condition is generated based on the premise. Therefore, even if some similar images exist, but there is no completely dissimilar example image, the tendency of the desired image can be properly taught to the apparatus side.

According to the invention described in claim 4 or claim 9 of this application, when only dissimilar exemplary images are designated, on the feature coordinates defined by the plurality of types of feature scales, The search condition is generated on the assumption that the desired image exists near the position where a straight line connecting the average position of the dissimilar exemplary image positions and the origin is extended by a predetermined distance in the direction of the origin. Therefore, only the illustrations that are not similar at all,
Even when there is no similar exemplary image at all, the tendency of the desired image can be accurately taught to the apparatus side.

According to the invention of claim 5 or 10 of this application, among the one or more images extracted and displayed, an image similar to the desired image, and / or
Alternatively, one or more dissimilar images are re-designated. Therefore, even if the desired image cannot be found by one search, the desired image can be finally searched efficiently while repeating the similarity redesignation operation.

[Brief description of drawings]

FIG. 1 is a block diagram showing a hardware configuration of a device of the present invention.

FIG. 2 is a configuration diagram of an image database applied to the device of the present invention.

FIG. 3 is a flowchart showing details of image search processing.

FIG. 4 is an explanatory diagram of a search condition generation process.

FIG. 5 is an explanatory diagram of a search condition generation process.

FIG. 6 is an explanatory diagram of a search condition generation process.

FIG. 7 is an explanatory diagram of a search condition generation process.

FIG. 8 is an explanatory diagram of a search condition generation process.

FIG. 9 is a diagram showing an example of a screen display during a similarity specifying operation.

FIG. 10 is a diagram showing a screen display example of search results.

FIG. 11 is a diagram showing a mutual relationship among a sensible feature amount, a physical feature amount, and a mapping rule.

[Explanation of reference symbols] 1 central processing unit 2 display unit 3 operation unit 4 printer 5 image storage unit 6 image scanner 7 system bus 8 sample pattern 9 screen 10 window (for image display) 11 scroll cursor (for image scroll) 12 similar Mark 13 Dissimilar mark 14 Image designation cursor 15 Mouse G1 to G9 Exemplary images included in exemplary image menu G11 to G19 Retrieved images P1, P2, P8, P12, P13, P20 Exemplary image positions for similar designation P3, P4 , P10, P15, P17, P18 Non-similarly designated exemplary image positions P5, P6, P14, P19 Average position of exemplary images P7, P9, P11, P16, P21 Obtained virtual search image position

Claims (10)

[Claims] 1. An image storing step of storing a plurality of images with a plurality of types of characteristic amounts stored therein, and an exemplary image menu displaying step of displaying an exemplary image menu including two or more exemplary images having different characteristics from each other. In the example images included in the displayed example image menu, an example image similar to the desired image and / or an example image specifying step of specifying one or more example images that are not similar to each other; A search condition generating step of generating a search condition for searching the desired image based on each feature amount of the illustrated image, and an image stored in the image storage means based on the generated search condition An image extracting step of extracting one or more images similar to the desired image from the image, and an image displaying step of displaying the extracted one or more images. Image retrieval method characterized by. 2. In the search condition generating step, when both a similar exemplary image and a dissimilar exemplary image are designated, the similar designation exemplary embodiment is performed on the characteristic coordinates defined by the plurality of types of characteristic amounts. A search condition is generated based on the assumption that a desired image exists in the vicinity of a position that extends a straight line connecting the average position of image positions and the average position of the dissimilar designated exemplary image positions in the similar designated exemplary image direction by a predetermined distance. The image search method according to claim 1, wherein: 3. In the search condition generating step, when only a similar exemplary image is designated, the average position and the origin of the similar designated exemplary image positions on the characteristic coordinates defined by the plurality of types of characteristic amounts. The image search method according to claim 1, wherein the search condition is generated based on the assumption that a desired image exists near a position where a straight line connecting and is extended by a predetermined distance in the similar designated illustration direction. 4. The search condition generating step, when only dissimilar exemplary images are designated, an average position of the dissimilarly designated exemplary image positions on the feature coordinates defined by the plurality of types of feature amounts. The search condition is generated based on the assumption that a desired image exists near a position where a straight line connecting the line and the origin is extended by a predetermined distance in the direction of the origin.
Image retrieval method described in. 5. The extracted or displayed 1 or 2
The similarity re-designating step for re-designating one or more images that are similar to and / or are not similar to the desired image among the above-mentioned images is provided. Image search method. 6. An image storage unit for storing a plurality of images with a plurality of types of characteristic amounts stored therein, and an exemplary image menu display unit for displaying an exemplary image menu including two or more exemplary images having different characteristics from each other. In the exemplary images included in the displayed exemplary image menu, an exemplary image similar to the desired image and / or one or more exemplary images dissimilar to the desired image can be designated, and the category Based on each feature amount of the designated exemplary image,
Search condition generation means for generating a search condition for searching the desired image, and one or more images similar to the desired image from the images stored in the image storage means based on the generated search condition. An image retrieval apparatus comprising: an image extracting unit for extracting; and an image displaying unit for displaying the extracted one or more images. 7. The search condition generating means, when both a similar exemplary image and a dissimilar exemplary image are designated, the similarity designation exemplary on the feature coordinates defined by the plurality of types of feature amounts. A search condition is generated based on the assumption that a desired image exists in the vicinity of a position that extends a straight line connecting the average position of image positions and the average position of the dissimilar designated exemplary image positions in the similar designated exemplary image direction by a predetermined distance. 7. The method according to claim 6, wherein
The image search device described in 1. 8. The search condition generation means, when only similar exemplary images are designated, on the feature coordinates defined by the plurality of types of characteristic amounts, the average position and the origin of the similar designated exemplary image positions. 7. The image search device according to claim 6, wherein the search condition is generated on the assumption that the desired image exists near a position where a straight line connecting the and is extended by a predetermined distance in the similar designated illustration direction. 9. The search condition generating means, when only dissimilar exemplary images are designated, on the feature coordinates defined by the plurality of types of feature amounts, the average position of the dissimilar designated exemplary image positions. 7. The image search apparatus according to claim 6, wherein the search condition is generated on the assumption that the desired image exists near a position where a straight line connecting the point and the origin is extended by a predetermined distance in the direction of the origin. 10. Among the one or more images extracted and displayed, an image similar to a desired image, and / or
Alternatively, the image retrieving apparatus according to claim 6, further comprising similarity re-designating means for re-designating one or more dissimilar images.