JP3284528B2 - Image search method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an example image input type image search apparatus for searching an image matching a user's preference from a large number of images prepared in advance based on an example image. The present invention relates to an image search device capable of efficiently searching for a desired image even when it is difficult to obtain an optimum image.

[0002]

2. Description of the Related Art For example, when designing clothes, curtains, wallpaper, embroidery, etc., a new design is selected by selecting an existing design sample that is close to a desired design concept and appropriately modifying it. It is often done among designers. In order to increase the efficiency of such a design work, it is necessary to find a design similar to a desired design concept from a design sample as quickly as possible. However, as the number of symbol samples has become enormous, there has been a limit to relying on humans for such symbol sample retrieval work.

In order to mechanize such a design sample search operation, various image search devices using computer technology have been proposed. In such an image search device, each of the above-mentioned symbol samples is converted into image data via an image scanner or the like, and is converted into a CD-RO.
M and the like are stored in a large-capacity storage medium. At the same time, each image data is subjected to various image processing such as density value conversion, noise elimination, blur restoration, contour detection emphasis, and connection portion extraction, and then, based on the image processing results. Various feature extraction operations are performed, and various image feature amounts (hereinafter, referred to as the number of boundary pixels, the number of holes, the degree of curve, the number of colors, the color distribution, the contrast, the boundary pixel distribution, the color distribution, etc.) (Referred to as “physical feature value” for comparison with “sensual feature value”). Suppose now that k images are y ₁ ,
_{y 2, y 3, ... y} k, n kinds of physical feature quantities x _1, x _2,
x _3, ... if put as x _n, each image n-dimensional vector _{(x 11, x 21, x} 31, ... x n1), (x 12, x 22, x 32,
.. _Xn2 ),... ( _X1k , _x2k , _x3k ,... _Xn1 ). Therefore, the image y _p that matches the desired design concept using physical characteristics of above vector (x _{1p,
x 2p, x 3p, ... x} np) represents As, which each image y ₁ described above, y _2, y _3, ... obtains the Euclidean distance between the feature vector corresponding to y _k, whose distance among them By selecting an image that is close to, an image most similar to the desired image can be mechanically searched.

As a technical document relating to the improvement of this type of image retrieval apparatus, Japanese Patent Application Laid-Open No. 5-6437 (Title of Invention: "Image Feature Extraction Apparatus, Image Feature Collation Apparatus and Image Retrieval Apparatus"), or information CG・ CAD Study Group 199
Vol. 94, no. 72, p. 43-48
(Paper title: "Kansei and image features of design images")
It has been known. In these documents, it is possible to perform an image search using a human emotional expression. That is, the physical feature amount used in the above-described image search device is mechanically and uniquely extracted by applying a predetermined feature extraction operation to the result of image processing of given image data. On the other hand, although it has advantages, its expression is not always necessary to express any design concept, such as boundary pixel number, hole number, curve degree, color number, color distribution, contrast, boundary pixel distribution, color distribution, etc. Lack of properness. Therefore, in this improved image retrieval device, human (designer) such as warm / cold, masculine / feminine, stability / instability, symmetric / asymmetric, simplicity / complexity, etc. ) Focuses on the kansei features used to identify the impression of an arbitrary image, and calculates the correlation between these kansei features and the physical features described above using a statistical method. Image retrieval based on emotional features using relationships was enabled. To explain conceptually, the emotional features of an arbitrary image are represented by w ₁ , w ₂ , w ₃ ,.
w _m, the physical feature quantities _{x 1, x 2, x 3} , ... when defined as x _n, each sensitive feature amount _{w 1, w 2, w 3} , ... w m are each n-dimensional vector ( _{a 11 · x 1, a 12} · x 2, a 13 ·
_{x 3, ... a 1n · x} n), (a 21 · x 1, a 22 · x 2, a 23 ·
_{x 3, ... a 2n · x} n), (a 31 · x 1, a 32 · x 2, a 33 ·
_{x 3, ... a 3n · x} n), ... (a m1 · x 1, a m2 · x 2, a m3
· X ₃ , ..., a _mn · x _n ). Therefore, the weighting value groups (a ₁₁ , a ₁₂ , a ₁₃ ,... A _1n ), (a ₂₁ , a ₂₂ , a ₂₃ ,
_{... a 2n), (a 31} , a 32, a 33, ... a 3n), ... (a m1,
If a _m2 , a _m3 ,... a _mn ) are obtained in advance by a statistical method through a questionnaire survey or the like to the designer, it is possible to mechanically extract the sensible feature amount from arbitrary image data. Therefore, in the image search process, the distance between the sensible feature of the input image and the kansei feature of the registered image is obtained, so that the sensibility of the designer is added in comparison with the case where the physical feature is used. Image search becomes possible.

[0005]

By the way, in this type of image search apparatus, in order to search for a desired image, it is necessary to present the characteristics of the image to be searched to the apparatus side by some method. This image feature presentation is complicated if all physical or sensual feature values are input as numerical values such as linearity [0.9], sharpness / softness [0.8], etc. Incessantly.

[0006] Conventionally, in this type of image retrieval apparatus, it is often practiced to use an example image in which the features of an image desired to be retrieved are exemplarily expressed using drawings. That is, the input example image is mechanically converted into a physical feature amount and / or a kansei feature amount through the above-described image processing process, and the feature amount and the feature amount of each registered image are compared. The distances are compared and one or more images similar to the input image are finally retrieved.

However, in such a conventional example image input type image retrieval apparatus, complicated numerical value input operation is not required when presenting image characteristics, but on the other hand, an example which exactly matches the characteristic of the image desired to be retrieved. If an image cannot be obtained, it is difficult to find an image having desired characteristics, which is rather inconvenient. Also, initially,
The image feature is presented by an illustration showing only the basic concept, and then the design concept is expanded while adding slight variations such as coloring change and pattern change to the example image, and the corresponding images are sequentially searched. In such a case, a color change or a pattern change must be actually made to the example image each time, and there is a problem that the usability is extremely low.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and has as its object to provide a case where an example image which exactly matches the characteristics of an image desired to be searched cannot be obtained, or a basic concept. In the case where the image features are presented by an example image representing only the image, and then the corresponding image is sequentially searched by expanding the design concept while adding a slight variation to the example image, these search operations are also performed. Is to provide an image search method and apparatus capable of efficiently performing the image search.

[0009]

According to a first aspect of the present invention, there is provided a registered image storing step of storing a plurality of images to be a registered image, and a plurality of images selected from the registered images. A mapping rule creating step of creating a mapping rule that defines a correlation between a kansei feature obtained by inputting a kansei feature for the sample image and a physical feature obtained by image processing the sample image. A feature amount associating step of associating a sensual feature amount with the registered image based on a physical feature amount based on the created mapping rule; and exemplarily expressing a feature of an image desired to be searched using a drawing. An example image inputting step of inputting an example image, and converting the feature amount of the input example image into a kansei feature amount scale and a physical feature amount scale. An example image feature amount manual correction step for correcting at an arbitrarily selected use scale, and from the images stored in the registered image storage step based on the corrected example image feature amount. A similar registered image extracting step of extracting a registered image similar to the example image based on the feature amount associated with the mapping rule, and visually displaying the registered image extracted in the similar registered image extracting step. A registered image output step to output;
On condition that the search is continued, one of the one or more images visually output in the registered image output step is input as a new example image for continuing the search, and the example image feature amount manual correction step is performed. The image re-executing step of shifting the process to the step S. Thus, the example image feature amount manual correction step, the similar registered image extracting step, and the registered image output step can be performed while selecting an arbitrary feature amount at any time. The desired image can be repeatedly extracted.

The "feature value" includes, for example, a physical feature value, a sensitivity feature value, and the like.

The "exemplary image inputting step" may include an external inputting step of inputting an arbitrary exemplary image from the outside using, for example, a video interface, an image scanner, a tablet, or the like. Alternatively, a preset input step of selecting and inputting one example image from a plurality of example images included in a previously prepared example image menu may be included. Of course, a combination of these is also possible.

[0012] In the first invention, further, a search continuation example for inputting one of one or more images output in the “image output step” as a new example image for continuation of the search. An image input step may be provided.

According to a second aspect of the present invention, the example image feature manual correction step in the first aspect is characterized in that a unique feature amount display step of displaying a unique feature amount for each feature of the input example image, and a predetermined correction operation A feature value correcting step of recognizing an input value input in step 2 as a desired feature value absolute value of the corresponding feature and correcting a unique feature value of the feature to the input value, and displaying the corrected feature value And displaying a corrected feature amount.

According to a third aspect of the present invention, the example image feature manual correction step in the first aspect is an adjustment range display step of displaying a predetermined adjustment range of a unique feature amount for each feature of the input example image, A feature that recognizes an input value input by the correction operation as a desired offset value with respect to a unique feature amount of a corresponding feature, and corrects the value of the unique feature amount by adding the input value to the unique feature amount. The method further comprises a quantity correction step, and a corrected feature quantity display step of displaying an offset value of the corrected feature quantity with respect to the unique feature quantity in association with the predetermined adjustment range.

The above-mentioned first invention can be realized as a fourth invention. That is, the fourth invention of the present application is a registered image storage unit that stores a plurality of images to be a registered image, and a feeling obtained by inputting a feeling characteristic for a plurality of sample images selected from the registered images. Rule creation means for creating a mapping rule that defines a correlation between a physical feature and a physical feature obtained by performing image processing on the sample image; and a physical feature based on the created mapping rule. A feature amount associating unit for associating a sensual feature amount with the registered image, and an example image inputting unit for inputting an example image obtained by exemplifying a feature of an image desired to be searched using drawings. , An example image feature for correcting the input example image feature at a use scale arbitrarily selected from a kansei feature amount scale and a physical feature amount scale Based on the characteristic amount of the example image after the correction, the registered image similar to the example image is associated with the manual correction unit from the images stored in the registered image storage unit using the mapping rule. Similar registered image extracting means for extracting based on the feature amount obtained, registered image output means for visually outputting the registered image extracted by the similar registered image extracting means, and outputting the registered image on condition that search continues. Image re-executing means for inputting one of the one or more images visually output by the means as a new example image for continuing the search, and shifting the processing to the example image feature amount manual correction means With
Thereby, the example image feature amount manual correction unit, the similar registered image extraction unit, and the registered image output unit can be repeated while selecting an arbitrary feature amount at any time to extract a desired image. .

The "feature value" includes, for example, a physical feature value, a sentiment feature value, and the like.

As an example of the "exemplary image input means", there is an external input device such as a video interface, an image scanner, a tablet, or the like, which is capable of externally inputting an exemplary image. As another example, there is a preset input device capable of selecting and inputting one example image from a plurality of example images included in a previously prepared example image menu. Of course, a combination of these is also possible.

[0018] In the fourth invention, further, an example image for search continuation for inputting one of one or more images output by the "image output means" as a new example image for continuation of the search. Input means may be provided.

The above-described second invention can be realized as a fifth invention. That is, in the fifth invention of the present application, the example image feature manual correction means in the fourth invention is characterized in that the characteristic image amount display means for displaying the characteristic amount of each characteristic of the input example image, and an input by a predetermined correction operation. Input value is recognized as a desired feature absolute value of the corresponding feature,
The apparatus comprises: a characteristic amount correcting unit that corrects a characteristic amount of the characteristic into the input value; and a corrected characteristic amount displaying unit that displays the corrected characteristic amount.

The above-described third invention can be realized as a sixth invention. That is, in the sixth invention of the present application, the example image feature manual correction means in the fourth invention is characterized in that an adjustment range display means for displaying a predetermined adjustment range of the characteristic feature amount of each characteristic of the input example image; A feature amount correction that recognizes an input value input by an operation as a desired offset value with respect to a unique feature amount of a corresponding feature, and adds the input value to the unique feature amount to correct the value of the unique feature amount. Means, and an after-correction feature amount display means for displaying an offset value of the after-correction feature amount with respect to the unique feature amount in association with the predetermined adjustment range.

[0021]

According to the first aspect of the present invention, a plurality of images are stored as registered images, and a sentiment characteristic amount obtained by inputting a sentiment characteristic for a plurality of sample images selected from the registered images. And a mapping rule that defines a correlation between the sample image and a physical feature obtained by performing image processing is created. Based on the created mapping rule, a sentiment is applied to the registered image based on the physical feature. A feature amount is associated with the input image, and an example image which exemplarily expresses the feature of the registered image desired to be searched for using a drawing is input, and the feature amount of the input example image is converted into a kansei feature amount scale and a physical The image is modified on a use scale arbitrarily selected from the feature amount scale, and is similar to the example image from the stored registered images based on the feature amount of the example image after the modification. A registered image is extracted based on the feature amount associated using the mapping rule, the extracted registered image is visually output, and is visually output in the registered image output step on condition that search is continued. By inputting one of the obtained one or more images as a new example image for continuing the search, the processing is shifted to the above-described example image feature amount manual correction step. The correction, the extraction of the similar registered image, and the output of the registered image can be repeated while selecting an arbitrary feature amount as needed, and a desired image can be extracted.

According to the second aspect of the present invention, the characteristic amount of each characteristic of the input example image is displayed, and the desired characteristic amount of the characteristic corresponding to the input value input by the predetermined correction operation is displayed. The feature value is recognized as an absolute value, the unique feature value of the feature is corrected to the input value, and the corrected feature value is displayed.

According to the third invention of this application, a predetermined adjustment range of the characteristic amount of each characteristic of the input example image is displayed, and an input value input by a predetermined correction operation is applied to a characteristic corresponding to the input value. The input value is recognized as a desired offset value with respect to the unique feature value, the value of the unique feature value is corrected by adding the input value to the unique feature value, and the offset value of the corrected feature value with respect to the unique feature value is calculated. It is displayed in association with the predetermined adjustment range.

According to the fourth aspect of the present invention, a plurality of images are stored as registered images, and a sentiment characteristic amount obtained by inputting a sentiment characteristic for a plurality of sample images selected from the registered images. And a mapping rule that defines a correlation between the sample image and a physical feature obtained by performing image processing is created. Based on the created mapping rule, a sentiment is applied to the registered image based on the physical feature. A feature amount is associated with the input image, and an example image which exemplarily expresses the feature of the registered image desired to be searched for using a drawing is input, and the feature amount of the input example image is converted into a kansei feature amount scale and a physical The image is modified at a use scale arbitrarily selected from the characteristic amount scale, and based on the characteristic amount of the example image after the modification, the stored registered image is classified into the example image. The registered image to be extracted is extracted based on the feature amount associated using the mapping rule, and the extracted registered image is visually output. By inputting one of the output one or more images as a new example image for continuing the search, the processing is shifted to the example image feature amount manual correction unit, and thereby, the example image feature amount is corrected. The manual correction, the extraction of the similar registered image, and the output of the registered image can be repeated while selecting an arbitrary feature amount as needed, and a desired image can be extracted.

According to the fifth aspect of the present invention, the characteristic amount of each characteristic of the input example image is displayed, and the desired characteristic amount of the characteristic corresponding to the input value inputted by the predetermined correction operation is displayed. The feature value is recognized as an absolute value, the unique feature value of the feature is corrected to the input value, and the corrected feature value is displayed.

According to the sixth invention of this application, a predetermined adjustment range of the characteristic amount of each characteristic of the input example image is displayed, and an input value input by a predetermined correction operation is applied to a characteristic corresponding to the input value. The input value is recognized as a desired offset value with respect to the unique feature value, the value of the unique feature value is corrected by adding the input value to the unique feature value, and the offset value of the corrected feature value with respect to the unique feature value is calculated. It is displayed in association with the predetermined adjustment range.

[0027]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention will be described below in detail with reference to the accompanying drawings. First, FIG. 1 shows a hardware configuration of an image search device according to one embodiment of the present invention.
As shown in FIG. 1, the hardware of the image search device includes a central processing unit 1, a display unit 2, an operation unit 3, a printer 4, an image storage unit 5, a video interface 6, a tablet 7, and an image scanner 8. It is configured as a personal computer system.

As is well known, the central processing unit 1 is mainly composed of an MPU, a RAM, a ROM, and the like.
8 is generally controlled.

The display unit 2 is composed of a CRT display, an LCD, and the like. On the display screen of the display unit 2, as will be described in detail later, an input example image and its physical characteristic amount Alternatively, one or two or more images extracted as a result of a search, a sentiment feature, or a search result are displayed in color.

The operation unit 3 is composed of a keyboard, a mouse, and the like. Various instruction inputs to the apparatus are performed using the operation unit 3.

The printer 4 is composed of a high-precision printer such as a laser color printer, and is used for obtaining a hard copy of a searched image.

The image storage unit 5 is composed of a large-capacity storage device such as a write-once optical disk or a rewritable optical disk. As will be described later, the image storage unit 5 stores a large number of images to be searched. , And its physical feature amount and / or sensibility feature amount.

The video interface 6 is for taking in an example picture from the outside in the form of a video signal. The video interface 6 is connected to a VCR, a TV and the like as necessary.

The tablet 7 is for drawing an arbitrary figure by hand on the screen of the display unit 2.
The tablet 7 is used to input a handwritten figure as an example image.

The image scanner 8 is for converting an arbitrary symbol sample 10 displayed in color to image data.
It is used for image registration processing or for inputting the design sample 10 as an example image.

It should be understood by those skilled in the art that various operating systems, drawing application software, and the like are required to realize the above-described personal computer system.

FIGS. 2 to 5 are control flowcharts showing the software configuration of the apparatus of the present invention. The operation of the apparatus of the present invention will be described below with reference to these flowcharts and the explanatory diagrams of FIGS. It will be described systematically.

First, the image registration processing will be described. As shown in FIG. 10, it is assumed that there are 1000 images to be temporarily registered. In such a case, according to the present invention, first, 50 of them are secured as a sample analysis image, and a kansei feature amount and a physical feature amount are extracted from the sample analysis image. The extraction of the emotional feature is performed by conducting a questionnaire survey on an appropriate number of designers (for example, 100) or general consumers. As a result, the sensual feature amount of each image constituting the sample analysis image is, for example, 0.8 for sharp softness, 0.7 for simple decorative, and 0.7 for dynamic
The static degree is quantified as 0.6, the modern classic degree is 0.3, and the abstract / realistic degree is quantified as 0.9. In this example, the value of each feature amount is 0.0 to 1.0
It is represented by each numerical value divided into 0 stages. On the other hand, the extraction of the physical feature is mechanically performed by computer processing. That is, each image constituting the 50 sample analysis images is first subjected to various image processing such as conversion of density values, removal of noise, restoration of blur, detection and enhancement of contours, and extraction of connected portions. Then, various feature extraction calculations are performed based on the image processing result, and finally, for example, the linearity is 0.9, the contrast is 0.8, the complexity is 0.1, and so on. Physical features are quantified.

After the sensual feature and the physical feature have been extracted from each of the images constituting the sample analysis image in this manner, the mapping rule f defining the correlation between the two features is then obtained. Is created by a statistical method. As a result, for example, sharpness / softness = 0.9 ·
A mapping rule f is obtained such as linearity−0.2 · complexity.

Hereinafter, the mapping rule f obtained above
, The emotional feature is mechanically obtained from the 1000 images to be registered, and the obtained physical and emotional features are respectively associated with the corresponding images to form a database. This database is finally stored in the image storage unit 5.

Next, an image search process which is a main part of the present invention will be described. As shown in FIG. 2, the image search processing according to the present invention includes an example image input processing (step 201) for inputting an example image that exemplifies the features of an image desired to be searched using drawings. ), An example image feature correction process (Step 202) for manually correcting the physical characteristics and / or sensibility characteristics of the input example image;
Based on the physical characteristics and / or sensibility characteristics of the example image after the correction of the previous period, the image image similar to the example image feature is selected from among a large number of images stored in the image storage unit 5 as the image storage unit. Image extraction processing for extracting an image to be extracted (step 203), search image display processing for outputting the extracted image to the display unit 2 (step 204), and continuation of the search (step 205 YES). And an image re-executing process (step 206) for inputting one of the output one or more images as a new example image for continuing the search.

FIG. 3 shows details of the example image input process (step 201). In the figure, when the process is started, the process enters a state of waiting for designation of an example mode (step 30).
1). In this state, when the external image input mode is selected by a predetermined operation of the mouse of the operation unit 3, image input processing (step 302) is subsequently executed, and any one of the video interface 6, the tablet 7, and the image scanner 8 is executed. Thus, an external image can be input as an example image. Next, after the input example image is subjected to image processing, its physical feature amounts (linearity, contrast, complexity, etc.) are extracted and stored (step 303), and thereafter, using the aforementioned mapping rule f. Then, the conversion and storage processing to the emotional feature amount is performed (step 304).

FIG. 6 shows an example of the display screen at the time of external input of the example image.
Shown in In this example, a square window 12 is set at the upper left of a screen 11 such as a CRT display device, and in this window 12, a video interface 6, a tablet 7, an image scanner 8 and the like are used to input externally. Image 13 is displayed. In the lower left part of the screen 11, sensible characteristic items 14 such as softness, warmth, brightness, etc. are displayed, and on the right side thereof, a linear indicator 15 and a pointer 16 are used. The feature amount corresponding to item 14 is displayed. As will be described later, in this embodiment, the correction of the feature amount can be performed by using either the absolute value or the offset value as needed.
In this example, the case of an offset value is shown. In either case, the pointer 16 displayed on the indicator 15 can be moved left and right by operating the mouse 17.

Returning to the flowchart of FIG. 3, when the preset image input mode is selected as an example mode, an example image menu is displayed (step 305), and then a state of waiting for a selection operation of any example image. (Step 306). In this state, if a predetermined selection operation is performed (YES in step 306), the physical feature amount and the sensual feature amount are obtained for the input example image in the same manner as in the case of the above-described external image input mode (step 306). 307, 308). However, when this example image menu is used, these feature amounts have already been obtained by the method shown in FIG. 10 at the time of registration of the example image. No calculations need to be performed.

FIG. 7 shows an example of the display screen at the time of inputting the example image preset. In this example, a square window 18 is set in a substantially left half of the screen 11, and an example image menu including nine images G1 to G9 is displayed in the window 18. A window 20 is set on the right side of the screen 11, and within this window 20, sensible feature amounts such as sharp, soft, simple, decorative, static, dynamic, modern, classical, abstract, etc. Are displayed. Further, at the lower right of the screen 11, an arbitrary feature amount can be displayed by its absolute value or offset value using a linear indicator 22 and a pointer 23. Select one of the 21
Thereafter, by moving the pointer 23 left and right on the indicator 22, the value of the feature amount of the corresponding item can be arbitrarily corrected.

Next, the example image feature correcting process (step 20)
The details of 2) are shown in FIG. In the figure, when the process is started, the process enters a state of waiting for setting of a scale used in the correction process (step 401). In this state, when the emotional feature amount is selected as the usage scale (step 401), a menu for the emotional feature item is displayed (step 402), and then a state of waiting for the selection of the feature item to be corrected is provided. (Step 40
3). That is, in the example of FIG. 6 described above, the characteristic items 14 such as softness, warmth, and brightness are displayed in the lower left portion of the screen 11, and a state in which the selection by the mouse 17 is awaited. Similarly, in the example of FIG. 7, the window 20 set on the right side of the screen 11 has features such as sharp, soft, simple, decorative, static, dynamic, modern, classic, and abstract. Item 21 is displayed, and it is in a state of waiting for one of them to be selected by the mouse 17. Note that, in the example of FIG.
Indicates a state in which "sharp" is selected.

In this state, when any characteristic item is selected (YES in step 403), a state of waiting for setting whether to use an absolute value or an offset value as a correction value is set (step 404). In this state, when an offset value is selected as a correction value, the offset value of the corresponding feature amount item is displayed (step 40).
5) The offset value input by the correction operation using the mouse is read (step 406), and a state of waiting for a predetermined correction completion operation is set (step 407 NO). In this state, if the setting completion is confirmed (step 407)
YES), the offset value read together with the correction operation is added to the feature value absolute value at that time (that is, the unique feature value of the example image) to obtain a new feature value, and the feature value of the feature item is obtained. Is modified to the desired value (step 408). The above operation is repeated until all the necessary feature item corrections are completed (step 413 YES).

FIG. 9 shows a specific example (part of the screen) of a display mode suitable for the feature amount correction processing using the offset value.
FIG. In this example, "simplicity" is displayed as the feature item 24, and an offset value is displayed on the right side thereof using the offset value indicator 28 and the hands 27 and 27 '. In this offset value indicator 28, with the offset value 0 as the center,
The offset value can be corrected in increments of 0.01 in the range of ± 0.05. At the beginning of the correction, the pointer 27 is located at the offset value 0. When a desired position is designated on the indicator 28 using a mouse, the pointer 27 is moved to the designated position as indicated by the pointer 27 '. Moves. If such a display mode is used, each feature item of the example image can be sensed without a sense of absolute value of the feature item at all and with a feeling of a normal designer such as "slightly simpler" or "slightly more complicated". Can be corrected in small increments.

Returning to the flowchart of FIG. 4, when an absolute value is selected as the correction value (step 404),
After the corresponding item is absolutely displayed (step 409), an absolute value input by a predetermined correction operation is read (step 410), and a state of waiting for a predetermined setting completion operation is entered (step 411NO). In this state, if a predetermined setting completion operation is confirmed (step 411 YES),
The input value input by the predetermined correction operation is replaced as it is with the unique characteristic amount at that time as a desired characteristic amount absolute value, whereby the example image characteristic amount correction operation using the absolute value is performed (step 412). ).

FIG. 9B shows a specific example of a display mode suitable for a feature value correction operation using an absolute value. In this example, “simplicity” is displayed as the feature item 24, and the absolute value of the feature amount of the feature item 24 is displayed on the right side thereof using the absolute value indicator 25 and the pointer 26. The absolute value indicator 25 has 0.0
Absolute value scales are provided in increments of 0.1 in the range of 1.0 to 1.0, and the feature value of the item is displayed on the absolute value indicator 25 by using a pointer 26 as an absolute value. At the beginning of the correction operation, the pointer 26 is located at the unique characteristic amount of the characteristic item (the characteristic amount inherent in the illustrated image), and the pointer 26 is moved to an arbitrary position on the absolute value indicator 25 by a cursor (not shown). It has been made movable. According to this example, at the beginning of the correction, the characteristic feature amount of the example image can be confirmed by the pointer 26. This characteristic feature is statistically obtained in light of the sensitivity of many designers or general consumers, especially when using kansei features. is there. That is, since the general public's sensibility can be known with respect to the input example image, design activities such as "simpler" or "more complex" can be easily performed according to the general sensibility.

It is needless to say that the feature amount correction processing of this embodiment can be performed also on physical feature amounts, in which case the feature items in FIGS. 6 and 7 are physical features. It is represented by linearity, contrast, complexity, etc. (step 414). That is,
Although the physical feature correction process (step 414) is not shown, the series of processes described above (steps 402 to 412) are executed for the physical feature amount. As a result, after selecting a desired item from the physical feature item menu, the feature amount can be corrected using the offset value or the absolute value.

Returning to FIG. 2, when the necessary feature quantity correction operation is completed in this way, the physical feature quantity and / or the kansei feature quantity which have been corrected or have similar features are obtained based on the modified physical feature quantity and / or kansei feature quantity. A similar image is extracted (step 203), and the extracted image is displayed on the screen of the image display device (step 204). FIG. 8 shows an example of a display screen corresponding to the result of the first search performed in this manner. This example shows the result of selecting the example image G1 on the display screen 11 of FIG. 7 as indicated by the circular ring-shaped selection mark 19. As is clear from the figure,
The example image G1 is composed of a vertical stripe pattern,
Based on the example image G1, the image is modified and slightly reduced in sharpness. As a result, as shown in FIG. 8, nine images G11 to G11 are similar to the example image feature thus modified. G19 is selected. If the target image is present in these selected images G11 to G19, the search process is completed for the time being (step 205N).
O) If nine images G11 to G19 are obtained, but there is still no favorite image (step 2)
05YES), an image re-exemplification process (step 206) is executed.

FIG. 5 shows the details of the image re-exemplification process. In the image re-exemplification process, the image selection operation from the mouse is awaited while the nine images G11 to G19 searched in the previous process are displayed (step 502N).
O). In this state, when any one of the nine images G11 to G19 is selected (step 502YE).
S) After reading out and storing the physical feature amount and the emotional feature amount of the selected image (steps 503 and 50).
4), the above-described example image feature correction processing (step 20)
2), similar image extraction processing (step 203) and search image display processing (step 204) are executed, and the search images can be further narrowed down.

As described above, according to the present embodiment, after selecting one of the external image input mode and the preset image input mode and inputting an arbitrary example image, the sensuous feature amount and the physical By selecting one of the objective features as needed, and then selecting either the offset value or the absolute value as the correction value, start with an image that roughly matches the design concept and gradually correct it Thus, the image search process is repeated while adding the image, and the image finally obtained can be efficiently searched.

[0055]

According to the first or fourth aspect of the present invention, a plurality of images are stored as registered images, and by inputting sensuous characteristics for a plurality of sample images selected from the registered images. A mapping rule that defines a correlation between the obtained kansei feature and the physical feature obtained by performing image processing on the sample image is created,
Sensitive feature amounts are associated with the registered image based on physical feature amounts based on the created mapping rule,
An example image, which exemplarily expresses the feature of the registered image desired to be searched using a drawing, is input, and the kansei feature amount or the physical feature amount of the input example image is manually corrected, Based on the feature amount of the modified example image, a registered image similar to the example image is extracted from the stored registered images based on the associated feature amount using the mapping rule. The registered image is output visually. Therefore, even when an example image that exactly matches the feature of the image desired to be searched is not available, first input an example image whose image feature roughly matches, and then add necessary feature correction to this. ,
An image matching the desired feature can be efficiently searched. Also, in the case of presenting image features by an example image representing only the basic concept, and then adding a slight variation to the example image, expanding the design concept and sequentially searching for corresponding images, By manually correcting a feature item corresponding to a slight variation portion, these search operations can be efficiently performed without making a correction to an actual example image.

In the first or fourth aspect of the present invention, if an arbitrary example image can be input from the outside using a video interface, an image scanner, a tablet, or the like, for example, a television image, a movie, a pamphlet, a handwritten figure, etc. Thus, the range of selection of the example image can be expanded.

In the first or fourth aspect of the present invention, if one example image is selected from a plurality of example images included in the example image menu prepared in advance and is input, the operator can select one of the example images. Even if an example image is not separately prepared, a desired image can be searched by selecting an arbitrary image from the example image menu. In particular, in the present invention, since the illustration features can be manually modified, the number of illustrations included in the illustration menu can be minimized as necessary to represent each design concept. Never come.

In the first or fourth aspect of the present invention, if one or more of the output images is input as a new example image for continuing the search, the example image input, the example image correction, While repeating image extraction and image output,
Images that match the designer's concept can be narrowed down more efficiently. Further, if a physical feature and / or a kansei feature is used as the feature, the type of the feature can be appropriately selected according to the design concept.

According to the second or fifth aspect of the present invention, the unique feature amount of each feature of the input example image is displayed, and the input value input by a predetermined correction operation corresponds to the corresponding feature. Recognized as the desired feature absolute value,
Since the unique feature amount of the feature is corrected to the input value and the corrected feature amount is displayed, the original feature amount of the example image is easily confirmed based on the unique feature amount. be able to. That is, for example, if a physical feature amount is used as the feature amount, it is possible to confirm the physical feature amount originally possessed by the example image. Similarly, for example, if a kansei feature is used as a feature,
It is possible to statistically confirm the emotional feature amount viewed from the general public. In particular, since this kind of sensible feature amount has a relatively large variation depending on the viewer, it is possible for the designer to know the sensibility of the general public with respect to the example image by the display of the unique feature amount, which assists the designer in his creative activities. Further, according to the second or fifth aspect, it is possible to correct the example image to the designated feature amount only for the necessary feature amount item.

According to the third or sixth aspect of the present invention, a predetermined adjustment range of the characteristic amount of each characteristic of the input example image is displayed, and an input value input by a predetermined correction operation is applicable. It is recognized as a desired offset value for the characteristic amount of the characteristic, and the value of the characteristic amount is corrected by adding the input value to the characteristic amount. The offset value is displayed in association with the predetermined adjustment range. For this reason, only the necessary feature value items are corrected within a predetermined adjustment range, for example, with relative fine adjustment instruction values such as "sharper" and "simpler" as the emotional feature values. Operations can also be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a hardware configuration of a device of the present invention.

FIG. 2 is a general flowchart schematically showing an image search process according to the present invention.

FIG. 3 is a flowchart illustrating an exemplary image input process according to the present invention.

FIG. 4 is a flowchart illustrating a feature correction process according to the present invention.

FIG. 5 is a flowchart illustrating an image re-exemplification process according to the present invention.

FIG. 6 is an explanatory diagram illustrating an example of a display screen when an example image is externally input;

FIG. 7 is an explanatory diagram illustrating an example of a display screen of an example image preset input hemorrhoid.

FIG. 8 is an explanatory diagram illustrating an example of a display screen of a first search result.

FIG. 9 is an explanatory diagram showing a specific example of a feature amount display mode in the case of absolute value correction and the case of offset value correction.

FIG. 10 is an explanatory diagram showing the interrelationship between a registered image, a sentiment feature, a physical feature, and a mapping rule.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Central processing unit 2 Display part 3 Operation part 4 Printer 5 Image storage part 6 Video interface 7 Tablet 8 Image scanner 9 System bus 10 Symbol sample 11 Screen 12 Window 13 Illustrative image 14 Feature item 15 Indicator 16 Pointer 17 Mouse 18 Window 19 Selection Mark 20 window 21 characteristic item 22 indicator 23 pointer G1 to G9 example image menu G11 to G19 searched image 24 characteristic item 25 absolute value indicator 26 pointer 27 pointer 27 offset value indicator

Continuation of the front page (56) References JP-A-8-249352 (JP, A) JP-A-8-87520 (JP, A) JP-A-4-348474 (JP, A) JP-A-4-326472 (JP) JP-A-4-246771 (JP, A) JP-A-4-241674 (JP, A) JP-A 1-239631 (JP, A) JP-A-1-209528 (JP, A) 1-189722 (JP, A) JP-A-1-189721 (JP, A) JP-A-1-188935 (JP, A) JP-A-62-226769 (JP, A) Naoki Utsuno, Toru Inohara et al. Sensitivity and Image Features ", Information Processing Society of Japan, Vol. 94, no. 7 Riki Horikoshi, Teruo Hamano, Kenichiro Ishii, "Structural Shape Retrieval System", Information Management Vol. 34, no. 7 (1991 Fusako HIRABAYASH I et al. "Information Retrieval U (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 17/30

Claims (6)

(57) [Claims] A registered image storing step of storing a plurality of images as a registered image; and a sentiment characteristic amount obtained by inputting a sentiment characteristic for a plurality of sample images selected from the registered images. A mapping rule creating step for creating a mapping rule that defines a correlation with a physical feature amount obtained by performing image processing on the sample image; and the registration based on a physical feature amount based on the created mapping rule. A feature value associating step of associating a sensual feature value with the image; an example image inputting step of inputting an example image that exemplarily expresses a feature of the image desired to be searched using a drawing; Kansei feature quantity scale characteristic of exemplary image was
Use scale arbitrarily selected from and the physical feature scale
And exemplary image feature quantity manual correction step for correcting at Lumpur, based on the feature amount of the exemplary image after the correction, from the stored image in the registration image storing step, similar to the illustrative image A similar registered image extracting step of extracting a registered image to be extracted based on the feature amount associated using the mapping rule, and a registered image outputting step of visually outputting the registered image extracted in the similar registered image extracting step In the registered image output step, on condition that the search is continued.
Search for one of one or more visually output images
Enter as a new illustration to continue searching
Image re-transfer processing to the feature amount manual correction step
And a step of manually correcting the example image feature quantity and a similar registration step.
The recording extraction step and the registered image output step
Repeat while selecting an arbitrary feature amount to extract the desired image
An image search method characterized in that it can be output. 2. The image feature manual correction step includes a unique feature amount display step of displaying a unique feature amount of each feature of the input example image, and an input value input by a predetermined correction operation. A feature value correcting step of recognizing a desired feature value absolute value of the feature and correcting a unique feature value of the feature to the input value; a corrected feature value displaying step of displaying the corrected feature value; The image retrieval method according to claim 1, comprising: 3. The example image feature manual correction step includes: an adjustment range display step of displaying a predetermined adjustment range of a unique feature amount of each of the input example image features; and an input value input by a predetermined correction operation. A characteristic value correction step of recognizing the characteristic value as a desired offset value with respect to the characteristic characteristic amount of the corresponding characteristic, and correcting the value of the characteristic characteristic amount by adding the input value to the characteristic characteristic amount. 2. The image search method according to claim 1, further comprising: displaying a corrected feature value displaying an offset value of the feature value with respect to the unique feature value in association with the predetermined adjustment range. 4. A registered image storing means for storing a plurality of images to be a registered image, and a sentiment characteristic amount obtained by inputting a sentiment characteristic for a plurality of sample images selected from the registered images. Mapping rule creating means for creating a mapping rule that defines a correlation with a physical feature amount obtained by performing image processing on the sample image; and the registration based on a physical feature amount based on the created mapping rule. A feature amount associating unit for associating a sensual feature amount with the image; an example image inputting unit for inputting an example image formed by exemplifying a feature of the image desired to be searched using a drawing; Kansei feature quantity scale characteristic of exemplary image was
Use scale arbitrarily selected from and the physical feature scale
And exemplary image feature quantity manual correction means for correcting at Lumpur, based on the feature amount of the exemplary image after the correction, from the stored image in the registration image storage means, similar to the illustrative image Registered image extraction means for extracting a registered image to be extracted based on the feature amount associated using the mapping rule, and registered image output means for visually outputting the registered image extracted by the similar registered image extraction means Visible on the registered image output means, provided that the search is continued.
Continue searching for one of the one or more images output
Enter as a new example picture for the line
Image re-exemplification means for shifting the process to the amount manual correction means;
The image feature amount manual correction means and the similar registration image
The extraction means and the registered image output means may be connected to any feature
Repeatedly to select the desired image.
An image search device, characterized in that: 5. An example image feature manual correction unit, comprising: a unique feature amount display unit for displaying a unique feature amount of each feature of an input example image; and an input value input by a predetermined correction operation. A feature value correcting unit that recognizes a desired feature value absolute value of the feature and corrects a unique feature value of the feature to the input value; a corrected feature value display unit that displays the corrected feature value; The image search device according to claim 4, comprising: 6. An example image feature manual correction unit includes: an adjustment range display unit that displays a predetermined adjustment range of a unique feature amount of each of the input example image features; and an input value input by a predetermined correction operation. Characteristic amount correcting means for recognizing the characteristic value as a desired offset value with respect to the characteristic amount of the corresponding feature, and correcting the value of the characteristic amount by adding the input value to the characteristic amount; The image search apparatus according to claim 4, further comprising: a corrected feature amount display unit configured to display an offset value of the feature amount with respect to the unique feature amount in association with the predetermined adjustment range.