KR100697339B1 - Sensory based searching system for color image and System of design simulation using thereof - Google Patents

Abstract

The present invention provides a color pattern image database constructed by matching color pattern image data with image attributes extracted from the image data. An emotional knowledge database constructed by matching an emotional evaluation value representing the emotion of the color pattern image with an image attribute corresponding to the emotional evaluation value; When the emotion evaluation value is input, the image attribute matching the input emotion evaluation value is searched in the emotional knowledge database, and the color pattern image data matching the image attribute similar to the searched image attribute is searched and presented in the color pattern image database. Emotion-based retrieval system consisting of; Emotion-based retrieval system consisting of; and by using it to apply the color pattern image retrieved based on emotion to the 3D virtual model to enable the design simulation, the user is sufficient verification through product data replacement in the virtual space And it provides an emotion-based image retrieval system and design simulation system using the same to save time and effort of product selection.

Emotion, image, visual, search, design, simulation

Description

Sensory based searching system for color image and system of design simulation using approximately}

1 is a block diagram schematically showing the configuration of an emotion-based image retrieval system according to the present invention;

2 is a flowchart illustrating an embodiment of a method of building a color pattern image database of FIG. 1;

3 is a flowchart illustrating an embodiment of a method for building an emotional knowledge database of FIG. 1;

4 is a flowchart illustrating an embodiment of an emotion-based image retrieval method using the emotion-based color pattern retrieval system shown in FIG. 1;

5 is a view showing an embodiment of an emotional knowledge database according to the present invention;

6 is a block diagram showing an embodiment of a design simulation system according to the present invention;

FIG. 7 is a flowchart illustrating an embodiment of a design simulation method using the design simulation system of FIG. 6.

8 and 9 illustrate an embodiment of a design simulation viewer to which the design simulation system of FIG. 6 is applied.

** Description of symbols for the main parts of the drawing **

10; Color pattern image database 20; Emotional Knowledge Database

30; 3D model image database 40; User interface

50; Administrator interface 100; Control

110; Viewer 120; Emotion-based Search Department

The present invention relates to an emotion-based image retrieval system and a design simulation system using the same, and in particular, an emotion-based image retrieval system for retrieving and presenting a color pattern image corresponding to an emotional adjective, and an image provided based on the emotion based on a 3D model. Emotion-based image retrieval system that provides design simulation by applying and a design simulation system using the same.

Color psychology evaluates human common feelings about a single color of average meaning through experiments. Color Image Scale, published in 1990 by Kobayashi of Japan, defined an image scale consisting of pairs of adjectives with opposite concepts, such as warm-cold, hard-soft, and so on. In addition, we arranged a single color and emotional adjectives on this image scale and associated a single color and emotional adjective. In addition, the color scheme is related to the sensitivity of color schemes, and this image scale is widely applied in the field of color design.

However, the color image scale shows only the sensitivities related to solid colors and colors, and it is insufficient to apply directly to interior products such as fabrics, wallpaper, flooring, carpets, and many other colors such as exterior materials, textiles, and cosmetics. .

In general, the content-based retrieval of an image is commonly used by using a representative representative color of the image, or by using a form or texture of an object contained in the image. However, such a search is a method of approach considering only the physical characteristics of an image, and thus the user's emotion is not considered.

Accordingly, the present invention is to solve the above problems, and to match and image the image of the color pattern and image attributes, such as MPEG-7 visual descriptor (color, texture) and the database, and to evaluate the sensitivity of the image Emotion-based image retrieval system is provided by matching emotion adjectives with image attributes and searching and presenting images with image attributes corresponding to emotional adjectives. The purpose of the present invention is to provide a design simulation system and method for directly simulating a design corresponding to an adjective.

According to an aspect of the present invention, a color pattern image database constructed by matching color pattern image data with a registered image attribute extracted from the color pattern image data, and a color pattern image data sampled from the color pattern image database. An emotional knowledge database constructed by matching an emotional evaluation value to which a color dictionary is applied and an image attribute extracted from the color pattern image, and searching for an image attribute matching an emotional adjective input by a user in the emotional knowledge database, Search for the registered image attribute similar to the image attribute in the color pattern image database using the key as a key, and search for the color pattern image data matching the retrieved registered image attribute among the color pattern image data stored in the color pattern image database. It provides an emotion-based image retrieval system comprising an emotion-based search unit to present.

According to the present invention, a color pattern image database constructed by matching color pattern image data and registered image attributes extracted from the color pattern image data is matched;
An emotional knowledge database constructed by matching an emotion evaluation value to which color scheme is applied with the color pattern image data sampled in the color pattern image database and an image attribute extracted from the color pattern image; When the user inputs an emotional adjective, the image attribute matching the input emotional adjective is searched in the emotional knowledge database, and the registered image attribute similar to the image attribute is searched in the color pattern image database using the retrieved image attribute as a key. An emotion-based retrieval unit for retrieving and presenting color pattern image data matching the registered image attribute among the color pattern image data stored in the color pattern image database; A three-dimensional model database storing a three-dimensional model image to be an object of design simulation; A viewer configured to display the 3D model image and the color pattern image; And a control unit configured to apply the color pattern image data presented by the emotion-based search unit to the 3D model image selected from the 3D model data face and display the color pattern image data through the viewer.

delete

delete

The image attribute and the registered image attribute may include any one of a visual descriptor, color, and texture of MPEG-7.

delete

The viewer may apply the color pattern image to the 3D model image and display the image by navigating, enlarging, reducing or modifying the applied image according to a user's selection.

There may be a plurality of embodiments of the present invention, hereinafter with reference to the accompanying drawings will be described in detail a preferred embodiment. This embodiment allows for a better understanding of the objects, features and advantages of the present invention.

1 is a block diagram showing an emotion-based search system according to the present invention, FIG. 2 is a flowchart showing a color pattern database construction method, FIG. 3 is a flowchart showing a emotion knowledge database construction method, and FIG. 4 is an emotion-based search method. It is a flow chart.

The emotion-based search system includes a color pattern image database 10, an emotion knowledge database 20, a controller 100, a viewer 110, a search engine 121, a user interface 40, and an administrator interface module 50. do. The color pattern image database 10, the emotional knowledge database 20, the control unit 100, the search engine 121, the user interface 40 and the administrator interface module 50 are included in the web server, and the user terminal and the administrator terminal Access to the user interface 40 and the manager interface module 50 through the network to provide a video-based emotional search service online.

The color pattern image database 10 is a database in which an image of a color pattern is matched with a registered image attribute of a color pattern and stored. Color patterns are used to color design objects such as wallpaper, paint, flooring, textiles, cosmetics, tiles, etc., and flattened colors such as painting, wallpaper, carpets, interior decoration products such as paint, color tiles, and paintings. Contains an image. The registered image attribute is an image attribute extracted by the visual descriptor of color, textures, or other image attribute extractor of MPEG-7.

As shown in FIG. 2, the color pattern image database is constructed by the following method. That is, the step of extracting the registered image attribute extracted by the visual descriptor or other image attribute extractor of MPEG-7 color and textures from the color pattern image (S11), the color pattern and the extracted registered image attribute A color pattern image database is constructed by performing matching (S12) and storing the matched color pattern and the registered image attribute (S13).

The emotional knowledge database 20 is a database in which emotional adjectives and image attributes are matched and stored. 3 shows a method of constructing the emotional knowledge database 20. As shown in the drawing, a color pattern image data sampling step (S21), a color pattern emotion evaluation step (S22), an emotion evaluation value and an image attribute matching step (S23), and a storing step (S24).

The color pattern image data sampling step S21 is to sample the color pattern image data to be equally evaluated in the image attribute space among the color pattern image data of the color pattern image database 10. In the color pattern emotion evaluation step (S22), an emotion evaluation value corresponding to the image data is obtained by performing emotion evaluation by a professional such as a designer on the sampled image data. Emotional estimates are emotional adjective image scales such as "warm," "dynamic," and "heavy." Emotion evaluation step (S22) of the color pattern image is an online evaluation that performs the process of providing the color pattern images sampled to the group of design experts connected online, the process of statistically processing and storing the emotional evaluation value input by the expert group Method can be used.

On the other hand, when a large amount of color pattern image data is designed by the design expert or the general consumer, the emotion classifier that mimics it is automatically generated, so that a large amount of image data can be automatically classified. do. For example, an image similar to the feature vector extracted from the color pattern image determined to be elegant may be automatically classified into an elegant image later.

In the emotional evaluation value and the image attribute matching step (S23), the image attributes and the emotional evaluation value matched with the color pattern on which the emotional evaluation is performed are matched. The storing step (S24) is to build the emotional knowledge database by storing the matched image attribute and the emotional evaluation value.

The relationship between the image data of the color pattern and the registered image attribute is constructed in the color pattern image database 10, and the relationship between the emotion evaluation value and the image attribute is constructed in the emotional knowledge database 20. Here, the emotional evaluation value is composed of adjectives expressing emotion, and the image attribute is composed of visual presenters such as color or texture of MPEG-7.

The controller 100 performs a search including a viewer 110 displaying a color pattern for searching an emotion-based image and a search engine used for the search. User interface (I / F) 40 provides for interfacing with a user. For example, an emotional adjective is input from the user and transmitted to the control unit 100. The user interface 40 is connected through a network with a user terminal (not shown in the drawing) including an input unit for inputting an emotional adjective for a search and a display unit for outputting a search result.

The search method using the emotion-based search system configured as described above is illustrated in FIG. 4. In the emotion-based image retrieval, when a user inputs an emotional adjective, the image of the emotion corresponding to the input adjective is retrieved and presented.

First, when an emotional adjective is input from the user, the emotional adjective is input to the user interface 40, and the user interface 40 transmits the input emotional adjective to the controller 100 (S31). The search engine 121 expands the input adjectives into adjectives included in the emotional knowledge database 20 (S32), and searches the image attributes matched with the adjectives using the emotional knowledge database 20 (S33). In operation S33, the color pattern image database 10 matching the registered image attribute whose image attribute and similarity is equal to or greater than the set value is retrieved from the color pattern image database 10 (S34). The image of the color pattern corresponding to the search result is displayed through the viewer 110.

The adjective query expansion in step S32 is a process of converting adjectives input by a user into adjectives used to build an emotional knowledge database. A hierarchical or association relationship between adjectives is used. It may be configured in a thesaurus form. Thesaurus is a kind of index (software) for performing machine search of information, which is the main task of computer, and the role of thesaurus is to first group similar words and limit the meaning of each term. By standardizing it, it is possible to unify the concept in the search and to allow anyone to search the same order and content. If the same adjectives as the adjectives used to build the emotional knowledge database are entered by the user, the query expansion step is skipped, and the adjective query expansion is also required when the user is presented with a limited number of adjectives and the user selects among the suggested adjectives. Step S32 can be omitted.

 In the emotional knowledge database 20, since a relationship between the emotional space corresponding to the adjective constituting the query input and the image attribute is established, the emotional knowledge database 20 utilizes this to find the corresponding image attribute. The searched image attribute is used to search the color pattern by serving as a search key for searching the corresponding image of the color pattern image database. Through this process, the searched color pattern corresponding to the emotional input adjective that is input to the query is presented to the user.

If the search engine 121 includes a multi-dimensional index such as a CS-tree, when the color pattern image data of the color pattern image database 10 is tens of thousands, the search engine 121 may add the CS-tree to the search engine 121 for the speed of the search. It can be used to find images by searching for multidimensional indexes.If the multidimensional index structure is not included, the number of images is searched starting from the image having the largest similarity by comparing the similarity with the registered image attribute of the color pattern image database 10. Present it to the user. CS-tree is a large-scale database retrieval technology, which is an image high-dimensional index storage structure that searches for hundreds of thousands of images within one to two seconds by linking with an existing database.

6 and 7 illustrate a design simulation system and a simulation method using the emotion-based retrieval system.

The design simulation system according to the present invention selects an object to be color designed using the 3D model database 30 in which an image of a 3D model, which is an object of design, is stored in a web environment, and simulates a design using a color pattern presented by emotion-based search. It also enables color schemes to verify the design results of objects through zooming in, zooming out, colored images, navigation, and corrections.

As shown, the design simulation system uses an emotion-based retrieval system, and includes a 3D model database 30 in which a 3D image to be colored is stored, and the controller 100 performs a retrieval function and a control function. The controller 100, the database 10, 20, 30, the user interface 40, and the manager interface module 50 are configured as a web server, and the user terminal and the manager terminal are connected to the user interface 40 and the administrator through a network. It is connected to the interface module 50 and made available in the online environment.

The 3D model database 30 stores three-dimensional modeled images of coloring objects and is three-dimensional virtual objects that are modeled and stored using Virtual Reality Markup Language. Such 3D objects may include a model house of an apartment, an indoor space such as an office, or an outdoor space such as a large apartment complex and a performance hall, a human body to which clothing and cosmetics are applied.

The user terminal is connected to the user interface 40 through the network to receive the emotion-based search service and the design simulation service online, and the administrator terminal is also connected to the manager interface module 50 through the network to perform database construction and management online. It can be done with The manager interface module 50 may be configured to sample the user's restrictions or the emotional knowledge database, to manage the MPEG-7 visual presenter and other image attribute extractors, and to construct the color pattern image database. Play the administrative role of

The viewer 110 is for display of the modeled 3D virtual object and the retrieved color patterns. By using the viewer 110, a searched color pattern may be applied, modified, enlarged, reduced, and navigated in the 3D virtual object.

The operation state of the design simulation system and the implemented screen will be described with reference to FIGS. 6 to 9. FIG. 8 is a diagram illustrating a searched color pattern and a model image applied thereto when a user inputs an emotional adjective called 'elegant', and FIG. 9 illustrates a model image in which a design simulation is performed by inputting an emotional adjective called 'mild'. Drawing.

First, when a user inputs a selection signal of an object image to be color-designed, such as a living room illustrated in FIGS. 8 and 9, in the 3D model database 30 through the user interface 40 (S41), through the viewer 110. The target image is displayed on the monitor (S42). The user may select an object to be colored in the image, for example, a wallpaper, a carpet, etc. The viewer 110 has an emotional adjective input window at one side, and the user inputs an emotional adjective to search for the color pattern based on the emotion (S43). .

The emotion-based retrieval system retrieves the color pattern of emotion corresponding to the emotional adjective input by the above-described procedure (S44). The retrieved color pattern is displayed through the viewer (S45). When the user inputs the color pattern selection signal through the user interface 40 (S46), the selected color pattern is applied to the model image displayed on the viewer 110 (S47). This process is displayed on the monitor of the user terminal by the viewer 110, occurs interactively (color), and color design simulation is performed.

Therefore, according to the present invention, by searching for and presenting a color pattern image based on emotion using the emotional property of the product image, the user can easily find a color pattern of a desired emotion, and directly to the desired model image. By applying it to design decisions.

In addition, through the use of the present invention, it is possible to automatically evaluate color images such as paintings and photographs, and the design simulation system recommends a product according to the user's target sensitivity in a 3D virtual space, such as indoors and outdoors, to complete a design. As a decision support system, it is possible to induce a safe purchase of a product through electronic commerce, etc. In the producer side, there is an advantage that a production plan of a product can be established through an analysis of sentiment trends of a sold product.

Hereinafter, an embodiment of a design simulation system according to the present invention will be described. The emotional knowledge database 20 uses a color dictionary of color combinations published by the IRI color research institute, uses a dominant color descriptor as a visual representation of MPEG-7, and shows an embodiment of wallpaper targeting an image of FIG. 8. It demonstrates with reference. The system is built on a JSP web server environment, and the database uses MS-SQL.

The Color Dictionary of the IRI Color Research Institute is a book titled "Color Combination" invented by the IRI Color Research Institute in Korea. It is a color dictionary of 14 adjective images. This book lists 110 color and 10 colorless colors with 10 tones and 11 colors in the Hue-Tone system, based on the adjective image. . The emotional adjectives listed in the emotional image space, expressed as "dynamic-static", "soft-hard", etc. in the color dictionary, are "cute", "light", "casual", "dynamic", "modern" "," Clear "," moderate "," natural "," smooth "," elegant "," gorgeous "," old-fashioned "," noble "," major ", etc., each representing the atmosphere of the adjective The color scheme was composed of combinations of the 120 colors described above.

In the system according to the embodiment, an emotional knowledge database was constructed by storing 14 adjectives and color schemes representing them using MS-SQL. In other words, if the emotional evaluation of 14 adjectives and 120 (R, G, B) values of 120 colors are used in the book, the emotional evaluation of the product images sampled from the color pattern image database is unnecessary. The step is omitted.

 Figure 5 shows the contents of the emotional knowledge database built in the administrator interface module consisting of JSP and MS-SQL of the Web server. 8 to 9 illustrate an embodiment using a color pattern image database for wallpaper. In the illustrated wallpaper image database, 500 wallpaper images are stored, and since the number of wallpaper images is not large, the multi-dimensional index structure using image attributes is not used in the emotion-based search. In the case of using a product image database of tens of thousands of wallpaper, a multi-dimensional index structure using image attributes such as CS-tree is needed to speed up the search. Each image of the color pattern image database 10 for wallpaper has a dominant color descriptor value of MPEG-7 as a field element. In addition, if the 14 adjectives from the Color Combination color scheme of the IRI Color Lab are used as they are, the step of expanding the adjective query can be regarded as an identity function, and thus the search engine is related to the adjectives that are the result of the emotional knowledge database. It is responsible for finding the wallpaper images in the color pattern image database 10 that most closely match the color scheme that is the dominant color descriptor. The similarity measure used here is Euclidean distance.

The 3D model database 30 is constructed using VRML, and the viewer 110 uses CORTONA. Since the color matching dictionary of the IRI color research institute is used as an emotional knowledge database, the user selects and queries one of 14 adjectives, and the selected adjective is assigned to have the corresponding emotion from the emotional knowledge database 20 in the emotion-based search unit 120. Extract the (R, G, B) values of the combination. These are displayed on the screen of the user terminal in the corresponding color with the area ratio, and furthermore, in the color pattern image database 10, wallpaper images having a similarity or more than a setting range in terms of MPEG-7 dominant color descriptor are presented. When the user selects one of the presented wallpaper images, the wallpaper is applied and displayed on the 3D model selected from the 3D model database.

As shown in FIG. 8, when the user selects "elegant", ideal color combinations searched from the emotional knowledge database are displayed at the upper left of the screen, and the selected color is at the top, and the wallpaper change, floor change, carpet The images displayed in the change are extracted from the wallpaper image database and present the image having the dominant color descriptor most similar to the selected color scheme. The longitudinal samples of the wallpaper, floor, and carpet represent priorities in terms of color selection and similarity, and display the situation in which the interiors of the leftmost samples taken and dressed are applied to the VRML virtual space.

Fig. 9 shows the completed interior space when the "mild" atmosphere is selected. The design simulation of these indoor spaces is constructed using the exemplary system invented.

The system of the present invention can be mixed with the content-based image retrieval system if the attribute of the product image is used using the MPEG-7 visual presenter. In other words, it can efficiently process queries such as image search similar to the product image selected in the emotion-based search. Since the color pattern image database is composed of the product image and the corresponding MPEG-7 visual expression, the image database of the content-based search Therefore, if the search engine 121 of FIG. 1 performs a function of simultaneously processing an emotion-based search and a content-based search, and provides a search interface that can accommodate all of them, There is a feature that can perform a content-based search.

The present invention performs emotion evaluation based on MPEG-7 Visual Descriptor and product image attributes expressing other colors and textures, and selects products based on emotion in electronic commerce.

Even if you are not a color design expert, you can use design simulation system to complete the design using the target products of space with enough sensitivity.

This system can induce safe purchases because users can directly apply and select products in a web environment, etc., which can contribute to the activation of e-commerce of these products. There is an advantage to planning.

In addition, the present invention can be used for automatic evaluation of color images such as paintings and photographs, and since it utilizes MPEG-7 visual presenter, it can be efficiently integrated with the contents-based image retrieval system that MPEG-7 is aimed at. Can be performed.

The design simulation system according to the present invention enables customized remodeling for model house displays and consumer preferences, can provide online simulation for customer service for customized apartments, and online / offline customer consultation of furniture makers. And suggestions. In addition, it is possible to predict demand by product through analysis of customer sentiment tendency such as cosmetics and fashion, and can be used as a product specification specification tool for online purchase between textile companies.

As described in detail above, the present invention searches for a color pattern such as wallpaper, flooring, etc. of a database corresponding to an emotional adjective, displays the searched products for user verification, and selects a product that matches the user's preference among the searched color patterns. By providing the user with the ability to apply and navigate to the 3D virtual space model, the user can fully verify through product data replacement in the virtual space and save time and effort in product selection.

In addition, the user can easily search the wallpaper and flooring material that the customer wants by using the system without going directly to the store. Confidence in the promotion will promote reasonable purchases.

In addition, the emotion-based search system and design simulation system according to the present invention can be applied to various products such as wallpaper, flooring, doors, carpets, textiles, and cosmetics, and when combined with offline, the market can be further activated.

In the case of using the design simulation system according to the present invention, the promotional cost is greatly reduced due to the effect of replacing a catalog, and unlike the catalog method, new products can be easily added, thereby promptly promoting new products. In other words, simply input the new product image into the color pattern image database, and the classification is automatically performed in the system according to the expert's sensitivity evaluation, so that the new product can be immediately searched through the emotion-based search system.

In addition, by using the search system and simulation system according to the present invention, it is possible to predict the change in sentiment trend of product demand, so that the company prepares and releases a product production plan suitable for this, thereby reducing inventory and increasing sales.

Claims (8)

A color pattern image database constructed by matching color pattern image data with registered image attributes extracted from the color pattern image data; An emotional knowledge database constructed by matching an emotion evaluation value to which color scheme is applied with the color pattern image data sampled in the color pattern image database and an image attribute extracted from the color pattern image; Search for the image attribute matching the emotional adjective input by the user in the emotional knowledge database, search for the registered image attribute similar to the image attribute in the color pattern image database using the retrieved image attribute as a key, and the color pattern image. And an emotion-based searcher for searching for and presenting color pattern image data matching the searched registered image attribute among color pattern image data stored in a database. delete delete The method of claim 1, And the image attribute and the registered image attribute comprise any one of a visual descriptor, color, and texture of MPEG-7. delete A color pattern image database constructed by matching color pattern image data with registered image attributes extracted from the color pattern image data; An emotional knowledge database constructed by matching an emotion evaluation value to which color scheme is applied with the color pattern image data sampled in the color pattern image database and an image attribute extracted from the color pattern image; When the user inputs an emotional adjective, the image attribute matching the input emotional adjective is searched in the emotional knowledge database, and the registered image attribute similar to the image attribute is searched in the color pattern image database using the retrieved image attribute as a key. An emotion-based retrieval unit for retrieving and presenting color pattern image data matching the registered image attribute among the color pattern image data stored in the color pattern image database; A three-dimensional model database storing a three-dimensional model image to be an object of design simulation; A viewer configured to display the 3D model image and the color pattern image; And a controller configured to apply the color pattern image data presented by the emotion-based search unit to the 3D model image selected from the 3D model data surface and to display the color pattern image data through the viewer. The method of claim 6, The viewer applies the color pattern image to the 3D model image, and design, navigation, enlargement, reduction, or modify the displayed image according to the user's selection design display system. In the emotion-based image retrieval method using the emotion-based image retrieval system of claim 1, Inputting an emotional adjective by the user; Expanding the input adjective into an adjective included in the emotional knowledge database; Retrieving the stored image attribute matching the input adjective using the emotional knowledge database; Retrieving a similar registered image attribute from a color pattern image database using the retrieved image attribute as a key; Retrieving color pattern image data stored in accordance with the retrieved registered image attribute from the color pattern image database; And And displaying an image of a color pattern corresponding to the search result.

Images