KR20020083071A - Emotion-Based Color Pattern Evaluation And Retrievals - Google Patents

Abstract

PURPOSE: A device and method for evaluating a color pattern are provided to approximate a non-linear relation between a physical attribute which is obtained through a video analysis of a color pattern and a sensitivity evaluation value of a sensitivity space comprising a pair of adjectives having different meanings therewith by applying an artificial neuron and an adaptive fuzzy system to a color pattern evaluation and analysis. CONSTITUTION: A non-linear converter for receiving a physical attribute extracted through a video analyzing process of a color video or a color pattern and extracting an output corresponded to a predetermined number of sensitivity spaces comprising a pair of adjectives having opposite meanings interactively which is corresponded to a feeling of a human being uses an artificial neuron structure. Elements of the artificial neuron structure are described as follows. An input layer consists of the number of calculation units corresponded to the physical attribute. An output layer consists of the number of calculation units corresponded to the sensitivity space. A concealment layer consists of a predetermined number of calculation units for connecting the input layer and the output layer thereto and combining input from the input layer in non-linearly.

Description

Color Pattern Evaluation Apparatus, Evaluation Method and Storage / Retrieval System Using Emotion Language {Emotion-Based Color Pattern Evaluation And Retrievals}

The present invention relates to color pattern evaluation and retrieval, and more particularly, an apparatus for performing an automatic evaluation of an emotional side so as to be close to a human being seeing and feeling a color pattern, and using the result to retrieve a color pattern of a database through an emotional language; It is about a method.

We live in a variety of color environments, and color environments have a lot of influence on human emotions. Such a relationship between color and human emotion can be found around our daily lives. Especially, in the fields of graphic design, fashion design, product design, interior design, urban color planning, the use of colors suitable for human emotion is important. have. Thus, how color affects human emotion has been an important research field in color psychology.

The feelings that a person sees and feels in color vary depending on the person's personality and may depend not only on the age, gender, or region, but also on the condition of the person's environment and mood. Therefore, color psychology evaluates human common feelings about a single color of average meaning through experiments. In Japan, Kobayashi published 'Kodansha' through 'Color Image Scale' published by Kohdansha in 1990. We defined an image scale consisting of pairs of adjectives with opposite concepts such as 'warm-cold', 'hard-soft', and so on. In addition, we arranged a single color and emotional adjectives on this image scale and associated a single color and emotional adjective. In addition, the color scheme is also related to the emotion of color scheme, and this image scale is widely applied in the field of color design.

However, such contents are merely related to the emotions related to a single color or a simple color scheme, and are insufficient to be directly applied to a color pattern in which a lot of colors used for fabrics or wallpaper, such as cloth, are randomly mixed.

Meanwhile, among many studies and attempts to classify and search such color patterns according to the human feeling, there is a research by Soen, for example, Soen. , T.) and Kawamoto, N., et al., 1993, Color Res. Appl. The "Objective Evaluation of Color Design II" article, published on pages 260-266 of Vol. 18, and by Sean (T.), Shimada (T.), and Akita (M), was published in 1987 by Color Res. Appl. In "Objective Evaluation of Color Design," published on pages 187-194 of Vol. 12, I attempted to analyze emotions about color patterns and to objectively evaluate the emotions of seeing and feeling color patterns.

In the above paper, Soen et al. Selected 13 image scales for the feeling of seeing and feeling color patterns. Selected image scales are 'like-dislike', 'beautiful-ugly', 'natural-unnatural', 'dynamic-static' ',' Warm-cold ',' gorgeous-gay ',' cheerful-dismal ',' unstable-stable ',' bright-dark ' 'light-dark', 'strong-weak', 'gaudy-plain', 'hard-soft', 'heavy-light' light) 'consists of 13 pairs of adjectives with opposite meanings. They generated 30 random color patterns, examined each random color pattern through the test subjects, and measured emotional evaluation values for 13 image scales.

According to the measurement result through the experiment, it is understood that the sensitivity evaluation value of the color pattern has the meaning of the average of variance because the emotion that a human sees and feels a color pattern is different for each person. Soen et al. Analyzed the results of the above experiments and found that the average color tone and luminance of the color pattern and the spatial frequency components of the color pattern are closely related to these images. Based on the experimental results, an objective emotional evaluation system was proposed to define the relationship between the energy distribution and the emotional effects of each band in the frequency spectrum for color patterns.

In general, content-based retrieval of an image is commonly used by using a representative representative color of the image, or by using a form or texture of an object contained in the image. However, such a search is approaching only considering the physical characteristics of the image, and thus the user's emotion is not considered. Therefore, as will be described later, it may be applied to a retrieval system for searching for color patterns based on emotions using the emotional properties of the color patterns obtained through the emotion evaluation system according to the present invention.

Retrieval of color images using emotion has been attempted by Hachimura as described in "Retrieval of Paintings Using Principal Color Information," published in Proceedings of ICPR'96dml, pages 130-134, 1996. The three primary colors were found and set to three colors, and a database of color images was constructed by matching the color image scale proposed by Nippon Color and Design Research Institute (NCD).

In the color image retrieval of the system proposed by Hachimura, when a desired emotional language is selected on a language image scale, a method of retrieving a color image using the three color schemes by searching for three color schemes on a color image scale corresponding to the emotional language is provided. Suggested. However, it only connects the three colors of the image and the emotional language, and does not consider the image properties of color patterns with various textures.

On the other hand, in order to implement a system for emotionally evaluating color patterns, data on how the emotions that people see and feel in color patterns is needed. However, Soen et al. Have the subjects display the degree of emotion they see and feel in color patterns, analyze the results, and propose an evaluation system based on nonlinear equations based on the results, and multiple regression analysis. The model parameters were obtained using the analysis).

Physical properties of the color pattern are the average brightness, the first component of the average color and the second component of the average color obtained through the image analysis process And D _L , D _M , and D _H , which are low, medium, and high frequency components of an image, and emotional space means adjective spaces of opposite meanings, such as "warm-cold."Sean's relationship was obtained through the psychological experiment, that is, experiments with people, and the relationship between the physical properties of the color image or color pattern and the evaluation values of various people in the emotional space. Got it.

However, according to the contents of such a table, this relationship is not suitable for making a linear equation and it has to be modeled as a nonlinear equation.

Here, nonlinear approximation of the experimental results is very important. Psychological experimental results can be approximated by nonlinear functions. If you have such a function, you need to find the physical property in any color image or color pattern and input it into the function. Approximate emotional evaluation is possible.

However, Sean attempted modeling with the statistical method of 'multiple regression analysis' as the method, and this nonlinear approximation equation is shown in Equation 1 below.

In Equation 1, i denotes the i-th emotional space or emotional scale by the pair of emotional adjectives, and Pi denotes the psychological value of the i- th scale among the 13 scales. And c _ik are obtained by multiple regression analysis using data obtained through psychological experiments.

That is, Equation 1 above means that 14 c values are found to approximate one emotional space, which means that 13 * 14 parameters to be found through training and stored for evaluation.

However, the problem with this prior art is that the image scales used in the experiments are highly correlated with each other, and the evaluation system presented by the system is too simple to assume the human view of the color pattern as a linear system. There is a problem that does not fit well.

An object of the present invention was devised to solve the shortcomings of the color pattern evaluation system according to the results of Sean et al., And the color pattern evaluation of an artificial neural network and an adaptive purge system which is known to have excellent approximation ability of a nonlinear function. Color pattern sensitivity evaluation to better approximate the non-linear relationship between the physical properties that can be obtained through image analysis of color patterns and emotional evaluation values of emotional spaces composed of adjective pairs with opposite meanings. Apparatus, a method for automating the emotional evaluation of color patterns using this evaluation apparatus, a system for evaluating and indexing color patterns with this automated evaluation system, and a system for retrieving the corresponding color patterns using emotional adjectives from stored color patterns To provide.

1 is a schematic diagram of an emotional evaluation and storage / retrieval system of a color pattern according to the present invention;

2 is a schematic explanatory diagram of an artificial neural network structure employed in the present invention;

3 is a schematic structural diagram of an adaptive fuzzy logic system according to the present invention;

4 is an explanatory diagram illustrating a color pattern search process for a language query according to the present invention;

5 is an exemplary diagram of rank modifier positions on an image scale used in a language query;

6 is an explanatory diagram for explaining a membership function for each grade modifier;

7 is a query showing a neural network system according to the present invention and the output result thereof,

8 is a diagram showing the result after inputting the same query as in FIG. 7 to the adaptive fuzzy logic system according to the present invention;

9 is a diagram showing a result of outputting another query to the neural network system according to the present invention;

FIG. 10 is a diagram illustrating an output result after inputting the same query as in FIG. 9 into the adaptive fuzzy logic system according to the present invention; FIG.

11 is a view showing a color pattern image recommended through the emotional evaluation of the neural network system according to the present invention for any query,

12 is a view showing a color pattern image recommended through the emotional evaluation of the adaptive fuzzy logic system according to the present invention for the same query as in FIG.

According to a preferred embodiment of the present invention, a predetermined number of emotions are composed of adjective pairs having opposite meanings corresponding to human emotions as input as physical properties extracted through image analysis of a color image or a color pattern. A nonlinear transducer is provided which extracts an output corresponding to a space, wherein the nonlinear transducer,

An input layer composed of a number of computational units corresponding to the physical attribute value, an output layer composed of arithmetic units corresponding to the number of emotional spaces, and input from the input layer by connecting each of the input and output layers together It is characterized by using an artificial neural network structure consisting of a hidden layer consisting of a predetermined number of computing units for combining non-linearly.

The nonlinear transducer of the artificial neural network structure determines the weights between units by learning. The learning uses an error back propagation algorithm, and the data used for learning is represented by C in Equation 1 The same data as the data when determining the value were used.

According to another suitable embodiment according to the present invention, a predetermined number of adjective pairs having opposite meanings corresponding to human emotions as physical inputs extracted through the image analysis process of a color image or a color pattern as an input A nonlinear transducer is provided for extracting a color image or a color pattern corresponding to the emotional space of the transducer.

The computation function is represented by a rule corresponding to an adaptive fuzzy logic system used for adaptive rule-based control, respectively, and the fuzzy set assumes a Gaussian function, and the average color (Va) and the average saturation (Vb) An adaptive fuzzy logic system composed of a first fuzzy logic system having an attribute and a second fuzzy logic system having an average brightness (I) and a texture (C) of a color pattern as the physical attributes: and an output of the two systems In order to output the emotional space is defined in the form of a hybrid operation of the intersection and sum operation -Characterized by having a model.

These two adaptive fuzzy systems and The parameters that define the fuzzy set and Gaussian-type fuzzy set in the model-based nonlinear transducer are also determined by learning. The learning method uses gradient descent. The same data as the data for determining the c value of 1] were used.

According to another suitable embodiment of the present invention, a color image or a color pattern is compared with the learned artificial neural network or the learned adaptive purge system. A color pattern evaluator which evaluates to correspond to human emotions through nonlinear transformation means of the model; A memory unit for indexing and storing a color image or a color pattern according to a predetermined rule by using the evaluation value evaluated by the evaluation unit; And a search and extraction unit for searching the memory unit in response to a query expressed in an adjective language and extracting a corresponding color image or a color pattern.

Further, according to another embodiment of the present invention, a first procedure of extracting a physical property through the image analysis process of a color image or color pattern; Inputting the extracted physical property into an artificial neural network or an adaptive purge system to perform emotional evaluation of a color image or a color pattern in a predetermined number of emotional spaces having predetermined adjective pairs having opposite meanings corresponding to human emotions. Second procedure; A third procedure of storing the extracted color image or color pattern in a recordable and readable memory means according to a predetermined rule; A fourth procedure of receiving a query for a color image expressed in an adjective language from the outside and converting the query into a numerical value to obtain a numerical vector corresponding to the adjective language; And a program for executing a fifth procedure of searching for a color image or color pattern stored in the memory means according to the numerical vector obtained through the fourth procedure and outputting a color image corresponding to the input. A record carrier is provided.

The present invention will now be described in detail by way of example only with reference to the accompanying drawings. The following description may more clearly understand the above features and other advantages of the present invention. In the detailed description of the present invention, a detailed description of well-known technology that may unnecessarily obscure the subject matter of the present invention will be omitted.

Again, the present invention relates to a color pattern emotion evaluation device for evaluating the degree of feeling of seeing and feeling the color pattern based on the psychological experiment results of Soen et al., And the non-linear emotion evaluation device. The present invention relates to a method for automating emotion evaluation and a system for indexing color patterns, storing color patterns with dichroic indexes, and retrieving color patterns that are appropriate for queries composed of adjectives.

In other words, in the color pattern evaluation system according to the present invention, two nonlinear relationships can be approximated between the physical properties obtained through the image analysis of the color pattern and the emotional evaluation values of the emotional spaces of the adjective pairs having opposite meanings. A branch evaluation system is used, one of which is modeled after the pseudo-neural network, and the other is modeled by the adaptive fuzzy logic system.

As mentioned above, the pseudo-neural networks and adaptive fuzzy logic systems are widely known for their ability to approximate nonlinear functions.

As will be described later, the emotional evaluation of the color pattern through the color pattern retrieval apparatus, method and system according to the present invention employing the two models are all non-linear by learning the psychological experimental result data than the color pattern evaluation method proposed by Sean et al. Better performance in approximating functions.

The inventor of the present invention, "A-TREE: A Dynamic Index Structure For Spatial," published in 1984 by Anotonin Guttman, which is widely used in indexing multidimensional spatial data for efficient indexing of color pattern data. "R-Tree" technique, as described in "Searching", was used.

As will be described later, the system according to the present invention may provide a user interface to be utilized for product selection using emotion in electronic commerce sites such as textiles, wallpaper, interiors and the like on the Internet.

Emotion evaluation system and fuzzy logic system of color packer using neural network, two evaluation system models used in the present invention; -Briefly explain each of the color evaluation system using color model.

(1) Color pattern emotional evaluation system using neural network

In the neural network model of multilayer feed-forward in the emotion evaluation system according to the present application, the six physical properties used by Sean et al. For comparison with Sean et al. And D _L , D _M , D _H were used as they are.

In the model using the pseudo neural network, the six inputs are combined nonlinearly through the hidden layer, so that the effects of not only the independent characteristics of the six characteristics but also the comprehensive characteristics of each combination of terms can be considered. have. The neural network structure according to the preferred embodiment of the present invention is composed of an input layer of six units, a hidden layer of five units, and an output layer of thirteen units, as shown in FIG.

Here the input is like Sean's system And D _L , D _M , and D _H , and the outputs are psychological values for 13 measures. Neural network learning is performed using an error back-propagation algorithm, and the data used for learning is the same as the data used by Sean in multiple regression analysis.

On the other hand, the artificial neural network used in the present invention has a structure similar to the human neural network, and the three-layer structure formed as shown in Figure 2, the input layer (input) in the form of a group of 'nodes' that are non-linear computing units This is because it was proved by the results of a mathematician, Kormogroff, that three layers, a hidden layer and an output layer, are sufficient for nonlinear approximation.

(2) fuzzy logic system -Emotion Evaluation System of Color Pattern Using Model

In another evaluation system according to the present invention, an adaptive fuzzy logic system is used. Unlike an evaluation system using a neural network, an evaluation system using a fuzzy logic system model shows physical properties obtained through image analysis of color patterns. It was defined differently from the case of et al.

The psychological experiments of Sean et al. Reported that the factors affecting the emotional evaluation were average color and saturation, average brightness, and texture of pattern. That is, it can be modeled with the following rules.

"If the average color tone is near red, it feels 'warm'."

"When the average color tone is near blue, it feels 'light'."

"A high average brightness gives you a 'light' feel."

Therefore, it is possible to assume average color, average saturation, average brightness, and texture of the pattern as input to the system.

On the other hand, the average brightness can be obtained by the average value of (R + G + B) / 3 of each pixel, and the average saturation, average color is similar to the color (H), saturation (S) and R, G, B color It can be calculated by taking the average over all the pixels by changing the brightness (I) color coordinates.

That is, in a preferred embodiment of the present invention, since the color tone of the color pattern cannot be expressed in the RGB space, the average value of each of red, green, and blue is converted into an HSI space representing hue, brightness, and saturation. It was. Thus, the HS plane of the HSI space corresponds to a hexagon representing phase and saturation. However, since the polar plane represented by the angle and the size of the HS plane is displayed, the color tone is expressed by converting V _a and V _b into a rectangular coordinate system as shown in Equation 2 below.

Here, 0 ° ≤ H ≤ 360 °, and 0 ≤ S ≤ 1, so 0 ≤≤1. Average brightness after HSI coordinate conversion I value was used as it is. The texture characteristic C was obtained from the gray level run-length matrix (GLRLM) of the brightness image. The GLRLM is a technical term known as a texture property for image analysis, and a detailed description thereof will be omitted. Meanwhile, the GLRLM of the brightness image is defined as in Equation 3 below.

Where _MN is the lattice of M x N, and I (i, j) is the pixel value at the (i, j) position in the lattice. τ (m, θ) corresponds to the run-length of the pixel value m in the θ direction, and Card is the cardinality of the set. In a suitable embodiment of the present invention, the GLRLM values for 0 °, 45 °, 90 °, and 135 ° were arithmetic averaged to eliminate the directional characteristics of the GLRLM and normalized so that the sum of all the run-lengths of the GLRLM was 1. Finally, the texture property C was used as Equation 4 below, which is a scalar measure of GLRLM.

That is, C = SRE. The fuzzy IF-THEN rule for obtaining psychological evaluation can be set as follows. [Equation 4] is also known to be used to extract the texture in the image analysis field, the evaluation system according to the present invention is employed to use to express the rough and smooth properties of the image.

On the other hand, the fuzzy rule base for V _a and V _b in the i-th image scale can be set as shown in [Equation 5] below.

For C can be expressed as shown in Equation 6 below.

Here, EV ₁ ⁱ and EV ₂ ⁱ are evaluation values for the i-th image scale. For the above two fuzzy rule bases, a fuzzy logic system with V _a and V _b as input and a fuzzy logic system with I and C as input are constructed, and the outputs of both systems are Krishnapuram (R.) and Lee (Lee). , J.), as described in Fuzzy-Set-Based Hierarchical Networks for Information Fusion in Computer Vision, 1992, pages 335-350 of Neural Networks Vol. It is fused by the model. The entire system is represented by an evaluation block diagram as shown in FIG. 3.

Rule configuration according to the present invention was considered as a method of constructing through learning depending on the measured data. The measured data used psychological evaluation values of 30 random color patterns such as Sean and V _a , V _b , I and C values of each random color pattern as input. In the first type of logic system with V _a and V _b as inputs, the number of rules is 7, and in the second type with I and C as inputs, the number of rules is 7.

Here, the number of rules means the number of rules as shown in [Equation 5] and [Equation 6]. In the adaptive purge logic system, the number of rules is fixed, but [Equation 5] and [Equation 5] 6] and "positive" in rules. The membership function defining fuzzy sets or terms such as "negative" and "zero" is not determined but determined by training, such as determining the connection strength of nodes in an artificial neural network. The M value of Equation 7 below The same meaning as the number of rules, which means that each of the two AFC blocks shown in FIG. 3 is seven. In Equation 7 below, the value of n is 2 for each AFC block of FIG. 3 because two V _a and V _b are input and two I and C are input.

Each fuzzy logic system may be expressed as Equation 7 below.

here Silver membership function Is the maximum value Assume Also, Is an adjustable parameter, and when the membership function is defined as [Equation 8], which is a Gaussian function, it is a parameter expressing maximum value, average, and variance.

The model appears earlier in the literature by Krishnapuram, R., Lee, J., and is a form of hybrid operator of cross and sum operations devised by Zimmermann and Zysno. (9).

Here, x _i has a value between 0 and 1, and δ _i represents a weight applied to the input value x _i . And, Is a parameter representing the degree of compensation between the sum operation and the intersection part. Considering the entire system as shown in FIG. 3, the parameter update of Equations 7 and 9 is performed through a "Gradient Descent" training method.

Meanwhile, according to the present invention Model Depending on the value, it is used as a union operator, intersection o [erator, and compensation operator, etc. The model determines the final estimate by combining the estimates according to average hue V _a , V _b and the estimates according to average brightness and texture I, C by appropriate calculations.

Next, with regard to data indexing and storage, the emotional evaluation value of the color pattern obtained in the evaluation system according to the present application is used to index the color pattern and store it in the color pattern database. The evaluation system uses the auto-evaluated emotional attribute for indexing the database. Nine of the 13 outputs of the evaluation system were used instead of the emotional attribute.

Excluded emotional attributes are those of four image scales: 'like-dislike', 'beautiful-ugly', 'natural-unnatural', and 'unstable-stable'. The reason for this is that the first 13 emotional attributes are used to reduce the dimension, and the excluded attributes are 'dislike', 'ugly', 'that you want the user's query to be generally' like ',' beautiful 'and' natural '. Because you don't want an unnatural pattern. Also, 'stable' is omitted because it is similar in meaning to 'static' and 'plain'.

As a result, each color pattern in the present invention provides an evaluation value in the form of a 9-dimensional vector in an emotional space composed of nine adjective pairs having opposite meanings, and these evaluation values are used to construct an indexing system. In order to quickly find the desired data among many color pattern data, an efficient index structure should be constructed. To this end, we utilize the R-Tree indexing technique, which is used to index multidimensional spatial data.

The R-Tree is a tree with a balanced height similar to a B-tree containing a pointer to a data object in a leaf node, so the structure is designed so that the index resides on disk and the spatial search visits only a small number of nodes. If so, the nodes correspond to disk pages. The index of the R-Tree is completely dynamic and insertion and deletion can be like a search. The image scale representing each emotional attribute has opposite meanings and has a value between 1 and 7.

Next, the query processor and search method used in the evaluation system according to the present application will be described. It is more convenient to express a user's query in a language composed of adjectives forming an emotional space than a 9-dimensional vector in each image scale. Do.

The detailed description of the evaluation system according to the present invention described above will be further summarized in order to understand.

In general, color images or color patterns collected using trained artificial neural networks and adaptive purge systems are first used for artificial neural networks. And the like is extracted and input to the neural network to evaluate the sensitivity of the image. In addition, in the case of an adaptive purge system, V _a , V _b , I, C, etc. are extracted from an image and input to the system to perform emotional evaluation.

Here, 'emotional evaluation' expresses the degree to which the learned system looks and feels like a human being in coordinates such as "warm-cold" with a number between 0-7. Nine pairs are used, i.e. nine data are obtained for one image, and such image will have a tag of a 9-dimensional vector with nine corresponding emotional estimates stored in the database with this tag. do.

Now, let's take a look at the case of using 'Emotion' mentioned above to find the saved image.

If the person who is looking for a video has a query that says, "Find a video with warmness of 4 and cheerfulness of 7", you can find the query value by referring to the emotional evaluation value tagged with the stored image.

However, it is inconvenient for a person to search by inputting a value such as "4" or "7", and the query "find a little warm, very cheerful video" is much more user friendly. Therefore, it may be desirable to find a desired color pattern using adjectives and a series of adverbs forming an emotional space.

Returning to the description of the present invention, the query processor is composed of adjectives having opposite meanings by using a correlation between adjectives and a role of converting a query expressed in an adjective language into a numerical value (9-dimensional vector). Excludes the query.

The query processing process according to the present invention is as shown in FIG. In FIG. 4, the input linguistic query assumes the CNF (Conjunctive Normal Form) form of adjectives modified by some adverbs.

Linguistic queries in the CNF form check consistency using the correlations of adjectives used in the query. After the consistency check is completed, the query is converted into a numerical query, that is, a 9-dimensional vector, and used to retrieve the indexed color patterns. The sentiment evaluation values of the retrieved patterns are re-evaluated and ranked according to the degree of correspondence with adverbs added to the query, and presented to the user according to the ranking.

Next, the language query method will be described. In the present invention, the language query is constructed based on 18 (9 pairs) emotional adjectives forming 9 image scales. When constructing a query composed of adjectives, each adjective has a grade modifier indicating a degree of sensitivity, that is, a grade on an image scale, and a weight modifier indicating a weight on the grade (ie, a total of nine grades are assumed). Where the grade modifiers are 'highly ', ' normally ', ' slightly ' and the weight modifiers are 'v ery ', ' normally ', ' more or less '.

Referring to FIG. 5, an explanatory diagram showing a position of a grade modifier on an image scale is shown. In FIG. 5, the adjective on the left is centered on the image scale 4 For the right side of the opposite meaning Is about.

Each adjective modified by the modifier represents an individual query, and the individual queries are connected by 'and' and 'or' connectors to form a language query vector. Two adjectives of opposite meaning , If the rank modifier is MD (Modifier of Degree) and the weight modifier is MW (Modifier of Weight), the language query vector is composed as follows.

Here, 'connective' is a connection word such as "and" or "or" expressing a connection relationship, and MD and MW are each three. K and n denote the kth and nth image scales of the nine image scales, respectively, and Q _p and Q _q denote the composed individual queries. And two adjectives of opposite meaning on an image scale cannot be connected at the same time by making separate queries.

Query Classification and Consistency Check

The input language query is classified based on the connector 'or', and the individual queries connected by the connector 'and' are checked to be consistent with each other and only the correct individual query is input. The following equation is an example of such a query classification.

Here, Q and Q, where the individual definitions are connected only with 'and', mean a subquery vector. Therefore, a consistency check is necessary for constructing such sub-vectors. In the present invention, the correlation between image scales is used in the consistency check.

According to the psychological experiment results of Sean et al., There are pairs of image scales that are correlated with each other. [Table 1] shows a correlation coefficient between image scales. It is shown using. According to Table 1, the 'cheerful-dismal' scale has a correlation of 0.74 with the 'light-dark' but has a correlation of -0.49 with the 'hard-soft' scale. Therefore, the query "cheerful and hard" is not appropriate in terms of semantics, so it is excluded from the query input to maintain the consistency of the query. In contrast, "cheerful and soft" is selected as the query input because a positive correlation of 0.49 holds.

Consistency It was determined that the query was consistent only when the negative correlation coefficient value was greater than the threshold T for the case where the correlation coefficient between the adjectives of the adjectives was negative. When checking the consistency, when c (A, B) is the absolute value of the correlation coefficient between two image scales A and B with negative correlation, c (A, B) Is computed as

Consistency check is performed whenever new individual query is connected with 'and', and newly added individual query is added only when the consistency check is valid with all previous individual queries.

Fuzzy Colorization of Color Pattern Sensitivity for Language Queries

Since the color patterns contained in the database have emotional evaluation values for nine image scales, that is, nine emotional attributes, in the present invention, each emotional attribute of the color pattern is fuzzy with respect to language queries. A fuzzy membership function is set for each image scale based on the position of the grade modifier shown in FIG. 5, as shown in FIG. 6.

As shown in FIG. 6, when the i-th emotional attribute value is 5, the fuzzy set for the i-th emotional attribute EF _i (Emotional Feature) is expressed by Equation 13 below.

Therefore, the membership function μ ( slightly adj ⁱ ) = 0.5 for the query " slightly adj ". Depending on the influence of the weight modifier, the membership function is calculated as follows.

Therefore, the fuzzy measure F of the color pattern for the query vector can be obtained using the membership function as follows.

Where N _or is the number of subquery vectors connected by 'or' of the query vector, N is the number of individual queries connected by 'and' in the jth subquery vector, and (Q _i ) is the membership function of the individual query. The intersection and sum of the membership function represent 'and' and 'or', and are calculated by min and max as shown in Equation 16 below.

Color Pattern Retrieval Using Fuzzy Scale

Since the fuzzy scale of Equation 15 indicates how the color pattern corresponds to a given query, the search for the color pattern corresponding to the given query is performed after obtaining the fuzzy scale of each color pattern for the given query. This is done by selecting as many as N by sorting by. However, retrieving the color pattern by obtaining the fuzzy measure of all the color patterns in the database for the input query is not preferable because the calculation amount is increased, and in the present invention, after the first search using the existing R-tree, 1 Second-order search using fuzzy scale was performed on the color patterns searched for.

The first-order search is first performed by digitizing individual queries of the query vector and then using the R-tree. The quantization of the individual queries is considered only for the grade modifier. The reason for this is that considering all nine possible combinations of both three formulas and three weight formulas did not significantly affect the search results during the experiment, and there was too much choice for the user to construct a query. Because it is rather uncomfortable.

That is, "very highly **", "normally highly **" or "more or less highly **", etc. is rather confusing to the user, the weight formula in the experiment according to an embodiment of the present invention all "normally" ", Ie" highly ** "" ** "(in the case of normally), or" slightly ** "and the like. The '**' part is the emotional adjective part.

Accordingly, the transform value on the image scale according to the rating modifier is a value on the image scale whose membership function value is 1 in FIG. 7, 'high ' is 6.5 or 1.5, ' normally ' is 5.5 or 2.5, and ' slightly ' is 4.5 Or 3.5. Here, the adjectives of the individual queries are converted to left if left on the image scale and right for right. Emotional attribute values of each Q _i of the 9-dimensional query vector Q = (Q ₁ , Q ₂ , Q ₃ , Q ₄ , Q ₅ , Q ₆ , Q ₇ , Q ₈ , Q ₉ ) to be input to the R-tree val _i If necessary, the query required for the first search using R-Tree for the query as shown in [Equation 11] is determined as follows.

Where × is an emotional property that is not considered and has no effect when searching in R-tree. Therefore, after searching K color patterns for sub-query vectors Q ' ₁ and Q' ₂ connected by 'or' to each other, the sentiment attribute of primary searched color patterns is fuzzy about the input query and fuzzy scale Then, the quadratic search is performed by selecting N color patterns in ascending order of fuzzy scale.

The 'K' becomes a fixed constant from the user's point of view, and becomes a variable that can be modified from the system builder's point of view. In other words, when the system is built, the stored image pattern of the database can be increased and reduced if it is small, but once the system is built, the user cannot change it arbitrarily.

Experimental Results of Emotion Evaluation System

The three emotional evaluation systems described above aim to approximate the cognitive system that indicates the degree of emotion that humans see and feel color patterns.

Correlate the output values obtained by inputting the physical properties of 30 random color patterns into each of these three systems and the emotional evaluation values provided by the subjects obtained from psychological experiments such as Sean. The performances of three models were evaluated. Here, in models such as Sean and neural network models, Six properties were used and four properties of V, V, I, and C were used as inputs in the fuzzy logic system model. The results of evaluating the performance of the three models are shown in Table 2 below.

Correlation is the correlation coefficient calculated as in Equation 18 below for each scale. The closer to 1, the more accurate the model.

Where i is the i-th image scale, x _i is the subject's emotional attribute value for 30 random color patterns, and y _i is the output value of the model. According to the experimental results as shown in Table 4, when comparing the neural network model with Sean's model, most of the correlation coefficient except for 'hard-soft' showed better or similar than Sean's model. The number of coefficients required for the conversion is 6 * 6 + 5 * 14 for the proposed neural network, compared to 14 * 13 for Sean, etc. ([Number of input layer nodes] * [Number of hidden layer nodes + 1] + [ Number] * [number of output layer nodes + 1]). Accordingly, the model proposed by the neural network according to the present invention was determined to be superior to the model of Sean et al.

Table 3 below shows the number of test patterns in 0.3, 0.5, 0.7 and 1σ, and Table 4 shows the scalar scale values for Table 3.

In the case of using the fuzzy logic system according to the present invention, the correlation coefficient is much better than that of Soen et al. Or neural network model. Thus, with fuzzy logic system It can be seen that the model is best used to model nonlinear systems that recognize the emotions that humans see and feel in color patterns.

However, this model has the disadvantage that there are many more coefficients involved than Sean and Neural Network model. In the case of neural network model, the number of coefficients is increased, but the nonlinear model can be approximated more accurately if the nonlinear characteristics are increased by the increase of hidden layers and the addition of units.

In addition, the proposed model was applied to the actual five color patterns used in Sean's model evaluation experiments to measure the performance of the model. Table 3 summarizes the results. The average and standard deviation of the results obtained by evaluating the actual images of six people were examined, and the deviations of the emotional attribute values obtained from Sean's model and the proposed neural network model were examined. .

As shown in Table 3, there were two cases where Sean included all five errors within 1σ, but there were four cases of neural network model and fuzzy logic system model. Overall, the performance of the fuzzy logic system model was poor only in the two image scales of 'warm-cool' and 'light-dark', but the neural network model and the fuzzy logic system model showed higher overall performance in the remaining image scales. It was shown to represent.

The contents of Table 3 are difficult to evaluate the performances at first glance, and the present inventor proposes a scalar scale as shown in Equation 19 below, and evaluates the performance of each model using the same.

In Equation 19, i is the i-th image scale, and j is an index representing a standard deviation error between 0 to 0.3, 0.3 to 0.5, 0.5 to 0.7, and 0.7 to 1, respectively. N _ij is the number of patterns corresponding to the j-th standard deviation error of the i-th image scale, and w _j is a weight for the number of patterns corresponding to the j-th standard deviation error, which is determined as follows.

Where σ = {0.3, 0.5, 0.7, 1}. Thus, ω = {0.3, 0.27, 0.24, 0.19}. The maximum value of S is 1.5 when the number of patterns is 5 in the first standard deviation error. Table 4 shows the scalar scale for the three models, and it can be seen that the model according to the present invention is closer to human evaluation than the model of Shawn et al.

Color Pattern Database Search Results

The color pattern search system constructed in accordance with the present invention was tested using a collection of textile designs and color patterns of 357 textile designs collected through the Internet.

First, we experimented by changing various emotional queries on the color pattern database created by R-Tree. FIG. 7 is a search result using a neural network sentiment evaluation value corresponding to “very highly dynamic”, and FIG. 8 is a part of a search result using an adaptive purge system sentiment evaluation value for the same query. As can be seen from the above figure, although it is subjective, it can be seen that the search result includes a color pattern that is relatively similar to human feeling, and the result of the adaptive purge system is relatively good for a given query. .

9 and 10 are part of a search result using the emotional evaluation values of the neural network and the adaptive purge system corresponding to "very highly gay", respectively, Figures 11 and 12 are "more or less highly warm and very highly light ( The color pattern using the evaluation values of the two evaluation systems of the query "light-dark) and very highlysoft" is part of the search results.

The evaluation system according to the present invention performs an automatic evaluation of the emotional side as humans see and feel the color pattern by a computer program, and uses this evaluation result to search for the color pattern of the database. Therefore, the evaluation system according to the present invention is a system for automatically evaluating the sensitivity of color patterns, a system for indexing and storing using these evaluation values, and a system for retrieving a color pattern corresponding to an emotional language input expressed as an adjective. It is composed.

The emotional evaluation through the system according to the present invention is performed by a transducer obtained through training and modeling the psychological evaluation result of the linguistic image scale of the random color pattern by Soen et al in Japan as a neural network or an adaptive fuzzy rule-based system. The converter converts the physical attributes obtained through image analysis of color patterns into an emotional space consisting of nine pairs of adjectives with opposite meanings. Through this process, any color pattern can be represented by 9-dimensional numerical vector.

As described above, in the present invention, the system of Sean's system in Japan is applied by applying a known technique that is superior in the ability to approximate the nonlinear function of the artificial neural network and the adaptive purge system to approximate the psychological experiment results with the physical properties of the image and sensitivity. The training resulted in a better approximation of the nonlinear system and provided better evaluation of the actual color pattern.

The 9-dimensional numerical vector obtained through the emotional evaluation according to the present invention is indexed and stored by the R-Tree method, and the stored color pattern is a color having a corresponding emotion according to a query composed of a combination of nine adjective pairs of adjectives. Search for the pattern and present it on the monitor.

The evaluation and retrieval system according to the present invention may be utilized for product search using emotion in evaluation of design results such as cloth or wallpaper and electronic commerce.

The description so far is directed to the preferred embodiments for the understanding of the invention, and the invention is not limited thereto, and is intended to those skilled in the art without departing from the scope and spirit of the appended claims. It is obvious that various modifications and variations are possible.

Claims (7)

A predetermined number of adjective pairs having opposite meanings corresponding to human emotions among the color images or color patterns according to an external control signal are input as physical attributes extracted through an image analysis process of a color image or a color pattern. In the non-linear converter for extracting a color image or color pattern corresponding to the emotional space of A nonlinear transducer characterized by using an artificial neural network structure. Input the physical properties extracted through the image analysis process of the color image or color pattern, and color image or color pattern corresponding to a predetermined number of emotional spaces composed of adjective pairs having opposite meanings corresponding to human emotions. In extracting nonlinear transducer The computing function is represented by a rule corresponding to an adaptive fuzzy logic system used for adaptive rule-based control, respectively, and the fuzzy set assumes a Gaussian function, and has a first color that has an average color and saturation as the physical properties. An adaptive fuzzy logic system consisting of a logic system and a second fuzzy logic system having the textures of average brightness and color patterns as said physical attributes: and In order to fuse the output of the two systems to output the emotional space is defined in the form of a hybrid operation of the intersection and sum operation A nonlinear transducer characterized in that it has a model. A color pattern evaluator for evaluating the color image or the color pattern to correspond to the human emotion through non-linear conversion means; A memory unit for indexing and storing a color image or a color pattern according to a predetermined rule by using the evaluation value evaluated by the evaluation unit; And And a search and extraction unit for searching the memory unit in response to a query expressed in an adjective language and extracting a corresponding color image or a color pattern. The method of claim 3, wherein The color pattern evaluation unit, A first step of conducting a consistency test using the correlations of the input adjectives; A second step of converting the query in which the consistency check is completed into a 9-dimensional vector corresponding to the emotional space; And A predetermined step of receiving a query for a color image expressed in an adjective language from an external source through a third step of searching for a color image or a color pattern stored in the index / search means by using the converted vector value. Emotion evaluation and storage / retrieval system of the color pattern, characterized in that for performing the query processing of. The method of claim 4, wherein The consistency check, Using the correlation between the two image scale (A, B) of the input query and the previously obtained correlation coefficient representing the correlation through the following equation (12), Emotional evaluation of the color pattern, characterized in that it is determined that the query is consistent only when the correlation coefficient between the image adjectives of the input adjectives is negative when the negative correlation coefficient value is larger than a predetermined threshold value (T) And storage / retrieval system. [Equation 12] (Where c (A, B) is the absolute value between the two image scales (A, B)) The method of claim 5, The consistency check is performed every time a new individual query is connected with " and ", and the newly added individual query is performed only when the consistency check of all individual queries is made and it is determined to be valid. Evaluation and storage / retrieval system. A first procedure of extracting a physical property through an image analysis process of a color image or a color pattern; A second procedure for inputting the extracted physical property into an artificial neural network or an adaptive purge system to perform an emotional evaluation of a color image or a color pattern in an emotional space composed of predetermined adjective pairs having opposite meanings corresponding to human emotions ; A third procedure of storing the color image or color pattern on which the emotion evaluation has been performed in a memory unit capable of recording and reading in accordance with a predetermined rule; A fourth procedure of receiving a query for a color image expressed in an adjective language from the outside and converting the query into a numerical value to obtain a numerical vector corresponding to the adjective language; And A computer program having a program for executing a fifth procedure of searching for and outputting a color image corresponding to the input by searching for a color image or color pattern stored in the memory means according to the numerical vector obtained through the fourth procedure. Recording media.