JP2002092611A - Device and method for processing image, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor capable of expressing sensibility common to a plurality of images. SOLUTION: A parameter for determining the input-output characteristic of a sensibility model for inputting image data and outputting a feature quantity showing the sensible expression contents of the image is optimized on the basis of a plurality of object image data. An input part 2 inputs an initial parameter, and a feature quantity computing part 3 extracts the feature quantity from each object image data, using the sensibility model having the input-output characteristic determined by an initial parameter and updated parameter. A feedback evaluation part 4 determines whether or not the updated parameter is an optimum solution on the basis of a plurality of feature quantities. A clustering part 6, a clustering result evaluation part 7 and a parameter control part 8 update the parameter again to improve an evaluation value specified on the basis of the extracted feature quantity, using the sensibility model if the updated parameter is not the optimum solution.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method for performing processing related to sensitivity based on image data.

[0002]

2. Description of the Related Art In recent years, with the widespread use of the Internet centering on the WWW, an environment in which anyone can easily access enormous information of various media has been provided. Under such circumstances, expectations for image creation support such as similar image search and image search based on sensibility based on interpretation of images according to each user's viewpoint and purpose are high, and research results are attracting attention from each industry. (Reference 1).

[0003] In the field of supporting design design based on sensibility, there has been reported a system for manipulating the attributes of an object with respect to an input image and proposing a design closer to the image (Reference 2). This mainly employs two types of methods. One is the simplest method, in which the KJ method is used to convert the image words into the physical characteristics of the design that determines them (Reference 3). The other is SD (Semantic)
A set of target image words is narrowed down by the Differentials scale, and a questionnaire survey is conducted in advance based on the narrowed target image words. A quantification class 1, a neural network, or a GA (Genetic Algorithm) is used.
m), the attributes of the object affecting the image word and their weights are obtained and saved, and when a new input is performed, a design close to the input image word is presented (Reference 3). 4).

[0004] In previous research on image data, efforts have been made to search for images that match the image of the user. Specifically, there is a method in which an image is expressed by sensible feature amounts (saturation, lightness, and the like) and these are compared. In addition, the relationship between the feature amount of an image and an image word is obtained and saved from questionnaire information to a user performed in advance, and in response to a newly input image word search request,
A method of finding an image set similar to them has also been reported (References 5 and 6).

[0005] The above documents are as follows. Reference 1: "Saburo Tsuji:" Science of Kansei ", Science,
1997 ”Reference 2:“ Mitsuo Nagamachi: “Technology to Incorporate Sensitivity into Design”, Journal of the Japanese Society of Fuzzy Science, Vol. 10, p.
p. 1063-1077, 1998 "Document 3:" Mitsuo Nagamachi: "The Story of Kansei Engineering", Japan Standards Association, 1995 "Reference 4:" Norihiko Mori et al .: "Relationship between Attributes in Kansei Evaluation of Products", Proc. Of the 1st An
natural Conference of JSKE,
1999 "Document 5:" Shunichi Kato: "Common Platform of Kansei Engineering Research and Application to Industrial Design" Kansei Kobo "", Pr
oc. of the 1st AnnualConf
Reference 6: "Toshiyuki Iwaoka et al .:" Image Retrieval Using Multidimensional Scale of Kansei Words ", Proc. of the 1st A"
natural Conference of JSKE,
1999 "

[0006]

The following problems have been studied in the above-mentioned conventional technology.

In the image clustering technique based on the user's sensibility, it is difficult to express different sensibility for each individual or image by a kansei word. -Difficulty in acquiring the relationship between non-linear image words and features. -Difficulty in expressing the image of the image data with kansei information.

In the conventional image retrieval based on the user's sensibility, it is necessary to input a natural language including a kansei word as input information, and accurately express sensibility that is slightly different for each individual or image. Was difficult.

Also, in presenting a design close to an input image word, in order to obtain a relationship between the image word and the feature amount, a user evaluation based on a pair of image words prepared in advance is obtained. Word pair SD
It takes time and effort to restrict by law, and moreover, the generated image word pairs include non-exclusions.

Furthermore, in the field of supporting design based on sensibility, a user can select a product suitable for his / her sensibility.
There have been many attempts to select an image. That is, the relationship between the image word and the feature amount is determined by evaluating the object using the above-described image word and utilizing the database. However, conversely, Kansei Diagnosis, which clarifies the features of design drawings, product images, and image sets as Kansei information and Kansei words, and infers what is the most appropriate Kansei for them, is said to be a future issue. (Reference 2).

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above circumstances, and has as its object to provide an image processing apparatus and an image processing method capable of expressing a feeling common to a plurality of images.

[0012]

SUMMARY OF THE INVENTION The present invention determines input / output characteristics of a kansei model in which image data is input and a feature quantity indicating sensible expression content held when the image data is displayed is output. An image processing apparatus for optimizing parameters based on a plurality of target image data, comprising: a parameter input means for inputting an initial value of the parameter; Thereafter, extracting means for extracting the feature amount from each of the plurality of target image data using the sensitivity model having input / output characteristics determined by the updated parameters, Determining means for determining whether or not the updated parameter has reached an optimal value based on the characteristic amount of the parameter. Alternatively, when it is determined that the updated parameter is not the optimal one, an update that updates the parameter based on a predetermined evaluation value defined by the feature amount extracted using the sensitivity model Means.

[0013] Preferably, the evaluation value for a certain parameter is a total combination or a specific combination when two are arbitrarily extracted from a plurality of representative image data selected by a predetermined method from the plurality of image data. For some combinations, two image data extracted from the two image data in each set using the certain parameter
The update means is a sum of distances of the two feature quantities in the feature quantity vector space, and the updating means changes the individual values of the parameters in the ascending or descending direction by a predetermined method. The evaluation values for the parameters are obtained, and among those parameters, the parameter having the best evaluation value and a higher evaluation value than the evaluation value for the parameter before being updated may be set as the updated parameter. Good.

Preferably, the updating means divides the plurality of target image data into a plurality of groups based on the plurality of feature amounts extracted by the extracting means using the parameters before being updated. Then, in each group, representative image data of the group may be selected, and the characteristic amount serving as a basis for defining the evaluation value may be obtained using the plurality of representative image data.

Preferably, the determining means uses the parameter before the update of the evaluation value relating to the variance of the feature amount of the plurality of target image data obtained when the updated parameter is used. When the parameter is not improved compared to the evaluation value regarding the variance of the feature amount of the target plurality of image data obtained when the parameter is updated, it is determined that the parameter before being updated has reached the optimal value. You may.

Preferably, image data input means for inputting a plurality of image data, and the plurality of image data to be targeted are selected from the plurality of input image data based on a predetermined selection criterion. May be further provided.

Preferably, the selection means selects a plurality of image data determined to have a common sensitivity as the object by performing inductive learning based on bibliographic information related to the image data. Good.

The present invention relating to the apparatus is also realized as an invention relating to a method, and the present invention relating to a method is also realized as an invention relating to an apparatus. Further, the present invention according to an apparatus or a method has a function of causing a computer to execute a procedure corresponding to the present invention (or causing a computer to function as means corresponding to the present invention, or a computer having a function corresponding to the present invention). The present invention is also realized as a computer-readable recording medium on which a program (for realizing the program) is recorded.

In the present invention, for example, the following method can be used. -For the image data set, the feature amount is operated by changing a parameter at the time of modeling, and image clustering is performed based on the user's sensitivity. When modeling image data, in order to enable flexible modeling, parameters corresponding to each image or parameters for operating the entire system may be prepared. In the parameter setting, inductive learning is performed from the bibliographic information on the image, the extracted rules are applied to the entire image, and the image is abstracted based on kansei information common to the image set determined from the evaluation. Manipulate parameters in a system (modeling system) for converting into a feature having an expression. That is, first, inductive learning is performed from bibliographic information on an image in order to select an image to be subjected to sensitivity information extraction. The method of selecting the target image may be a method of allowing the user to arbitrarily select an image. Also,
The method of inductive learning may be logic-based learning, decision tree generation, or neural network. The extracted rules are applied to the entire image, thereby determining a target image set. The target image set is classified into several sets, a representative image is selected from each set, and parameters in the modeling system are manipulated in a direction to enhance sensibility information common to them. Note that the classification of the target image set may be a clustering method or a classification method. This series of operations corresponds to a search for an optimal parameter based on the kansei information of the image. Search is repeatedly performed on the target image set using the variance as an evaluation scale, modeling is performed based on the finally obtained optimal parameters, and the feature amounts and parameter sets at that time are stored.
The search is repeated for the target image set using the variance as an evaluation scale, and the feature amount of the image modeled based on the finally obtained optimal parameters can be regarded as expressing some sensibility, in the future It can be applied to Kansei diagnosis.

According to the present invention, a parameter of a kansei model for extracting a characteristic amount related to kansei of image data is optimized based on a plurality of images, so that a common kansei feature is obtained for a certain set of image data. Can be clarified. That is, according to the present invention, it is possible to obtain a kansei model having kansei characteristics common to the plurality of image data.

Further, according to the present invention, in clustering of still image data, an image data set supported by a specific user is determined using an instruction of an image or their category information as input information, and features common to them. , It is possible to accurately express subtly different sensibility for each individual and each image.

[0022]

Embodiments of the present invention will be described below with reference to the drawings.

According to the present invention, there is provided an image processing apparatus comprising: a feature quantity indicating an emotional expression content when image data is input and displayed (for example, a feature quantity relating to an image color of the image data or a composition of the image data; Features related to structure or features related to texture of image data)
Is a process of extracting a feature amount from target image data using a sensitivity model having an output of (i.e., a parameter that determines the input / output characteristics of the sensitivity model (that is, a given parameter). Using a kansei model with input / output characteristics determined by parameters to extract feature values from target image data), consider multiple image data that are presumed or assumed to have some common sensibility in common. By performing a process for optimizing the parameters of the kansei model, the user (for example, a plurality of users having the same attribute) clarifies the kansei of the plurality of (displayed) image data. (I.e., the user can create a kansei model having input / output characteristics determined by the optimized parameters). Sensibility and a model of the Taki Mito).

For example, the kansei model for extracting a feature quantity relating to an image color indicates a model such as how the user perceives the image color from the image data when the user looks at the image data. In this case, for example, a first process for narrowing down the number of types of colors to be extracted as image color candidates and a process for evaluating whether or not the color extracted in the first process is a color to be an image color A second process and a third process of creating a feature amount to be finally output based on the information regarding the color evaluated to be an image color in the second process are performed. Is a parameter for specifying the criteria for narrowing down the number of color types.
The parameter in the second process is a parameter for designating a criterion for evaluating whether the extracted color is an image color (in the third process,
The specification of the feature amount to be finally output can also be a parameter))). Then, for example, a plurality of image data that a user having a specific attribute tends to have a favorable impression or an image data group corresponding to a product group that a user having a specific attribute tends to have an effect intention to purchase is used. By optimizing the parameters of the sensitivity model, it is possible to know how the user perceived the image colors of the plurality of pieces of image data.

The specific examples of the processing, the specific examples of the parameters, and the specific examples of the processing results exemplified in the present embodiment are merely examples, and are not limited to those exemplified in the present embodiment.

FIG. 1 shows a configuration example of an image processing apparatus according to the present embodiment.

As shown in FIG. 1, the image processing apparatus includes an image data input unit 1, a parameter input unit 2, a feature amount calculation unit 3, a feedback evaluation unit 4, a feature amount management unit 5,
It includes a clustering unit 6, a clustering result evaluation unit 7, a parameter control unit 8, a parameter storage unit 9, and a data storage unit (not shown).

The data storage unit stores the input image data,
Generated features, input parameters (initial values),
This is for storing parameters and the like obtained by processing described later, and is constituted by, for example, a hard disk, an optical disk, a semiconductor memory, or the like. The data storage unit may be configured by a separate storage device for each type of data, or may be configured such that all or a part of the various types of data is stored in the same storage device.

This image processing apparatus may have a function of exchanging information with a communication partner via a network.

This image processing apparatus can be realized by executing software on a computer. In this case, an OS having a desired function and software such as driver software, packet communication software, encryption software, and the like are installed as necessary. In this case, a graphical user interface (GUI) is used for inputting information from the user, presenting information to the user, and the like.
Is preferably used.

FIG. 2 shows an example of a processing procedure of the image processing apparatus.

Here, as an example of the sensibility model, a kansei model for determining an image color of image data will be described as an example.

First, in the image processing apparatus, the image data (10) is input by the image data input unit 1. Various input forms such as a keyboard input, an online input via a network, and reading from a magnetic tape, a magnetic disk, or an optical disk as a medium for transmitting information are possible. Or an input form may be selected from a plurality of forms).

The image data may be any digitized data (color bitmap data or data that can be converted into color bitmap data). Further, in the present embodiment, it is assumed that the image data is a color image, but of course, the black and white image data also has a characteristic amount related to color, and the black and white image data is not excluded from the processing target. . In the present embodiment,
An example in which already digitized image data is input from the image data input unit 1 has been described. However, the present image modeling apparatus uses a source of image data that has not yet been digitized as a digital source. It is also possible to provide a function of converting and capturing.

Incidentally, a configuration is also possible in which a plurality of image data is input from the image data input unit 1 and the image data to be used for the parameter optimizing process is automatically selected or designated by the user. It is also possible to adopt a configuration in which a plurality of pieces of image data input from are used for the parameter optimization processing.

The parameter input unit 2 inputs an initial value of a parameter in a predetermined input form such as a keyboard input, online input via a network, or reading from a magnetic tape as a medium for information transmission (step S1). ). The input of the initial value of the parameter may be performed at an arbitrary timing before the processing of the first step S2 (extraction of the feature amount) is performed. Note that the initial values of the parameters may not be given from outside, but may be predetermined initial values (for example, they may be created in a program or described in a predetermined file, and Refers to the predetermined file).

FIG. 3 shows an example of the parameter information. As will be described in detail later, these parameters are parameters in an example of a sensitivity model for determining an image color. The parameter r is a threshold value of color mixture, the parameter c is the limited number of colors, and the parameter v ( s, t,
u) is the influence ratio (hue,
And the parameter m (e, f, g) is the influence ratio (color area size, attractiveness, and contrast) for calculating the degree of conspicuousness. And the parameter i
Is the designated number of colors (the number of types of image colors to be finally output). In the case of this example, in the parameter optimization processing, the number of types of image colors to be used as feature amounts is separately determined. In the present embodiment, the original output of the sensibility model is obtained and stored in the feature amount calculation unit 3, and the output used for the parameter optimization processing is extracted from the output. Therefore, the parameter i is also input as an initial value. However, when the feature amount calculation unit 3 determines only the feature amount necessary for the parameter optimization processing, the initial value of the parameter i may not be input (that is, the parameter i is Not be subject to optimization). Also, other parameters r, c, s, t, u, e,
It is also possible to fix some of f and g and exclude them from the object of optimization.

Here, the specification of the sensibility model and the parameter is described by taking a case where the specification is predetermined as an example.
The user may be able to select what to use.

Next, in the feature amount calculation section 3, given parameters (initial parameter initial values, updated parameters thereafter) are assigned to each of the plurality of image data.
A feature amount of each image data is extracted by applying a sensitivity model based on the image data (step S2).

FIG. 4 shows an example of an image color determination result. In this example, six types (that is, the parameter i =
6), an evaluation value L (for example, 1.668346) as an image color and H
SV space coordinate values (for example, (0.599099, 0.1
83168, 0.789062) and RGB space coordinate values (for example, (165, 180, 202)).

The extracted features are added to the original image data or stored in association with the identification information of the image data.

A specific example of the process performed by the feature value calculator 3 will be described later in detail.

In the case of a configuration in which a plurality of image data is input from the image data input unit 1 and image data to be used for parameter optimization processing is automatically selected or designated by the user, the input is made at this time. Instead of extracting the feature amounts of all the image data, after the image data to be used for the parameter optimization processing is selected or designated, the feature amounts may be extracted only from the selected or designated image data ( For example, the extraction of the feature amount of the image data may be performed for the first time after step S12).

Next, in the present embodiment, image data to be used for parameter optimization processing is automatically selected or designated by the user from a plurality of image data input from the image data input unit 1.

Here, as an example of automatic selection, a questionnaire result obtained by a survey on purchase motivation of a product or a product is used, and a user who has been motivated to purchase a certain product is determined by clustering image data based on sensibility information. , From the commercial (CM) image corresponding to the product,
A case where the sensitivity of those users is clarified will be described as an example.

In this case, first, the feedback evaluation unit 4
It is determined whether the learning of the rule defining the buyer has not been performed on the image data set presented by the image data input unit 1 or the learning has been completed and the feedback has been made ( Step S3).

If the learning has not been performed, the process proceeds to step S12. Although details will be described later, if the learning has already been performed, the process proceeds to step S4 for determining whether the distribution of the input image in the feature amount space (HSV space) has contracted.

In step S12 (purchase feature extraction), the condition of the consumer who has been motivated to purchase a specific product by the influence of the CM is learned.

For example, as shown in the contents of the input data in FIG. 5 and the specific examples of the data in FIG. 6, based on bibliographic data on the product and the consumer, the willingness to purchase the product is set as a learning target, and the consumer attribute is set as a background. As knowledge, try to acquire the characteristics of consumers who are willing to buy.

In the example shown in FIG. 5, program parameters, mode declarations, positive cases, and background knowledge are used as the types of input data, and the contents of the program parameters are a noise allowable value, a limit on the number of search nodes, an input mode, and a mode. The content of the declaration is the type of rule to be generated, the content of the correct case is the relationship between the purchaser and the product to be purchased, and the background knowledge is the personal information of the purchaser (sex, generation, income, family relationship, etc.) In this case, the case of using life information of the prospective purchaser (hobbies, utilization rates of various organizations, subscribed newspapers, life consciousness, etc.) is illustrated. FIG. 6 is an example of the embodiment of FIG.

Here, in order to handle information indicating the relation between attributes as background knowledge in addition to the hierarchical relation of attributes, S.N. Pro developed by Muggleton
gol (“S. Muggleton:“ Inverse
entailment and Progol ”,
New Generation Computing,
Vol. 13, pp. 245-286, 199
5, "Sam. Roberts:" An Intro
"Ducation of Progol", 199
7)), but of course, other methods such as logic-based learning, decision tree generation, and neural networks can also be used.

By such learning, for example, a rule as shown in FIG. 7 can be obtained.

Next, in order to reflect the rule illustrated in FIG. 7 in the image data, first, a consumer set B described by the rule (for example, a set of consumers having an attribute that they do not like a foreign brand) Obtains a set A of products to be purchased in addition to the specific product (for example, a video game) as follows (step S1).
3). (1) The condition part of the rule is applied to the entire set, and the target set (for example, DVD software, personal computers, mobile phones, ...). (2) For each of the products included in this target product group,
The product (for example, DVD software) in the consumer set B
If the percentage of consumers who have a willingness to purchase is greater than the percentage of consumers who have a willingness to purchase the product (for example, DVD software) among all consumers, the product (for example, DVD software) determines that the consumer set B is a product that has motivation to purchase together with a specific product (for example, a video game).

(3) The specific product (for example, a video game) and the product group satisfying the condition (2) among the target product groups of (1) are defined as a product group T.

When the product group T is determined, a set of image data corresponding to each product belonging to the product group T is set as target image data used in the parameter optimization processing (step S14).

Next, an example of user designation will be described.
In this case, first, in step S3, it is determined whether or not the target image data has already been specified. If not specified, the user specifies the target image data instead of steps S12 to S14. There are various ways for the user to specify the target image data. For example, when attribute information is added to the image data, the user specifies the condition regarding the attribute information that the target image data should have. Alternatively, for example, image data may be displayed one by one or a plurality of images, and an instruction as to whether or not each image data is targeted may be received from a user who has viewed the image data.

Now, once the target image data is determined,
The feature management unit 5 retrieves the feature amount of the target image data from among the feature amounts obtained by the feature amount calculation unit 3 for subsequent processing. At this time, if it is necessary to process the stored feature amount, it is processed and stored. For example, the feature amounts illustrated in FIG. 4 are stored, and in the subsequent processing, H of the upper two image colors is stored.
In the case of using a feature amount which is made up of 6-dimensional vector data by connecting SV space coordinate values, 6-dimensional vector data (FIG. 4)
For example, (0.599099, 0.183168,
0.789062, 0.666667, 0.02083
3, 0.187500)).

Next, clustering of feature amounts of the target image data (step S6), selection of a representative image (step S6)
7) Perform parameter adjustment (step S8).

Here, as an example of the feature value, the 6-dimensional vector data obtained by connecting the HSV space coordinate values of the upper two image colors obtained from the kansei model for determining the image color of the image data is used. And

Here, a case where parameter optimization is performed using the target image data selected by the above automatic selection will be described as an example.

In this case, it is considered that the features shared by the CM image data corresponding to the individual products belonging to the product group L selected by the automatic selection are elements appealing to the consumer set B.

First, the clustering unit 6 performs clustering on the feature amount of image data (for example, CM image data) (step S6). Note that here, CM
The impression of the image data is expected to differ depending on the product category, and the number of product categories belonging to the product group is set as the number of clusters. However, other methods are also possible for setting the number of clusters. Here, the K-Means method ("Statistics Encyclopedia (p.389), edited by Satoshi Takeuchi, Toyo Keizai Shimpo ISBN4-492-01038")
-6 "), but of course, other clustering methods or classification methods may be used.

Next, the clustering result evaluation unit 7
Evaluate the result of clustering.

It is presumed that the representative elements of each cluster have little relationship with each other, and image data corresponding to the feature amount closest to the average value of the feature amounts in the cluster is set as representative image data (step S7).

For example, when the feature quantity is two-dimensional data and the number of clusters is 2 for simplicity, as shown in FIG. 8, the feature quantity vector closest to the intra-cluster average value of the feature quantity vector in the cluster K1 is used. The image data corresponding to k1 is determined as the representative image data in the cluster K1. Similarly, the image data corresponding to the feature vector k2 closest to the average value of the feature vectors in the cluster K2 is defined as the representative image data in the cluster K2. Determined as image data. Of course, the same applies when the number of dimensions of the feature quantity vector is three or more, or when the number of clusters is three or more.

Next, it is considered that the features common to the representative image data can be regarded as similar sensitivity features based on the impression of the CM image data, for example, and the parameter control unit 8 calculates the feature amount so as to emphasize them. A parameter (see FIG. 3) for controlling a result (see FIG. 4) obtained by the unit 3 is operated (step S8).

More specifically, first, feature amounts (for example, six-dimensional vector data) relating to representative image data obtained using the parameters which have been optimized so far (the parameters before adjustment or before updating) are obtained. ) Is obtained (the value calculated in the previous process may be stored and referred to, or may be calculated here). Then, with respect to all (or a part of) the parameters, the values are operated in both upward and downward directions (for example, in a multi-loop operation) with respect to the parameters that have been determined to be optimal so far. Originally, for each of the updated candidate parameters, a feature amount (for example, 6-dimensional vector data) relating to the representative image data obtained by the feature amount calculation unit 3 when using the parameter is obtained. , A predetermined evaluation value based on the characteristic amount (six-dimensional vector data) relating to the representative image data is obtained. Then, it is determined whether or not a predetermined evaluation value for a parameter that is a candidate for an updated value is the best and is improved more than a predetermined evaluation value for a parameter that has been optimized so far. If there is a parameter having such a predetermined evaluation value, the parameter is determined as an optimal solution at the present time (actually operating the parameters in any order to obtain the evaluation value,
Various variations are possible as to whether to compare the evaluation values.) If such parameters cannot be obtained, the previous parameters are determined as the optimal solution.

The above-mentioned predetermined evaluation value is, for example, as shown in FIG. 8, a distance in the vector space between the characteristic amount vectors of the representative image data. In this case, the smaller the distance, the better the evaluation value. That is, as the distance decreases, the evaluation value improves. In the case of the example of FIG. 9, the distance between the feature vector k1 ′ and the feature vector k2 ′ is smaller than the distance between the feature vector k1 and the feature vector k2, and Since it is short, a parameter that provides the feature vector k1 ′ and the feature vector k2 ′ is selected as an updated parameter.

If there are three or more representative image data pairs, the sum of the distances between the feature values for all combinations of representative image data pairs is used as the evaluation value. There is a method in which the sum of the distances between the feature values of a specific part of the evaluation value is used as an evaluation value. In the latter case, for example, assuming that the representative image data is # 1 to #n, the distance between the characteristic amount # 1 of the representative image data # 1 and the characteristic amount # 2 of the representative image data # 2, the characteristic amount # 2 and the characteristic amount Distance # 3, feature # 3 and feature # 4
,..., The sum of the distances between feature # n−1 and feature #n, or the sum of the distance between feature #n and feature # 1. is there.

Note that the above-described parameter updating method is an example, and other methods can be used.

When an updated parameter having an improved predetermined evaluation value is obtained (step S9), the updated parameter is stored in the parameter storage unit 9 (step S11).

On the other hand, if the updated parameter whose predetermined evaluation value has been improved cannot be obtained, the process is terminated (step S10). Then, the parameters that have been optimized so far are determined as optimal solutions.

The contents of the feature amounts used in steps S6 to S8 may be specified by the user.

Now, if updated parameters with improved predetermined evaluation values are obtained and stored, feedback is provided. First, in step 2, the feature values of the target image data are obtained using the updated parameters. Is extracted.

Next, in step S4, the feedback evaluation section 4 sets the target image data (for example, the product group T
It is determined whether or not the distribution in the vector space of the feature amount vector of the image data corresponding to each product belonging to.

Various variations can be considered as a scale for determining whether or not aggregation has occurred.

For example, as a scale for determining whether or not aggregation has occurred. There are three types of evaluation value weighted addition values (including a case where the weight is 0). -Total distance value between each element in the cluster and the center-Distance between cluster centers-Dispersion of feature amount of target image data set For example, for the parameter before update and the parameter after update, the value of the above scale is respectively (The parameters before the update are obtained based on the clustering result when the parameters before the update are used, and the parameters after the update are obtained based on the clustering result when the parameters after the update are used). In the case of the above example, if the value of the parameter scale after the update is smaller than the value of the parameter scale before the update, it is determined that the distribution of the target image data set in the feature amount space has contracted. . In this case, the result of the clustering using the parameters before the update may be the one already obtained, but the result of the clustering using the parameters after the update has not yet been obtained. May be performed at this point.

For example, if the distribution of the target image data set in the feature quantity space for the updated parameters is as shown in FIG. 10, the distribution in the feature quantity space of the target image data set for the parameter before the update in FIG. It is determined that it contracted. In this case, the processing will be continued.

The content of the scale used in step S4 for determining whether or not aggregation has occurred may be specified by the user.

If it is determined that the distribution of the target image data set in the feature space has not shrunk (the distribution has diverged) (step S4), this processing is terminated (step S5), and the updated The previous parameter, not the parameter, is determined as the optimal solution.

If it is determined that the distribution of the target image data set in the feature quantity space has shrunk, it is determined that a better parameter representing the sensibility based on the impression of the target image data set has been obtained. Until it is determined that a better parameter has not been obtained or the distribution has diverged in step S4, step S6
Steps S9 and S2 are repeated. Finally, parameters determined to be optimal are obtained.

In the following, there will be described a kansei model in which image data is input and a feature quantity indicating sensuous expression content possessed when the image data is displayed is output. An example of extracting a feature amount from image data using a sensitivity model having the following parameters will be described.

FIG. 11 shows an example of the internal configuration of the feature value calculator 3 in this case.

FIG. 12 shows an example of a processing procedure for extracting an image color in color-based modeling.
The process of FIG. 12 is performed for each image data.

The parameters are as shown in the example of FIG. The parameter r is a threshold value of color mixture, the parameter c is a limited number of colors, and the parameter v (s, t, u) is an influence ratio (hue, saturation, lightness) for calculating the attractiveness. And the parameter m (e, f, g)
Is the influence ratio for calculating the degree of conspicuousness (the ratio of influence on the size of the color region, the height of the attractiveness, and the height of the contrast feeling), and the parameter i is the designated number of colors.

First, the color list creation unit 33 creates a color list for the target image data.

The color list creating section 33 divides the target image data into pixels and reads color information corresponding to each pixel (step S21). Each color is represented by three primary colors R (red), G (green), B (blue), and 8-bit gradation. These values are converted into an L * a * b * space based on a known coordinate conversion formula.

Next, the color list creation unit 33 determines whether L * a
A color list composed of registered colors represented by * b * space coordinate values and the number of appearances obtained by adding the number of pixels for each of them is created (step S22).

However, in the present embodiment, when a color list is created, a process of mixing colors is used for the vector converted into the L * a * b * space. To do.

First, L * of the color of each pixel of the input image data
One a * b * space vector value is read, and its three-dimensional vector value and the number of appearances = 1 are registered in a color list.

Hereinafter, the L of the color of each pixel of the input image data
Regarding the * a * b * space vector values, one of the unprocessed ones is read, and the vector value of the newly read color and the vector value of the registered color already registered in the color list in the previous process are , L * a * b * Euclidean distance in space (when there are multiple registered colors, the minimum value)
If the calculated Euclidean distance is larger than the color mixture threshold (parameter r), it is determined that there is no color close to the color list, and the three-dimensional vector value of the newly read color and the number of appearances = 1 are calculated as Register to the color list. On the other hand, if the obtained Euclidean distance is equal to or smaller than the threshold value of the color mixture (parameter r), it is determined that the colors are close to each other, and color mixing is performed for the newly read color and the registered color given the Euclidean distance as described later. When color mixing is performed, the three-dimensional vector value of the registered color before color mixing and the number of appearances are invalidated. Instead, the color after color mixing is set as the registered color, and the three-dimensional vector value and the number of appearances = color mixing. The previous number of occurrences + 1 is registered in the color list.

The above description is based on the L of the color of each pixel of the input image data.
This process is repeated until all * a * b * space vector values are read and processed.

The L * of the color of each pixel of the input image data
The order in which the a * b * space vector values are read one by one may be determined as appropriate. In the above description, when there are a plurality of registered colors, the minimum value of the Euclidean distance is calculated.However, the Euclidean distance becomes equal to or less than the threshold value of the mixed color only during the calculation of the Euclidean distance with each registered color. In such a case, the calculation may be terminated and the registered color at that time may be adopted. In the above description, the color list is created by reading all the L * a * b * space vector values of the colors of each pixel of the input image data. L * a *
It is also possible to read the b * space vector value and create a color list.

Here, the color mixing method will be described.

Registered color B mixed with newly read color A
Let a and b be three-dimensional coordinate vectors in the space of
Assuming that a coordinate vector of a color X (new registered color after color mixing) X generated as a result of mixing the colors A and B is x, x is defined by the following color. _{^{x → = w A · a →}} + w B · b → _{here, w A = N A ÷ (} N A + N B) w B = N B ÷ (N A + N B) N A: the number of occurrences N _B of color A : Number of appearances of color B. Further, the number of occurrences N _x of the generated result of the mixing color X is _{N x = N A + N B} . However, in this embodiment, N _A = 1 because the colors are read and processed one by one.

In the above description, one color list is created by reading the colors of the input image data one by one. For example, half of the pixels of the input image data are used to create the same color list. A method that integrates the color list that has been created and a color list that is created in the same manner as described above using the remaining half of the pixels of the input image data into one color list (by performing further color mixing processing, etc.) , Other methods are possible.

Next, in the created color list,
A registered color whose appearance number is equal to or less than a reference (for example, a number equal to or less than 0.5% of the total number of pixels) is regarded as noise, and information on the corresponding registered color is deleted from the color list (step S23).

Next, the number of registered colors registered in the color list at this point is checked, and if the number is larger than the limited number of colors (parameter c) (step S24), a process for limiting the registered colors is performed. Perform (Step S25). If the number of registered colors is equal to or less than the limited number of colors (parameter c), the process of limiting registered colors is skipped.

In the process of limiting the registered colors, the registered color limiting unit 34 performs the color mixing process until the number of registered colors in the color list becomes equal to or less than the limited number of colors (parameter c). One color can be reduced). That is,
At each time point, the closest two colors of all registered colors, ie, L * a
The two colors that minimize the Euclidean distance in the * b * space are searched, and they are mixed according to the above-described color mixing method, and the color list is updated. This is repeated until the number of registered colors in the color list becomes equal to or less than the limited number of colors (parameter c).

Next, the registered color limiting section 34 determines a color to be used for the input image data from the registered colors registered in the color list.

The processing of steps S26 to S28 is a repetitive processing.

First, the registered color limiting section 34 sets L * a * b *
Are divided into regions, and a histogram based on the number of appearances in the region is created for each axis of L * a * b * (step S26). Note that the method of dividing the area may be determined as appropriate.

Next, in the histogram corresponding to the L axis, the number of projections formed by a graph connecting the number of appearances of each region (FIG. 13)
Shows an example in which the number of protrusions is 3), the number of protrusions formed by a graph connecting the number of appearances of each region in the histogram corresponding to the a-axis, and the number of appearances of each region in the histogram corresponding to the b-axis. The number of projections formed by the connected graph is checked, and the largest number of projections is used as the number of colors used in the input image data, and the number of colors used is stored (step S2).
7).

Next, the number of colors used this time is compared with the number of colors used last time, and if there is any change (step S28),
It returns to step S26. In the first time, since there is no previous number of colors used, it is assumed that there is a change in the number of colors used (or, for example, the initial value of the number of colors used may be set to 0).

When returning to step S26, step S26 is repeated until the number of used colors obtained in step S27 is reached.
In the same manner as in step 25, the closest colors in the color list are mixed, and a histogram is created again. Further, in step S27, the number of colors used in the input image data is obtained again and stored.

The above processing is repeated until it is determined in step S28 that the number of colors used has not changed.

If it is determined in step S28 that the number of colors used this time has not changed from the number of colors used last time, this loop processing ends, and the registered colors in the color list at this point are This means that the input image data has been determined as the use color.

Next, the degree of conspicuity is calculated by the conspicuous degree calculating unit 35 for the colors used in the input image data (ie, the registered colors in the color list).

First, the prominence degree calculating unit 35 uses the HSV which is considered to be closer to the human sense from the L * a * b * coordinate system for the colors registered in the color list.
(Hue: Hue, Saturation: Saturation, Val
ue: lightness) is converted to a coordinate system (step S2)
9).

Next, based on the influence ratio (parameter v (s, t, u)) on hue, saturation, and lightness, the height C ₂ of the attractiveness, which is an element of the degree of prominence, is calculated as follows. (Step S30). C ₂ = (s × influence value due to hue + t × saturation + u × lightness) ÷ (s + t + u) (1) The influence value used here corresponds to the range of hue H values.

Next, the image color extracting section 36
Select an image color.

First, the image color extracting unit 36 determines the degree of conspicuousness L for the registered colors in the color list by the influence ratios on the size of the color region, the height of the attractiveness, and the height of the contrast. Based on the parameter m (e, f, g)), calculation is performed as follows (step S31). L = ((e · C ₁ ) ² + (f · C ₂ ) ² + (g · C ₃ ) ² ) ^1/2 (2) Here, the region of the color of interest with respect to each registered color in the color list The size C ₁ and the sense of contrast C ₃ , respectively,
It is defined as the ratio of the number of appearances to the total number of pixels, and the difference in brightness from other colors. C ₂ is the above-mentioned attractiveness.

As a result, a prominent degree L for each registered color in the color list is obtained.

Then, the image color extracting section 36 selects the registered colors of the color list for the input image data from the one having a high degree of conspicuousness L as the image colors designated by the designated number of colors (parameter i). (Step S32).

The technique of narrowing down the number of colors is based on the PA method (see P. Heckbe in Reference 7) instead of the color mixing method described above.
rt: Color Image Quantizati
on Frame Buffer Displa
y, Computer Graphics, Vol.
16, No. 3, 1982 ") and the MCA method (Reference 8).
"Toshikazu Suzuki et al .: Limited color display method with flexibility in color selection, Journal of the Institute of Television Engineers of Japan, Vol. 43, No. 3, p.
p. 268-275, 1989 ").

In the above description, the degree of conspicuousness is used as an evaluation value for selection, but other methods are also possible.

As described above, the feature value extracted by the feature value calculation unit 3 is stored in a manner added to the image data on which the feature value is based or associated with the identification information of the image data.

In the above description, an example of a sensibility model relating to image color features as color-based features has been described.
Various kansei models such as a kansei model for a feature based on composition, a kansei model for a feature based on an object, and a kansei model for a feature based on texture can be used. A sensibility model that handles a plurality of types of sensibility-based feature amounts, such as a kansei model using both image color feature amounts and composition-based feature amounts, is also possible.

For example, when a kansei model related to a feature based on a composition is used, the feature is used as information that can specify the angle and position of a dividing line that divides image data on the basis of kansei, and the parameter is used as a parameter. It may be provided as a list structure including a restriction on a division order for a direction and a restriction on a division angle for each division direction. In this case, processing is performed to determine information that enables the angle and position of the dividing line to be specified within the range of the restriction given as the list structure. Also, this division is performed under a predetermined division condition (for example, the area of the divided area is equal to or larger than a threshold value, or the variance of the hue in the area before the division and the area between the two areas after the division) It is also possible to repeat recursively while satisfying that the evaluation value indicating the strength of division defined by the variance of the shade is equal to or more than a threshold value.

In the above description, the kansei model to be used is predetermined, but a plurality of kansei models may be prepared so that the user can select a desired kansei model.

It is to be noted that the kansei model of each of the following documents is modified so that image data is input, and a feature amount indicating the sensible expression content possessed when the image data is displayed is output. By newly defining a sensitivity model having parameters for determining the input / output characteristics, this can be used as the sensitivity model of the present embodiment. Object extraction method Reference 9: "Computer Vision, New Technology Communications, 1998" Contour extraction using mathematical morphology Reference 10: "J. Serra: Image Anal
ysis and Mathematical Mor
philology, Academic Press,
London, 1982 "Extraction of characteristic color Reference 11:" Yuhiro Morohara et al .: Method of selecting image color in textile design image, Transactions of Information Processing Society of Japan,
Vol. 36, no. 2, pp. 329-337, 19
95 "Document 12:" Akihiro Kuroda et al .: Integrated System for Design Applying Kansei Retrieval of Image Database, 9th N
ICOGRAPH Transactions, pp. 113-122, 19
93 ”Composition Reference 13:“ Hidenori Yamamoto et al .: Similar Image Search Considering Spatial Distribution of Color Information, IEICE Technical Report, EID9
8-171, IE98-162, 1999-02 "Document 14:" Hideyuki Kobayashi et al .: Image Retrieval System Integrating Features and Approaching Human Sensitivity, IEICE Technical Report, PRMU97-261, 1998-03 Reference 15: “Shigeru Mase et al .: Morphology and Image Analysis [1], Journal of the Institute of Electronics, Information and Communication Engineers, Vol.
2, pp. 166-174, 1991 "Reference 16:" Sh 修 ko Ueda et al .: Morphology and Image Analysis [2], Journal of the Institute of Electronics, Information and Communication Engineers, Vol.
3, pp. 271-279, 1991 "Note that each of the above functions can also be implemented as software. In addition, the present embodiment is a computer-readable computer that stores a program for causing a computer to execute predetermined means (or for causing a computer to function as predetermined means, or for causing a computer to realize predetermined functions). It can also be implemented as a recording medium.

The configuration illustrated in the present embodiment is an example, and is not intended to exclude other configurations. Instead, a part of the illustrated configuration may be replaced with another one, or the configuration illustrated in FIG. Another configuration obtained by omitting a part, adding another function to the illustrated configuration, or combining them is also possible. In addition, another configuration that is logically equivalent to the illustrated configuration, another configuration including a portion that is logically equivalent to the illustrated configuration, another configuration that is logically equivalent to the main part of the illustrated configuration, and the like are also possible. is there. In addition, another configuration that achieves the same or similar purpose as the illustrated configuration,
Another configuration that has the same or similar effect as the illustrated configuration is also possible. Further, in the present embodiment, various variations of various components can be implemented in combination as appropriate. In addition, the present embodiment encompasses and includes inventions relating to various viewpoints, stages, concepts or categories, such as inventions as devices, inventions for components inside the devices, and inventions for methods corresponding thereto. is there. Therefore, the present invention can be extracted from the contents disclosed in the embodiments of the present invention without being limited to the illustrated configuration.

The present invention is not limited to the above-described embodiments, but can be implemented with various modifications within the technical scope.

[0124]

According to the present invention, the parameters of the kansei model for extracting the characteristic amount related to the kansei of the image data are optimized based on the plurality of images, so that the parameters are common to the plurality of image data. It is possible to obtain a kansei model having kansei characteristics.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is an exemplary flowchart illustrating an example of a processing procedure of parameter optimization in the image processing apparatus according to the embodiment.

FIG. 3 is an exemplary view illustrating an example of parameters of a sensitivity model in the embodiment.

FIG. 4 is a view showing an example of a feature amount related to kansei extracted from image data in the embodiment.

FIG. 5 is an exemplary view for explaining feature extraction of a buyer according to the embodiment;

FIG. 6 is a view showing an example of an input to a feature extraction of a buyer in the embodiment.

FIG. 7 is an exemplary view showing an example of rule information obtained by extracting a feature of a purchaser in the embodiment;

FIG. 8 is a diagram for explaining clustering in the embodiment;

FIG. 9 is an exemplary view for explaining parameter updating in the embodiment.

FIG. 10 is an exemplary view for explaining an example of a method for determining the optimality of a parameter in the embodiment.

FIG. 11 is a diagram showing an example of an internal configuration of a feature amount calculation unit according to the embodiment.

FIG. 12 is an exemplary flowchart illustrating an example of a processing procedure for extracting an image color in the embodiment.

FIG. 13 is an exemplary view for explaining an example of processing relating to a histogram in the embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 image data input unit 2 parameter input unit 3 feature calculation unit 4 feedback evaluation unit 5 feature management unit 6 clustering unit 7 clustering result evaluation unit 8 parameter control unit 9 parameter storage unit 33 Color list creation unit 34: Registered color limitation unit 35: Prominence degree calculation unit 36: Image color extraction unit

Claims (11)

[Claims] 1. A plurality of parameters for determining input / output characteristics of a kansei model that outputs image data and outputs a feature amount indicating sensible expression content possessed when the image data is displayed. An image processing apparatus for optimizing based on image data, comprising: a parameter input unit for inputting an initial value of the parameter; the first time, the parameter input as an initial value; and the second and subsequent times, the updated parameter. ,
Extracting means for extracting the feature amount from each of the plurality of target image data using the sensitivity model having determined input / output characteristics; and updating based on the extracted plurality of feature amounts. Determining means for determining whether or not the updated parameter has reached an optimum value; and determining whether or not the parameter has an initial value or has determined that the updated parameter has not reached an optimum value. An image processing apparatus comprising: updating means for updating the parameter based on a predetermined evaluation value defined by the feature amount extracted using a model. 2. The method according to claim 1, wherein the evaluation value for a certain parameter is a total combination or a specific one when two are arbitrarily extracted from a plurality of representative image data selected by a predetermined method from the plurality of image data. For the combination of parts, the distances in the feature vector space of the two feature quantities extracted using the certain parameter for the two image data in each set are totaled, and the updating means Calculating the evaluation value for the parameter in each case while changing the individual value of the parameter in the upward or downward direction in a predetermined manner, and among these parameters, the evaluation value is the best evaluation value and is updated. A parameter having an evaluation value better than the evaluation value for the previous parameter is set as the updated parameter. The image processing apparatus according to claim 1. 3. The updating means divides the plurality of target image data into a plurality of groups based on the plurality of feature amounts extracted by the extracting means using the parameters before being updated. Selecting, in each group, image data that is representative of the group, and using the plurality of representative image data to determine the feature amount that defines the evaluation value. 3. The image processing device according to 1 or 2. 4. If the updating means does not obtain a new parameter that further improves the evaluation value as compared with the parameter having the best evaluation value at that time, The image processing apparatus according to any one of claims 1 to 3, wherein a parameter having the best evaluation value at a point in time is set as an optimized parameter. 5. The evaluation means according to claim 1, wherein the evaluation value relating to the variance of the characteristic amount of the plurality of target image data obtained when the updated parameter is used uses the parameter before the update. When the parameter is not improved as compared with the evaluation value regarding the variance of the feature amount of the plurality of target image data obtained in the case, it is determined that the parameter before being updated has reached an optimum value. The image processing apparatus according to claim 1, wherein: 6. An image data input means for inputting a plurality of image data, and for selecting the plurality of image data to be targeted from the plurality of input image data based on a predetermined selection criterion. The image processing apparatus according to claim 1, further comprising a selection unit. 7. The selection means selects a plurality of image data determined to have a common sensibility as the target by performing inductive learning based on bibliographic information related to the image data. The image processing device according to claim 6. 8. Each of the image data is image data corresponding to each product, and the bibliographic information includes an attribute of the subject and information on a product for which the subject has a purchase intention. The selecting means determines that the subjects having the attribute determined to have in common by the subjects having a tendency to purchase a certain product as a result of the inductive learning are determined to have a tendency to have the purchasing intention And obtaining image data corresponding to the certain product and one or more of the other products as the plurality of image data determined to have the common sensibility. 8. The image processing device according to 7. 9. The selecting means according to claim 1, wherein a ratio of a target person having a purchase intention to the specific product among the target persons having the attribute is determined to be a purchase intention for a specific product among all target persons. If the percentage of subjects with
9. The apparatus according to claim 8, wherein the target person having the attribute is determined to be the other product that tends to have a purchase intention.
An image processing apparatus according to claim 1. 10. A plurality of parameters for determining input / output characteristics of a kansei model that outputs image data and outputs a feature amount indicating sensible expression content possessed when the image data is displayed are set as a plurality of parameters. An image processing method for optimizing based on image data, comprising: inputting an initial value of the parameter, using the sensitivity model having input / output characteristics determined by the parameter input as an initial value, Extracting the feature amount from each of a plurality of target image data; updating the parameter so as to improve a predetermined evaluation value defined based on the feature amount extracted using the sensitivity model; Using the kansei model having input / output characteristics determined by the updated parameters, from each of the plurality of target image data, Extracting the collected amount, determining whether or not the updated parameter has reached an optimal value based on the extracted plurality of feature amounts, and thereafter, determining that the updated parameter has reached an optimal value. Updating the parameter based on a predetermined evaluation value defined by the feature value extracted using the kansei model until the determination is made, and input / output characteristics determined by the updated parameter. Extracting the feature amount from each of the plurality of target image data by using the sensitivity model having the above-described image processing method. 11. A plurality of parameters for determining input / output characteristics of a sensibility model which outputs image data and a characteristic amount indicating sensuous expression content when the image data is displayed are set as a plurality of target objects. A computer-readable recording medium in which a program for causing a computer to function as an image processing apparatus that optimizes based on image data is recorded, wherein an initial value of the parameter is input, and the parameter is input as an initial value. Using the kansei model having the determined input / output characteristics, the feature amount is extracted from each of the plurality of target image data, and is defined based on the feature amount extracted using the kansei model. The parameter is updated so as to improve a predetermined evaluation value to be determined, and a determination is made based on the updated parameter. Using the kansei model having the input / output characteristics thus obtained, the feature amount is extracted from each of the plurality of target image data, and the parameter is updated based on the extracted plurality of feature amounts. Is determined whether or not the optimal parameter is reached.After that, until the updated parameter is determined to have reached the optimal parameter, the predetermined parameter defined by the feature amount extracted using the sensitivity model Updating the parameter based on the evaluation value, and using the sensitivity model having input / output characteristics determined by the updated parameter, the feature amount is calculated from each of the plurality of target image data. And a computer-readable recording medium on which a program for causing the computer to repeatedly perform the extraction is recorded.