JP6900170B2 - Information processing systems, information processing methods, and programs - Google Patents

Description

The present invention relates to information processing systems, information processing methods, and programs, and is particularly suitable for use in determining the aesthetics of an image.

Criteria for judging whether an image is beautiful (aesthetic criteria) include criteria specific to a specific person in addition to criteria for obtaining consensus among an unspecified number of people. As a technique for estimating the latter criterion, there is a technique described in Patent Document 1. In Patent Document 1, a decoded image is generated by changing the encoding / decoding parameters and presented to the user, the user's subjective quality value for the decoded image is acquired, and based on the acquired subjective quality value. A technique for classifying users into any of a plurality of groups is disclosed.

JP-A-2007-336406

Ritendra Datta, Dhiraj Joshi, Jia Li and James Z. Wang, "Studying Aesthetics in Photographic Images Using a Computational Approach," Lecture Notes in Computer Science, vol. 3953, Proceedings of the European Conference on Computer Vision, Part III, pp. 288-301, Graz, Austria, May 2006. Luca Marchesotti, Florent Perronnin, Diane Larlus, Gabriela Csurka. Assessing the aesthetic quality of photographs using generic image descriptors. ICCV 2011: 1784-1791 Xin Lu, Zhe Lin, Hailin Jin, Jianchao Yang and James Z. Wang, "RAPID: Rating Pictorial Aesthetics using Deep Learning," Proceedings of the ACM Multimedia Conference, pp.-, Orlando, Florida, ACM, November 2014.

However, in the technique described in Patent Document 1, it is assumed that the subjective quality value input by the user is the aesthetic standard for the image of the user. That is, in the technique described in Patent Document 1, the "appearance" of the image seen by the user is estimated as it is as the aesthetic standard of the user. Therefore, from the viewpoint of "beauty" and "beauty" for an image, it is not possible to obtain an aesthetic standard for an image peculiar to a specific person.

The present invention has been made in view of such a problem, and as a criterion for determining the aesthetics of an image of a specific person, a criterion for determining "beauty" or "beauty" of an image can be obtained. The purpose is to.

The information processing system of the present invention, an image captured by the image capturing unit, an input unit in which a plurality of users by regarding the aesthetics of the image to input the supplementary information including the content of evaluation of the evaluation, the plurality of users Is divided into clusters to which users with similar evaluation contents for a plurality of images input by the input means belong, a plurality of images input by the input means, and users belonging to the same cluster with respect to the images. It is characterized by having a generation means for generating a judgment criterion of aesthetics of a user belonging to the cluster based on the result of machine learning using the incidental information including the evaluated evaluation content.

According to the present invention, as a criterion for determining the aesthetics of an image of a specific person, it is possible to obtain a criterion from the viewpoint of "beauty" or "beauty" for an image.

It is a figure which shows the schematic structure of a digital camera. It is a figure which shows the functional structure of a digital camera. It is a flowchart explaining the processing of a digital camera. It is a figure which shows the detailed structure of the aesthetics evaluation part. It is a figure which shows the detailed structure of the personal aesthetic standard generation. It is a figure which shows the information which is stored in the aesthetic standard storage part. It is a figure which shows the user interface at the time of inputting the evaluation index. It is a figure which shows the display example of the evaluation index. It is a figure which shows the 1st example of an information processing system. It is a flowchart explaining the 1st example of processing of an information processing system. It is a figure which shows the 2nd example of an information processing system. It is a flowchart explaining the 2nd example of the processing of an information processing system. It is a figure which shows the 3rd example of an information processing system. It is a flowchart explaining the process at the time of evaluating the aesthetics of an image.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
(First Embodiment)
First, the first embodiment will be described. In the present embodiment, a case where the information processing device is a digital camera equipped with a function for evaluating the aesthetics of an image will be described as an example.
FIG. 1 is a diagram showing an example of a schematic configuration of a digital camera 100 (imaging device).
The image sensor 102 forms an image of the light that has passed through the image pickup optical system 101 (optical lens system) on the image pickup element 102, and converts the imaged light into an electric signal. The image sensor 102 has a photoelectric conversion element that converts an optical signal formed by an image formed on a light receiving surface into an electric signal for each light receiving pixel at a corresponding position. The A / D converter 103 converts the electric signal output from the image sensor 102 into a digital signal and inputs it to the image processing unit 104.

The system controller 109 includes a CPU, a ROM, a RAM, and the like, and controls the digital camera by executing a computer program stored in the ROM and the like. The image pickup optical system control unit 106 controls the operation of the image pickup optical system 101 based on an instruction from the system controller 109.

The image processing unit 104 performs various image processing on the image data output from the A / D converter 103. The image data that has undergone image processing is stored in the image recording medium 108 or displayed by the display unit 105. In this embodiment, the display unit 105 has a touch panel function. The storage unit 107 has, for example, an auxiliary storage device such as an HDD, and stores various types of information.
The digital camera can be realized by a known technique, and is not limited to the one shown in FIG. 1. Further, the digital camera only needs to be able to capture at least one of a moving image and a still image. Further, instead of the digital camera, for example, a mobile terminal provided with an imaging means may be used.

FIG. 2 is a diagram showing an example of a functional configuration of the digital camera 100. FIG. 3 is a flowchart illustrating an example of processing of the digital camera 100. 2 and 3 show only the part related to the function of acquiring the aesthetic standard among the functions of the digital camera 100. The aesthetic standard in the present embodiment is a judgment standard when the user judges the aesthetics (whether or not the image taken by the digital camera 100 is beautiful) of the image taken by the digital camera 100.

In FIG. 2, the digital camera 100 has a function input unit 201, an aesthetic evaluation unit 202, a personal aesthetic standard generation unit 203, and an aesthetic standard storage unit 204. The function input unit 201 is realized by using, for example, the system controller 109 and the storage unit 107. The aesthetic evaluation unit 202 and the personal aesthetic standard generation unit 203 are realized by using, for example, the system controller 109. The aesthetic standard storage unit 204 is realized by using, for example, the storage unit 107. FIG. 4 is a diagram showing an example of a detailed configuration of the aesthetic evaluation unit 202. FIG. 5 is a diagram showing an example of a detailed configuration of the personal aesthetic standard generation unit 203. FIG. 6 is a diagram showing an example of information stored in the aesthetic standard storage unit 204.

In the present embodiment, the digital camera 100 stores a general aesthetic standard 601 and a personal aesthetic standard 602 (first personal aesthetic standard 602a to nth personal aesthetic standard 602n). The general aesthetic standard is a judgment standard regarding the aesthetics of an image that can be agreed upon by an unspecified number of people. On the other hand, the personal aesthetic standard is a judgment standard regarding the aesthetics of an image peculiar to a specific person, and it does not matter whether or not an unspecified number of people can agree.

<Aesthetic evaluation function input process: S301>
The digital camera 100 of the present embodiment extracts a feature amount from an image in order to determine the aesthetics of the captured image. This feature quantity extraction method is roughly divided into two types: a method of "extracting a feature quantity designed by a human being" and a method of "a machine automatically extracts a feature quantity". As a method of "extracting a human-designed feature amount", for example, a local region or a global region in an image such as a color distribution, a color combination, a number of colors, an edge distribution, a high frequency component distribution, a contrast, an average saturation, and an average brightness value. There is a method to extract statistical information in the area. Such a method is described in Non-Patent Document 1.

On the other hand, as a method of "machine automatically extracting features", for example, a local general-purpose feature descriptor (SIFT, etc.) is applied to a local general-purpose feature (luminance gradient or color average / variance), and a global feature descriptor is applied. There is a method to obtain features by (codebook and Fisher vector, etc.). Further, as a method of "the machine automatically extracts the feature amount", there is a method of using the intermediate layer of the network learned by the neural network as the feature amount. The former is described in Non-Patent Document 2. The latter is described in Non-Patent Document 3. After extracting the feature amount, the digital camera 100 machine-learns the relationship between the feature amount and the teacher value (good / bad information or score information). Examples of the machine learning method include a support vector machine and a support vector regression.

There is a large amount of image data on the Internet. The digital camera 100 can obtain a high-precision general aesthetic standard by utilizing such big data. When a high-precision general aesthetic standard is established, the digital camera 100 automatically selects and sorts aesthetically good images from a large number of images stored in the storage unit 107. Can be done. Further, the digital camera 100 can display the aesthetic score in real time on the live view image displayed on the display unit 105. In this way, the user can take an aesthetically pleasing image.

On the other hand, among the users of the digital camera 100, it is assumed that some users are not satisfied with the general aesthetic standards alone and want to obtain aesthetically good images based on the personal aesthetic standards. As a specific example of personal aesthetic standards, there is "an aesthetic standard that makes a photograph taken by a professional photographer Mr. A have a high degree of aesthetics".

In the present embodiment, the digital camera 100 is equipped with general aesthetic standards as standard before shipment, and personal aesthetic standards are generated by machine learning after shipment.
Therefore, in S301, the function input unit 201 inputs a function for evaluating aesthetics. Since aesthetics is the beauty and beauty that "humans feel", it is necessary for humans to "teach" machines about aesthetics in order to evaluate it in machines. There is a rule-based method for this teaching. For example, taking a composition as an example, "the aesthetics are high if the rule of thirds is followed" or "the aesthetics are low if the composition is decapitated or skewered". Such rules are based on empirical rules and heuristics that human beings think are related to aesthetics. However, it is difficult to comprehensively extract such rules, and even if the rules can be comprehensively extracted, it is difficult to properly adjust the relationship between those rules and aesthetics. is there.

Therefore, in the present embodiment, a method of deriving the relationship between the feature amount of the image and the evaluation index related to aesthetics by machine learning is adopted. The evaluation index regarding aesthetics is, for example, a quality label or an aesthetic score. The quality label is information indicating whether or not the image is beautiful. Specifically, a "good label" is given to an image having high aesthetics, and a "bad label" is given to an image having low aesthetics. The aesthetic score is information obtained by scoring aesthetics. For example, the higher the score, the more beautiful the information can be used as the aesthetic score. For example, information such as "3 points out of 10 points" and "8 points out of 10 points" is given to the image as an aesthetic score.

In machine learning, teacher data (GT (Ground Truth) data) is required instead of rules. The teacher data is a pair of an image and an evaluation index (quality label or aesthetic score) related to the aesthetics of the image. Humans determine the evaluation index for the aesthetics of images. In this embodiment, machine learning called "supervised learning" is performed using this teacher data. As a learning device for supervised learning, a discriminator can be used when the teacher data is "image and pass / fail label", and a regression device is used when the teacher data is "image and aesthetic score". Can be done. An example of a discriminator is a support vector machine, and an example of a regressionr is a support vector regression.

In order to learn the learner using the teacher data, it is necessary to express the image by some feature quantity. As a first method for designing a feature amount, there is a method in which a human designs a feature amount that is likely to be related to aesthetics. As a second method, there is a method of designing a general feature amount containing a large amount of information while ignoring the relation with aesthetics. As a third method, there is a method of letting the machine design even the features. The first method is described in Non-Patent Document 1, the second method is described in Non-Patent Document 2, and the third method is described in Non-Patent Document 3.

The learner (discriminator or regressionr) can learn these relationships by inputting the feature amount of the image and the teacher data (evaluation index (good / bad label or aesthetic score) regarding the aesthetics of the image). .. When the learning is completed, a learning result called a "dictionary" is obtained, which becomes an "aesthetic standard" for the teacher data. The learner (identifier or regressionr) enters the learning complete state by reading the dictionary. When the learning device that reads the dictionary in this way inputs the feature amount extracted from the image to be evaluated, it outputs an evaluation index (good / bad label or aesthetic score) related to the aesthetics of the image. The learner outputs a "good / bad label" if it is a discriminator, and outputs an "aesthetic score" if it is a regression device.

From the above, in the present embodiment, in S301, the function input unit 201 inputs a learning device as a function for evaluating aesthetics. Specifically, the process of S301 is a process of mounting the learning device as software or a chip inside the digital camera 100. By doing so, the learner can be used with the digital camera 100. At this time, the format of the feature amount input to the learner (identifier or regressionr) is determined as a specification.

In the present embodiment, the function input unit 201 inputs a learning device for which a dictionary has not been read as a learning device. Dictionaries include dictionaries that correspond to general aesthetic standards and dictionaries that correspond to personal aesthetic standards. A dictionary corresponding to a general aesthetic standard is an example of information indicating a general aesthetic standard, and provides information that correlates an image feature amount with an evaluation index (good / bad label or aesthetic score) related to the aesthetics of the image. Including. A dictionary corresponding to a personal aesthetic standard is an example of information indicating a personal aesthetic standard, and provides information that correlates an image feature amount with an evaluation index (good / bad label or aesthetic score) related to the aesthetics of the image. Including. In the present embodiment, the case where the learning device is a support vector machine or a support vector regression will be described as an example, but the learning device is not limited to these, and the evaluation index regarding the aesthetics of the image is good or bad. Not limited to labels or aesthetic scores. The learner may be, for example, a neural network. If the learner is a neural network, the dictionary corresponding to the aesthetic criteria will be the weighting factor. The processing of S301 is performed before the shipment of the digital camera 100.

<General aesthetic standard input process: S302>
In this embodiment, the external information processing device of the digital camera 100 generates a dictionary corresponding to general aesthetic standards. In the following description, the information processing device external to the digital camera 100 will be referred to as an external device, if necessary. "General" refers to the general public, and "general aesthetic standards" are the overall and statistical aesthetic standards of the general public. In order to generate this, a large amount of image data labeled or scored by the general public is required as teacher data. Such a large amount of teacher data can be easily collected in the recent Internet big data era.

The external device has the same learning device as the learning device input by the function input unit 201. The external device collects a large amount of image data labeled or scored by the general public as teacher data for general aesthetic standards, and inputs the teacher data for general aesthetic standards to the learner. The above-mentioned machine learning is performed. The image data included in the teacher data for general aesthetic standards is featured when input to the learner. When the learning is completed, as a learning result, a dictionary corresponding to the general aesthetic standard is obtained, and this becomes the general aesthetic standard 601.

In S302, the digital camera 100 inputs the general aesthetic standard 601 from an external device and stores it in the aesthetic standard storage unit 204. The digital camera 100 causes the learner input in S301 to read the general aesthetic standard 601. The digital camera 100 obtains an aesthetic evaluation according to the general aesthetic standard 601 for the image data by inputting the image data to be evaluated for aesthetics into the learning device into which the general aesthetic standard 601 is read. Of the processes of S302, at least the process of inputting the general aesthetic standard 601 is performed before the shipment of the digital camera 100. Therefore, the general aesthetic standard 601 is standard equipment on the digital camera 100. In S302, the digital camera 100 may input a learner into which a dictionary corresponding to a general aesthetic standard is read.

<Image / incidental information input process: S303>
Next, in S303, the image / incidental information input unit 501 inputs the image / incidental information. Specifically, after the user takes a picture using the digital camera 100, the photographed image and incidental information of the image are input to the digital camera 100. The incidental information includes an evaluation index (good / bad label or aesthetic score) regarding the aesthetics of the image that the user thinks. The pair of this image and the evaluation index regarding the aesthetics of the image becomes the teacher data for the personal aesthetic standard.

FIG. 7 is a diagram showing an example of a user interface when inputting an evaluation index related to the aesthetics of an image. When the user performs a predetermined operation on the digital camera 100 while the live view image is displayed on the display unit 105, the image / incidental information input unit 501 superimposes the live view image on the operation unit 701 and displays the operation unit 701. It is displayed on 105. The operation unit 701 is a graphical user interface for inputting an evaluation index regarding the aesthetics of the live view image. The user operates the operation unit 701 by using the function of the touch panel of the display unit 105. Based on this operation, the image / incidental information input unit 501 inputs an evaluation index (good / bad label or aesthetic score) relating to the aesthetics of the image displayed on the display unit 105. In the example shown in FIG. 7, it is shown that the user determines that the image displayed on the display unit 105 is 75 points out of 100 points.

The incidental information may include camera information in addition to the evaluation index (good / bad label or aesthetic score) relating to the aesthetics of the image. The camera information includes information on the shooting conditions of the digital camera 100 when the image is taken. For example, the exposure value and the presence / absence of strobe light emission are included in the camera information. In addition, the incidental information may include environmental conditions (for example, temperature, humidity, weather, ambient illuminance). As described above, the incidental information may include information that affects the aesthetics of the image taken by the digital camera 100. When the incidental information includes information other than the evaluation index (good / bad label or aesthetic score) related to the aesthetics of the image, the information can also be used as teacher data for personal aesthetic standards.

<Personal aesthetic standard generation process: S304>
In this process, the digital camera 100 creates a personal aesthetic standard. In the present embodiment, "personal" refers to a user of the digital camera 100. In contrast to the general aesthetic standard, which is the overall and statistical aesthetic standard of the general public, the "personal aesthetic standard" is the personal aesthetic standard of the user of the digital camera 100.

First, the feature extraction unit 502 extracts the feature amount from the image data included in the teacher data for the personal aesthetic standard input in S303. The learning unit 503 uses the feature amount, an evaluation index (good / bad label or aesthetic score) included in the teacher data for personal aesthetic criteria, and a learning device (discriminator or regression device) input in S301. , Perform the machine learning described above. When the machine learning is completed, as a learning result, a dictionary corresponding to the personal aesthetic standard is obtained, which becomes the personal aesthetic standard 602. By generating such a personal aesthetic standard 602 using each of a plurality of teacher data groups for the personal aesthetic standard, a plurality of personal aesthetic standards (first personal aesthetic standard 602a, ... , Nth personal aesthetic criteria 602n) is obtained.

In this case, for example, when the user inputs the teacher data group for the personal aesthetic standard, the user inputs the identification information of the teacher data group into the digital camera 100. The personal aesthetic standard output unit 504 correlates the personal aesthetic standard 602 generated by the learning unit 503 with the teacher data group for the personal aesthetic standard and the identification information, and the aesthetic standard storage unit 204. Store in. In this embodiment, only one personal aesthetic standard 602 is required. When the personal aesthetic criteria 602 of a user other than the user of the digital camera 100 is input to the digital camera 100, a plurality of personal aesthetic criteria 602 are stored in the aesthetic criteria storage unit 204. The digital camera 100 causes the learning device input in S301 to read the personal aesthetic standard 602 (any one of the first personal aesthetic standard 602a, ..., The nth personal aesthetic standard 602n). .. By inputting the image data to be evaluated for aesthetics into the learning device into which the personal aesthetic standard 602 is read, the aesthetic evaluation based on the personal aesthetic standard for the image data can be obtained.
The processing of S303 and S304 is performed after the digital camera 100 is shipped.

<Evaluation of aesthetics for the image to be evaluated>
When at least one of the general aesthetic standard 601 and the personal aesthetic standard 602 is stored in the aesthetic standard storage unit 204, the aesthetic evaluation unit 202 determines the image data (good / bad label and aesthetic score unknown) to be evaluated for aesthetics. It is possible to evaluate the aesthetics of (image data). First, the image input unit 401 inputs image data to be evaluated for aesthetics. Further, the image input unit 401 inputs information for specifying which aesthetic standard is used for evaluation of the image data based on the operation of the user's digital camera 100. When a plurality of personal aesthetic criteria 602 (first personal aesthetic criteria 602a, ..., nth personal aesthetic criteria 602n) are stored in the aesthetic criteria storage unit 204, the user can use the above-mentioned identification information. Can be used to select one of a plurality of personal aesthetic criteria. When there is one personal aesthetic standard 602 stored in the aesthetic standard storage unit 204, it is not necessary to input information for specifying which aesthetic standard is to be evaluated or to select one of a plurality of personal aesthetic standards. become.

Next, the feature extraction unit 402 extracts the feature amount in the format determined by the specifications from the image data input by the image input unit 401. Next, the evaluation unit 403 selects and reads out the aesthetic standard designated by the user from the general aesthetic standard 601 and the personal aesthetic standard 602, and causes the learning device input by the function input unit 201 to read the aesthetic standard. The evaluation unit 403 inputs the feature amount of the image into the learner. Then, the learner outputs an evaluation index (good / bad label or aesthetic score) regarding the aesthetics of the image. The aesthetics output unit 404 displays the evaluation index (good / bad label or aesthetic score) obtained by the learning device on the display unit 105. For example, when the image input unit 401 inputs a live view image displayed on the display unit 105 at the time of shooting, the display unit 105 superimposes an evaluation index on the aesthetics of the live view image on the live view image and displays it in real time. be able to. The aesthetics output unit 404 performs at least one of storage in the storage unit 107 and transmission to an external device in place of or in addition to displaying an evaluation index (good / bad label or aesthetic score) relating to the aesthetics of the image. You may. The aesthetics output unit 404 can store the image data input by the image input unit 401 and the evaluation index related to the aesthetics of the image data in association with each other. In this way, for example, the digital camera 100 can sort the captured image group according to the personal aesthetic standard 602, and display the image group on the display unit 105 in the sorted order.

In addition, the user can select a plurality of aesthetic criteria when inputting image data by the image input unit 401. In this case, the processing of the feature extraction unit 402, the evaluation unit 403, and the aesthetic output unit 404 described above is repeatedly performed using each of the plurality of aesthetic criteria. As a result, an evaluation index (good / bad label or aesthetic score) related to aesthetics based on a plurality of aesthetic criteria can be obtained for the same image data.

With reference to FIG. 8, it is a figure explaining an example of the usage method of the general aesthetic standard 601 installed as standard in the digital camera 100, and the personal aesthetic standard 602 generated by the method of this embodiment. FIG. 8 is a display example of the evaluation index (aesthetic score) in various display modes displayed on the live view image. FIG. 8A is an example of displaying the aesthetic score according to the general aesthetic standard 601. The digital camera 100 has a mode for displaying only the general aesthetic standard 601. The aesthetics output unit 404 displays as shown in FIG. 8A by the user selecting the mode even when the personal aesthetics standard 602 does not exist or exists in the digital camera 100. be able to.

FIG. 8B is an example of the display of the aesthetic score according to one personal aesthetic criterion 602 selected by the user. The digital camera 100 has a mode for displaying only one personal aesthetic standard 602. The aesthetics output unit 404 can perform the display as shown in FIG. 8B when the user selects the mode and selects any one of the plurality of personal aesthetic criteria 602. The aesthetics output unit 404 automatically selects and displays the personal aesthetic standard 602 when the user selects the mode when one personal aesthetic standard 602 exists.

FIG. 8C is an example of displaying the aesthetic score according to all the aesthetic standards (general aesthetic standard 601 and personal aesthetic standard 602) stored in the aesthetic standard storage unit 204. In FIG. 8C, an aesthetic score based on a general aesthetic standard (General Score), an aesthetic score based on a personal aesthetic standard (Personal1 Score), and an aesthetic score based on a personal aesthetic standard (Personal2 Score) are displayed. The digital camera 100 has a mode for displaying all aesthetic standards. The aesthetics output unit 404 can perform the display as shown in FIG. 8C when the user selects the mode. With such a display, the user can confirm the evaluation indexes based on all the aesthetic standards stored in the aesthetic standard storage unit 204. The aesthetics output unit 404 may display a plurality of aesthetic standards selected by the user instead of all the aesthetic standards stored in the aesthetic standard storage unit 204.

FIG. 8D is an example of an integrated aesthetic score display. In FIG. 8D, in addition to the aesthetic score shown in FIG. 8C, an aesthetic score (Merged Score) in which a plurality of aesthetic scores are integrated is displayed. The digital camera 100 has a mode for displaying an aesthetic score in which a plurality of aesthetic scores are integrated. When the user selects the mode, the evaluation unit 403 derives an aesthetic score that integrates the individual aesthetic scores in addition to the individual aesthetic scores, and the aesthetic output unit 404 outputs those aesthetic scores. .. An aesthetic score that integrates a plurality of aesthetic scores is derived using, for example, a weight for each aesthetic standard specified by the user and an aesthetic score according to each aesthetic standard. In the example shown in FIG. 8 (d), the weight of the aesthetic score (General Score) according to the general aesthetic standard is 50%. In addition, the weight of the aesthetic score according to the personal aesthetic standard (Personal1 Score) is 30%. In addition, the weight of the aesthetic score (Personal2 Score) based on the personal aesthetic standard is 20%. Therefore, the evaluation unit 403 derives, for example, 56 points (= 0.5 × 70 + 0.3 × 50 + 0.2 × 30) as the aesthetic score (Merged Score) based on the integrated aesthetic criteria.

As described above, in the present embodiment, the digital camera 100 uses information in which the image taken by the digital camera 100 and the evaluation index related to the aesthetics of the image are associated with each other as teacher data for personal aesthetic standards. Use to perform machine learning. The digital camera 100 uses the personal aesthetic standard 602 (dictionary) obtained as a result of this machine learning to derive an evaluation index based on the personal aesthetic standard for the image to be evaluated for aesthetics. Therefore, as a judgment standard regarding the aesthetics of a specific person, it is possible to obtain a judgment standard from the viewpoint of "beauty" or "beauty" for the image.

(Second Embodiment)
Next, the second embodiment will be described. In the first embodiment, the case where the digital camera 100 generates the personal aesthetic standard 602 and the information processing system is constructed by one information processing device (digital camera 100) has been described as an example. On the other hand, in the present embodiment, there is an example in which an external device generates a personal aesthetic standard 602, the digital camera 100 inputs it, and an information processing system is constructed using a plurality of information processing devices. It will be explained by listing in. As described above, the present embodiment and the first embodiment are mainly different in configuration and processing due to the difference in the subject that generates the personal aesthetic standard 602. Therefore, in the description of the present embodiment, detailed description of the same parts as those of the first embodiment will be omitted by adding reference numerals to FIGS. 1 to 8.

FIG. 9 is a diagram showing an example of an information processing system according to the present embodiment. FIG. 10 is a flowchart illustrating an example of processing of the information processing system according to the present embodiment.
<Aesthetic evaluation function input process: S1001>
Since this step is the same as step S301 of FIG. 3, detailed description of this step will be omitted.
<General aesthetic standard input process: S1002>
Since this step is the same as S302 in FIG. 3, a detailed description of this step will be omitted.

<Image / incidental information input process: S1003>
In the first embodiment, after the digital camera 100 captures an image, a teacher data group for personal aesthetic criteria (the captured image and an evaluation index (good / bad label or aesthetic) regarding the aesthetics of the image that the user thinks). Enter the pair group) with the score). On the other hand, in the present embodiment, the information processing apparatus 901 inputs the teacher data group for the personal aesthetic standard based on the operation of the user. The information processing device 901 can upload an image taken by the user and input and set an evaluation index (good / bad label or aesthetic score) regarding the aesthetics of the uploaded image. In addition, the camera manufacturer prepares a default image on the site, and the information processing device 901 accesses the site based on the operation by the user, and sets an evaluation index regarding the aesthetics of the image on the site. May be good. In this step, if the user has obtained information on what kind of image is evaluated as having high aesthetics and what kind of image is evaluated as having low aesthetics, any method can be used for personal aesthetics. The teacher data group for the reference may be input. The information processing device 901 has, for example, a CPU, a ROM, a RAM, an HDD, and various interfaces. A computer display is connected to the information processing device 901.

The camera manufacturer may collect a usage fee (a fee for creating the personal aesthetic standard 602) from the user. Further, the camera maker may provide the personal aesthetic standard 602 for a predetermined number of times free of charge as a purchase privilege of the digital camera 100. The camera manufacturer may also be able to provide the user with personal aesthetic criteria 602 completely free of charge.

<Personal aesthetic standard generation process: S1004>
In the first embodiment, the digital camera 100 main body generates a personal aesthetic standard 602. However, the computational resources of the digital camera 100 itself are generally poorer than those of a PC or the like, and there is a large difference in terms of processing time and processing capacity. Therefore, in this step, the personal aesthetic standard 602 is not generated by the digital camera 100 main body, but the personal aesthetic standard 602 is generated by the server 902 prepared by the camera manufacturer. For example, this process may be started by the user performing a predetermined operation on the information processing apparatus 901 after the input of the teacher data group for the personal aesthetic standard by S1003 is completed. For example, the user can start the generation of the personal aesthetic standard 602 by pressing the button instructing the start of the generation of the personal aesthetic standard 602 among the buttons displayed on the above-mentioned site provided by the camera manufacturer. Can be instructed.

When instructed to start generating the personal aesthetic criteria 602, the server 902 starts creating the personal aesthetic criteria 602. At this time, the teacher data group for the personal aesthetic standard is input to the server 902. The server 902 has the same learner as the learner input to the digital camera 100 in S1001. The server 902 extracts the feature amount of the image data included in the teacher data group for the personal aesthetic standard. Then, the server 902 includes the extracted feature amount, the evaluation index (good / bad label or aesthetic score) regarding the aesthetics of the image included in the teacher data group for the personal aesthetic criteria, and the learner (discriminator or regression device). Is used to perform the machine learning described in the first embodiment. When the learning is completed, as a learning result, a dictionary corresponding to the personal aesthetic standard is obtained, which becomes the personal aesthetic standard 602. When the generation of the personal aesthetic standard 602 is completed, the server 902 may transmit information indicating that to the information processing device 901. For example, the server 902 may display a message to the effect that the generation of the personal aesthetic criteria 602 has been completed in a predetermined area of the site described above. If the server 902 takes a long time to generate the personal aesthetic standard 602, the server 902 notifies the user by sending an e-mail indicating that to the e-mail address registered in the server 902. You may. The server 902 has, for example, a CPU, a ROM, a RAM, an HDD, and various interfaces.

<Personal aesthetic standard input process: S1005>
In the first embodiment, since the digital camera 100 main body generates a personal aesthetic standard, this step is not necessary. However, in the present embodiment, since the server 902 (information processing device other than the digital camera 100) generates the personal aesthetic standard 602, the digital camera 100 needs to input this. In this step, the user first acquires the personal aesthetic standard 602 generated in S1004 from the camera manufacturer's site. For example, when the camera manufacturer inputs information to the camera manufacturer's site that the user wishes to mail the personal aesthetic standard 602, the camera maker stores the personal aesthetic standard 602 in the recording medium and mails it to the user. can do. Further, the camera maker may make the personal aesthetic standard 602 available for download on the camera maker's site, and the information processing apparatus 901 may download the personal aesthetic standard 602 from the above-mentioned site. When the user acquires the personal aesthetic standard 602, the user inputs the personal aesthetic standard 602 into the digital camera 100. The input to the digital camera 100 of the personal aesthetic standard 602 may be performed by, for example, a recording medium, or may be performed by wired communication or wireless communication.
By going through the above steps, the aesthetics of the image can be evaluated by the user's personal aesthetic criteria using the digital camera 100 as in the first embodiment.

(Third Embodiment)
Next, a third embodiment will be described. In the first and second embodiments, the case where the personal aesthetic standard 602 of the user of the digital camera 100 is used has been described as an example. On the other hand, in the present embodiment, a case where the digital camera 100 uses the personal aesthetic standard 602 of a professional photographer as a third party other than the user of the digital camera 100 will be described as an example. Specifically, the camera maker asks a professional photographer to create the aesthetic standard of the professional photographer as the personal aesthetic standard 602. The camera maker deploys the created personal aesthetic standard 602 so that the user of the digital camera 100 can acquire it. The user of the digital camera 100 acquires the personal aesthetic standard 602 from the user, and causes the digital camera 100 owned by the user to input the personal aesthetic standard 602. As described above, the present embodiment and the second embodiment are mainly different in the configuration and processing due to the difference in the input main of the teacher data for the personal aesthetic standard. Therefore, in the description of the present embodiment, detailed description of the same parts as those of the first and second embodiments will be omitted by adding reference numerals to FIGS. 1 to 10.

FIG. 11 is a diagram showing an example of an information processing system according to the present embodiment. FIG. 12 is a flowchart illustrating an example of processing of the information processing system according to the present embodiment. In FIG. 11, each of the professional photographers A, B, C, and other users has the information processing device 901 described in the second embodiment, and the camera maker has the second embodiment. It is assumed to have the described server 902.

<Aesthetic evaluation function input process: S1201>
Since this step is the same as step S301 of FIG. 3, detailed description of this step will be omitted.

<General aesthetic standard input process: S1202>
Since this step is the same as S302 in FIG. 3, a detailed description of this step will be omitted.

<Image / incidental information input process: S1203>
In the first embodiment and the second embodiment, the input method of the teacher data for the personal aesthetic standard is different, but the input owner is the user of the digital camera 100. On the other hand, in the present embodiment, the input person of the teacher data for the personal aesthetic standard is a professional photographer. The camera maker asks professional photographers A, B, and C, and professional photographers A, B, and C input teacher data for personal aesthetic standards based on their own aesthetic standards. For example, as described in the first embodiment, a professional photographer can take a picture using his / her own digital camera, and then input an evaluation index regarding the aesthetics of the taken image into the digital camera. .. Further, as described in the second embodiment, the professional photographer may input the image and the evaluation index related to the aesthetics of the image by using the site on the Internet prepared by the camera manufacturer. .. When the camera manufacturer requests a professional photographer to input teacher data for personal aesthetic standards, the camera manufacturer may pay the professional photographer a consideration (contract fee).

<Personal aesthetic standard generation process: S1204>
In the first embodiment, the digital camera 100 main body generates the personal aesthetic standard 602. In the second embodiment, the camera manufacturer's server 902 is used to generate the personal aesthetic criteria 602. In the present embodiment, the subject of generating the personal aesthetic standard 602 may be the digital camera 100 or the camera manufacturer's server 902. That is, the digital camera 100 may generate the personal aesthetic standard 602, or the server 902 may generate the personal aesthetic standard 602. Alternatively, large clusters may be used to generate personal aesthetic criteria 602. In this way, if the camera maker can acquire the teacher data group for the personal aesthetic standard of the professional photographer and perform machine learning using this teacher data group and the learner, the personal aesthetic standard 602 can be generated. The subject does not matter. When the machine learning is completed, as a result of learning, a dictionary corresponding to the personal aesthetic standard is obtained, which becomes the personal aesthetic standard 602 of a professional photographer.

<Personal aesthetic standard deployment process: S1205>
In this step, the professional photographer's personal aesthetic standard 602 generated in S1204 is deployed so that it can be acquired by a general user of the digital camera 100. For example, the camera manufacturer prepares a site similar to the site described in the section of S1003 of the second embodiment. The camera maker deploys a professional photographer's personal aesthetic standard 602 so that a general user of the digital camera 100 can obtain the professional photographer's personal aesthetic standard 602 from this site. If there are personal aesthetic standards 602 for a plurality of professional photographers, the camera manufacturer deploys the personal aesthetic standards 602 for the plurality of professional photographers. As in the second embodiment, the camera manufacturer may collect a usage fee (usage fee of the professional photographer's personal aesthetic standard 602) from the user. Further, the camera maker may provide the personal aesthetic standard 602 for a predetermined number of times free of charge as a purchase privilege of the digital camera 100. The camera manufacturer may also be able to provide the user with personal aesthetic criteria 602 completely free of charge.

<Personal aesthetic standard input process: S1206>
First, the user operates the information processing device 901 to acquire the personal aesthetic standard 602 of a professional photographer from the camera manufacturer's site. As a method for acquiring the personal aesthetic standard 602 of a professional photographer, for example, the method described in the second embodiment (S1005) can be adopted. When the user acquires the personal aesthetic standard 602 of a professional photographer, the user causes the digital camera 100 to input the personal aesthetic standard 602. The input to the digital camera 100 of the personal aesthetic standard 602 may be performed by, for example, a recording medium, or may be performed by wired communication or wireless communication.

Through the above steps, the user can evaluate the aesthetics of an image using his / her own digital camera 100 according to the personal aesthetic criteria of a professional photographer. For example, a general user can display a live view image of his / her digital camera 100 in real time with an aesthetic score obtained by a learner learned from teacher data of a professional photographer. In this way, it becomes possible to take a picture close to the picture taken by the professional photographer. Further, the digital camera 100 is adopted by a professional photographer if a plurality of images are sorted by the aesthetic score obtained by using a learning device learned from teacher data for a certain professional photographer's personal aesthetic standard. Such an image can be displayed at the top.

(Fourth Embodiment)
Next, a fourth embodiment will be described. In the third embodiment, a case where the digital camera 100 uses the personal aesthetic standard 602 of a professional photographer as a third party other than the user of the digital camera 100 has been described as an example. On the other hand, in the present embodiment, a case where the digital camera 100 uses the personal aesthetic standard 602 of a user other than a professional photographer as a third party other than the user of the digital camera 100 will be described as an example. Specifically, the camera maker prepares a site where the personal aesthetic standard can be downloaded, and the user A of the digital camera 100 uploads the personal aesthetic standard 602 created by the user A to the site. User B of another digital camera 100 downloads the uploaded personal aesthetic criteria 602 of user A from the site. As described above, the present embodiment and the third embodiment are mainly different in the configuration and processing due to the difference in the input main of the teacher data for the personal aesthetic standard. Therefore, in the description of the present embodiment, detailed description of the same parts as those of the first to third embodiments will be omitted by adding reference numerals to FIGS. 1 to 12.

FIG. 13 is a diagram showing an example of an information processing system according to the present embodiment. Since each step of the processing of the information processing system in the present embodiment is the same as the flowchart of FIG. 12, here, the specific processing of each step will be described using the flowchart of FIG. In FIG. 13, each semi-professional photographer A, B, C, and the other user has the information processing apparatus 901 described in the second embodiment, and the camera maker describes it in the second embodiment. It shall have a server 902.

<Aesthetic evaluation function input process: S1201>
Since this step is the same as step S301 of FIG. 3, detailed description of this step will be omitted.

<General aesthetic standard input process: S1202>
Since this step is the same as S302 in FIG. 3, a detailed description of this step will be omitted.

<Image / incidental information input process: S1203>
In a third embodiment, the input source of teacher data for personal aesthetic standards is a professional photographer. On the other hand, in the present embodiment, the input master of the teacher data for the personal aesthetic standard is a general user of the digital camera 100. The core user base among the general users is semi-professional and high-end photographers who are supported by a specific community. Many photographers do it as a hobby, even if they don't work in photography. Taking railroads and mountaineering as an example of their hobbies, photographs are frequently exchanged within the railroad community and mountaineering communities, and through them, members of the community who like the pictures of specific people come out. .. In such a situation, there is a system in which the personal aesthetic standard 602 generated by using the teacher data for the personal aesthetic standard of the supported user is delivered to the user who supports the user. It is convenient.

In this step, for example, a user supported by other general users (for example, semi-professional photographers A, B, C) inputs teacher data for personal aesthetic standards based on his / her own aesthetic standards. .. For example, as described in the first embodiment, the user can take a picture using his / her own digital camera, and then input an evaluation index regarding the aesthetics of the taken image into the digital camera. Further, as described in the second embodiment, the user may input an image and an evaluation index relating to the aesthetics of the image by using a site on the Internet prepared by the camera manufacturer.

In the third embodiment, it was explained that when the camera maker asks the professional photographer to input the teacher data for the personal aesthetic standard, the professional photographer may be paid a fee. In the present embodiment, the opposite case is also conceivable. That is, a general user pays a registration fee when uploading teacher data for his or her personal aesthetic standards to the camera manufacturer's site. After that, when another general user pays the usage fee to the camera maker and the personal aesthetic standard 602 generated by using the teacher data for the personal aesthetic standard of the certain user is downloaded, the camera maker causes the camera maker. Part of it is paid to the user as an incentive. In addition, instead of the system that charges a fee in this way, a completely free system may be used.

<Personal aesthetic standard generation process: S1204>
Since this step is the same as S1204 in FIG. 12, detailed description of this step will be omitted. However, in the third embodiment, this step produces a professional photographer's personal aesthetic criteria 602. On the other hand, in the present embodiment, the personal aesthetic standard 602 of a general user (for example, semi-professional photographers A, B, C) of the digital camera 100 is generated.

<Personal aesthetic standard deployment process: S1205>
Since this step is the same as S1205 in FIG. 12, detailed description of this step will be omitted. However, in the third embodiment, this step produces a professional photographer's personal aesthetic criteria 602. On the other hand, in the present embodiment, the personal aesthetic standard 602 of a general user (for example, semi-professional photographers A, B, C) of the digital camera 100 is deployed.

<Personal aesthetic standard input process: S1206>
Since this step is the same as S1205 in FIG. 12, detailed description of this step will be omitted. However, in the third embodiment, by this step, a general user acquires the personal aesthetic standard 602 of a professional photographer from the site of the camera manufacturer. On the other hand, in the present embodiment, the general user acquires the personal aesthetic standard 602 of another general user that he / she supports from the site of the camera manufacturer.

By going through the above steps, the same effect as that described in the third embodiment can be obtained. That is, the user can use his / her own digital camera 100 to evaluate the aesthetics of the image according to the personal aesthetic criteria of other general users that he / she supports. For example, a general user can display a live view image of his / her digital camera 100 in real time the aesthetic score by the learner learned by the teacher data of another general user supported by the user. In this way, the user of the digital camera 100 can take a picture close to a picture taken by another user supported by the user. Further, if the digital camera 100 sorts a plurality of images according to the aesthetic score by the learner learned from the teacher data of a certain user, the image to be adopted by the user can be displayed at a higher level.

(Fifth Embodiment)
Next, a fifth embodiment will be described. In the present embodiment, a specific example of a method of deriving and displaying an evaluation index (aesthetic score) related to the aesthetics of an image to be evaluated for aesthetics by using various aesthetic criteria existing in the digital camera 100 will be described. .. That is, a specific example of the processing described with reference to FIG. 8 in the first embodiment will be described. The method of this embodiment can be applied to any of the first to fourth embodiments.

FIG. 14 is a flowchart illustrating an example of processing of the digital camera 100 when deriving and displaying an evaluation index (aesthetic score) relating to aesthetics of an image to be evaluated for aesthetics.
<Personal aesthetic standard presence / absence judgment process: S1401>
The evaluation unit 403 determines whether or not the digital camera 100 has the personal aesthetic standard 602. Although the digital camera 100 is equipped with the general aesthetic standard 601 as standard, there is a case where the digital camera 100 does not have another personal aesthetic standard 602. Therefore, the evaluation unit 403 determines whether or not the personal aesthetic standard 602 exists in the digital camera 100. As a result of this determination, if the personal aesthetic standard 602 exists in the digital camera 100, the process proceeds to S1402. On the other hand, when the personal aesthetic standard 602 does not exist in the digital camera 100, the process proceeds to S1403a described later.

<Evaluation method selection process: S1402>
In S1402, the evaluation unit 403 selects a method of evaluating aesthetics using one or more personal aesthetic criteria 602 existing in the digital camera 100, based on the result of the user's operation on the digital camera 100. In S1401, when it is determined that the personal aesthetic standard exists in the digital camera 100, there is a degree of freedom in how to evaluate and display the existing personal aesthetic standard. As this degree of freedom, in the present embodiment, the evaluation unit 403 uses any one of the evaluation methods of "alternative evaluation", "all evaluation", or "integrated evaluation" based on the operation of the digital camera 100 by the user. To select. When selecting "alternative evaluation", the evaluation unit 403 also selects which personal aesthetic criteria to use based on the result of the operation of the digital camera 100 by the user.

<Evaluation process: S1403a to S1403d>
Next, the evaluation unit 403 evaluates the aesthetics of the image data input by the image input unit 401 according to the determination results of S1401 and S1402. First, if it is determined in S1401 that the personal aesthetic standard 602 does not exist in the digital camera 100, the process proceeds to S1403a. In S1403a, the evaluation unit 403 inputs the feature amount of the image data input by the image input unit 401 into the learner into which the general aesthetic standard 601 is read. The evaluation unit 403 obtains an aesthetic score according to the general aesthetic standard 601 as an aesthetic score of the image data input by the image input unit 401 from the learning device.

If "alternative evaluation" is selected in S1402, the process proceeds to S1403b. In S1403b, the evaluation unit 403 inputs the feature amount of the image data input by the image input unit 401 into the learning device into which one personal aesthetic standard 602 selected by the user is read. The evaluation unit 403 obtains the aesthetic score according to the personal aesthetic standard 602 selected by the user as the aesthetic score of the image data input by the image input unit 401 from the learning device.

If "all" is selected in S1402, the process proceeds to S1403c. In S1403c, the evaluation unit 403 inputs the feature amount of the image data input by the image input unit 401 into the learning device that has read the general aesthetic standard 601 stored in the aesthetic standard storage unit 204. The evaluation unit 403 obtains an aesthetic score according to the general aesthetic standard 601 as an aesthetic score of the image data input by the image input unit 401 from the learning device. Further, the evaluation unit 403 inputs the feature amount of the image data input by the image input unit 401 into the learning device that has read one of the personal aesthetic standards 602 stored in the aesthetic standard storage unit 204. .. The evaluation unit 403 obtains an aesthetic score according to the personal aesthetic standard 602 as an aesthetic score of the image data input by the image input unit 401 from the learning device. The evaluation unit 403 derives the aesthetic score according to the personal aesthetic standard 602 for all of the personal aesthetic standards 602 stored in the aesthetic standard storage unit 204.

If "integration" is selected in S1402, the process proceeds to S1403d. In S1403d, the evaluation unit 403 derives an integrated aesthetic score in addition to the processing of S1403c. For example, in S1403d, it is assumed that the evaluation unit 403 derives an aesthetic score based on a general aesthetic standard (General Score) and an aesthetic score based on two personal aesthetic standards (Personal1 Score and Personal2 Score). At that time, the user is asked how much each aesthetic score is emphasized. Therefore, the user operates the digital camera 100 and inputs the weight for each aesthetic score. The user, for example, inputs 50% as a weight for the aesthetic score (General Score) according to the general aesthetic standard. Further, the user inputs 30% and 20% as weights for the aesthetic scores (Personal1 Score and Personal2 Score) according to the two personal aesthetic criteria, respectively. Then, the evaluation unit 403 derives the weighted linear sum of each aesthetic score as an integrated aesthetic score.

<Evaluation value display process: S1404a to S1404d>
Next, in S1404a, S1404b, S1404c, and S1404d, the aesthetics output unit 404 displays the aesthetic scores derived in S1403a, S1403b, S1403c, and S1403d on the display unit 105, respectively. For example, as shown in FIG. 8, the aesthetics output unit 404 displays the aesthetic score on the live view screen displayed on the display unit 105.

FIG. 8A is a display example in the process of S1404a. FIG. 8A shows that the aesthetic score according to the general aesthetic standard 601 is 70 points out of 100 points. FIG. 8B is a display example in the process of S1404b. FIG. 8B shows that the aesthetic score according to the personal aesthetic standard 602 selected by the user is 50 points out of 100 points. FIG. 8C is a display example in the process of S1404c. In FIG. 8 (c), the aesthetic score according to the general aesthetic standard 601 is 70 points out of 100 points, and the aesthetic score according to the personal aesthetic criteria 602 (Personal1 and Personal2) is 50 points and 30 points out of 100 points, respectively. Indicates that there is. FIG. 8D is a display example obtained by the processing of S1404d. In FIG. 8D, in addition to the display in FIG. 8C, it is shown that the integrated aesthetic score is 56 points (= 0.5 × 70 + 0.3 × 50 + 0.2 × 30) out of 100 points. ..

By going through the above steps, when a plurality of aesthetic standards exist inside the digital camera 100, evaluation and display suitable for the user's request can be performed. In addition, by using an evaluation index that integrates each evaluation index (aesthetic score), for example, an ensemble evaluation within a community can be calculated. Specifically, the personal aesthetic criteria 602 of each member in the community are collected, the weights are set so that all the weights are equivalent, and the integrated evaluation is performed. This will provide a personal aesthetic standard 602 as a community-wide trend. At this time, when the number of personal aesthetic criteria 602 becomes enormous, only the integrated evaluation index (aesthetic score) can be displayed.

(Sixth Embodiment)
Next, the sixth embodiment will be described. In this embodiment, an example of a method for deriving the personal aesthetic standard 602 when the number of teacher data for the personal aesthetic standard is small will be described. The method of this embodiment can be applied to any of the first to fifth embodiments.

The learning unit 503 determines whether or not the number of teacher data for personal aesthetic standards input by the image / incidental information input unit 501 is equal to or less than a predetermined number. As a result of this determination, when the number of teacher data for personal aesthetic standards input by the image / incidental information input unit 501 is not less than or equal to a predetermined number, the learning unit 503 shall be as described in the first embodiment. To derive the personal aesthetic standard 602. On the other hand, when the number of teacher data for personal aesthetic standards input by the image / incidental information input unit 501 is less than or equal to a predetermined number, the learning unit 503 uses the general aesthetic standard 601 as a base for personal aesthetics. Additional learning is performed using the teacher data for the standard. For example, the learning unit 503 has a feature amount extracted from an image included in the teacher data for the personal aesthetic standard, an evaluation index regarding the aesthetics of the image included in the teacher data for the personal aesthetic standard, and a general index. Machine learning is performed using a learner that has read the aesthetic standard 601.
In this way, the personal aesthetic standard 602 can be derived even when the number of teacher data for the personal aesthetic standard is small.

(7th Embodiment)
Next, a seventh embodiment will be described. In this embodiment, an example of a method of clustering a plurality of users and deriving a personal aesthetic standard 602 for each clustered user will be described. The method of this embodiment can be applied to any of the first to sixth embodiments. The user may include the professional photographer described in the third embodiment and the semi-professional photographer / hiama photographer described in the fourth embodiment.

The image / incidental information input unit 501 inputs teacher data for personal aesthetic standards of each user of the plurality of digital cameras 100.
The personal aesthetic standard generation unit 203 clusters a plurality of users by using the teacher data for the personal aesthetic standards of the plurality of users. For example, the personal aesthetic criteria generation unit 203 can cluster a plurality of users using a decision tree. The explanatory variables of the decision tree are represented using, for example, an evaluation index (good / bad label or aesthetic score) relating to the feature amount of the image and the aesthetics of the image. The objective variable of the decision tree is the cluster to which the user belongs. The method of clustering a plurality of users is not limited to the method using a decision tree, and may be, for example, a method using a neural network.

The feature extraction unit 502 extracts a feature amount from an image included in teacher data for personal aesthetic criteria of a plurality of users belonging to the same cluster. The learning unit 503 includes the feature amount of the image included in the teacher data for the personal aesthetic criteria of the plurality of users, the evaluation index (good / bad label or aesthetic score) regarding the aesthetics determined by each user for the image, and the aesthetic score. Machine learning is performed using a learner. When this machine learning is completed, as a learning result, a dictionary corresponding to the personal aesthetic standard is obtained for each cluster, and this becomes the personal aesthetic standard 602 of each cluster.
In this way, it is possible to obtain the aesthetic standards of users who have similar aesthetic standards.

(8th Embodiment)
Next, the eighth embodiment will be described. In this embodiment, an example of a method for updating the general aesthetic standard 601 will be described. The method of this embodiment can be applied to any of the first to seventh embodiments.
In this embodiment, the personal aesthetic standard generation unit 203 is also used to update the general aesthetic standard 601. The image / incidental information input unit 501 inputs teacher data for personal aesthetic standards of each user of the plurality of digital cameras 100. The feature extraction unit 502 extracts a feature amount from an image included in teacher data for personal aesthetic standards of a plurality of users input by the image / incidental information input unit 501. The learning unit 503 is an evaluation index regarding the aesthetics included in the learning device into which the general aesthetic standard 601 is read, the feature amount extracted by the feature extraction unit 502, and the teacher data for the personal aesthetic standards of a plurality of users. Machine learning is performed using (good / bad label or aesthetic score). When this machine learning is completed, the updated general aesthetic standard 601 is obtained as a learning result.
In this way, the general aesthetic standard 601 can be made more accurate.

It should be noted that the above-described embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Digital camera, 201: Function input unit, 202: Aesthetics evaluation unit, 203: Personal aesthetics standard generation unit, 204: Aesthetics standard storage unit

Claims (7)

An image taken by the imaging means, an input means for a plurality of users by regarding the aesthetics of the image to input the supplementary information including the content of evaluation of the evaluation,
A clustering means for dividing the plurality of users into clusters to which users having similar evaluation contents for a plurality of images input by the input means belong.
The aesthetics of the user belonging to the cluster based on the result of machine learning using the plurality of images input by the input means and the incidental information including the evaluation contents evaluated by the user belonging to the same cluster with respect to the image. And the generation means to generate the judgment criteria of
An information processing system characterized by having. The information processing system according to claim 1, wherein the determination criterion includes information indicating a relationship between a feature amount related to the aesthetics of an image and an evaluation content related to the aesthetics of an image. The information processing system according to claim 1 or 2 , further comprising a derivation means for deriving the evaluation content regarding the aesthetics of the image to be evaluated for aesthetics based on the above-mentioned determination criteria. Input by the input means, based on the additional information of the image and the image, in any one of claims 1 to 3, further comprising a updating means for updating the previous SL decisions based The information processing system described. An information processing device having the input means and
It has an information processing device having the generation means, and has
The information processing device having the input means further has a transmitting means for transmitting the image and the incidental information of the image to the information processing device having the generating means.
The information processing system according to any one of claims 1 to 4 , wherein the information processing apparatus having the generation means further has an output means for outputting a judgment criterion regarding aesthetics of a user belonging to the cluster. .. Input means for information processing systems have included in an input step of accepting input of the supplementary information including the image captured by the imaging means, the evaluation contents in which a plurality of users by regarding the aesthetics of the image was evaluated,
The clustering means provided in the information processing system includes a clustering step of dividing the plurality of users into clusters to which users having similar incidental information belong.
Generating means provided in the information processing system, based on said plurality of images have been accepted et al by the input step, the machine learning users belonging to the same cluster with respect to the image is used and the supplementary information including the content of evaluation of the evaluation result And the generation process to generate the aesthetic criteria of the users belonging to the cluster,
An information processing method characterized by having. A program characterized in that a computer functions as each means of the information processing system according to any one of claims 1 to 5.

Images