JP2010026017A - Image display method, program, and image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide: an image display method which specifies an impression dominating a digital image and performs processing having an effect of enhancing the impression of the digital image to display the display image; a program for executing this method; and an image display device. <P>SOLUTION: In the image display method, impression labels of an input image 200 are determined per determination part 11 in an impression determination processing part 10 having a plurality of determination parts 11, and an impression decision part 40 decides an impression label whose determination result is equal to or larger than a threshold, as an impression label of this image, and the image is subjected to processing of enhancing the impression of the decided impression label when being displayed on a display device 3. The program executes the image display method, and the image display device is provided. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image display method, a program for executing the method, and an image display apparatus.

2. Description of the Related Art Conventionally, a technique has been developed that not only displays a digital image but also improves the viewing environment by using sound or light contrast, so that the user can enjoy and give satisfaction. For example, when a user plays back a digital image by arranging another display in the space in addition to a lighting facility, a sound facility, or a display that displays the digital image linked to the display device of the digital image, the digital image Based on the location and time at which the image was taken, the equipment other than the display that displays the digital image is linked to generate sound and video, which can be enjoyed not only visually but also auditorily. A forming technique is disclosed (for example, see Patent Document 1). However, in such a method, facilities are large-scaled, and it is not easy for the user to enjoy due to problems such as space for arranging them. Therefore, a perceptually dominant color is extracted from a digital image based on a certain rule, and when the digital image is displayed, the extracted color light is generated and emitted from the periphery of the digital image. Thus, a technique that can be enjoyed visually has been proposed (for example, Patent Document 2).
US Patent Application Publication No. 2006/020312 US Patent Application Publication No. 2007/0242162

  However, in this method, the peripheral part of the digital image is divided into six parts, and the light based on the extracted color of the image is only emitted from each of the six divided areas. It is not something that can be done. In other words, in this technique, when a dominant color is extracted from the digital image and displayed, the light based on the color is only emitted from the surroundings, so the digital image and the emission color are harmonized. There has been a problem that the image becomes monotonous visually.

  The present invention has been made in view of such problems, and it is possible to identify an impression that dominates a digital image, and to display the image by performing processing that has an effect of enhancing the impression of the digital image. An object of the present invention is to provide an image display method that can be enjoyed and that provides high satisfaction, a program that executes the image display method, and an image display device.

  In order to solve the above-described problem, the image display method according to the present invention includes an image analysis unit, and can access a learning result storage unit in which learning results of image impressions are stored for each of a plurality of previously registered impression labels. An image display method that is executed by a computer, determines an impression of an input image, enhances the impression, and outputs the image to a display device. The image analysis unit calculates a feature amount of the image from the input image Then, an image impression determination step for determining the degree of proximity of the impression to each of a plurality of impression labels in the image based on the feature amount and the learning result stored in the learning result storage unit, and the image impression determination step Selecting an impression label whose degree of proximity of the impression to the impression label is equal to or greater than a threshold value, and setting the impression label as an impression label of the image; When an image is displayed on the shown apparatus has an impression enhancement image processing step of outputting performs image processing to enhance the impression having the impression label determined in impressive label determining step for the image, a.

  Such an image display method is preferably configured to perform preprocessing on the image and calculate a feature amount of the image subjected to the preprocessing in the image impression determination step.

  In addition, in such an image display method, in the impression label determination step, the impression label of the image is determined by removing the contradictory impression label from the impression label that is equal to or higher than the threshold with respect to the impression label having the highest degree of proximity of the impression. It is preferable to be configured to do so.

  Further, in such an image display method, the impression analysis unit can access an impression frame definition table in which an enhanced color that enhances an impression of an image having the impression label is stored for each impression label, and the impression enhanced image processing In the step, an enhanced color for enhancing the impression of the image corresponding to the impression label of the image is acquired from the impression frame definition table, and the enhanced color is arranged around the image and displayed on the display device. It is preferable.

  In such an image display method, the impression frame definition table stores a color that contradicts the reinforced color for each impression label, and the impression label for the image determined in the impression label determination step is stored in the impression reinforced image processing step. When there are a plurality of colors, the contradictory color corresponding to the impression label having the highest degree of proximity of the impression is acquired from the impression frame definition table, and the contradictory color is not arranged in the image in the impression enhancement image processing step. preferable.

  In addition, a program for causing a computer to execute the image display method according to the present invention can be created. This program is, for example, a storage medium such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, a hard disk, or the like. It is stored in a storage device. Moreover, it may be distributed as a digital signal via a network. At this time, intermediate processing results are temporarily stored in a storage device such as a main memory.

  The image display device according to the present invention executes a learning result storage unit that stores a learning result of an image impression for each of a plurality of pre-registered impression labels, a display device that displays an image, and the above-described program. And a computer.

  When the image display method, the program, and the image display device according to the present invention are configured as described above, the image impression is specified based on the partial region giving the image impression, and the user visually The image can be displayed with a process having an effect of enhancing the specified impression so that the user can enjoy and have high satisfaction.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, the configuration of the image display apparatus 100 according to the present embodiment will be described with reference to FIG. The image display device 100 includes a CPU, a RAM, a ROM, and the like, and includes a computer 1 that can execute a program, a display image storage device 2 that stores images, and a display device 3 that displays images. Configured. The computer 1 is provided with an image analysis unit 30 realized by a program. The image analysis unit 30 includes an impression determination processing unit 10 that performs impression determination of an image read from the display image storage device 2. An impression determination unit 40 that performs an impression enhancement process on the evaluated image with reference to the data in the impression label relation definition storage unit 41 with respect to the evaluation result of the impression determination processing unit 10. An image is displayed on the display device 3 together with the result of the impression determination unit 40.

  Here, the impression determination processing unit 10 is provided for each impression label, and evaluates a digital image 200 (hereinafter referred to as “image 200”) read from the display image storage device 2 using a learning result based on the learning image. A plurality of impression label determination units 11, a learning image storage unit 21 in which learning images are stored, a learning result storage unit 22 in which learning results from the impression label determination unit 11 are stored, and a plurality of impression label determination units 11 And a determination result output unit 14 that outputs the determination result by the impression determination unit 40. Details of the impression determination method and the like of the image in the impression determination processing unit 10 will be described later, and what kind of impression enhancement processing is performed on the image displayed on the display device 3 will be described with reference to FIGS. Will be described.

  When the user wants to enhance the impression of the image and enjoy it visually, the image analysis unit 30 is executed in the computer 1 and the display and reproduction of the image is started. First, an image 200 desired to be displayed on the display device 3 is selected by the user using an input device (not shown) or the like, and the image analysis unit 30 reads the image 200 from the display image storage device 2 (step S100), and impression determination described later. The processing unit 10 determines the impression of the image 200 and evaluates the degree of proximity of the impression for each impression label with respect to the image 200 (step S101 (image impression determination step)). When the process of the impression determination processing unit 10 is completed, the impression determination unit 40 next determines the final impression label of the image 200 to be evaluated from the impression determination result of each impression label (step S102 ( Impression label determination step)). At this time, the impression determination unit 40 refers to the impression label relationship definition storage unit 41. The impression label relationship definition storage unit 41 stores an impression label definition table 411 shown in FIG. 3A and an impression frame definition table 412 shown in FIG. Therefore, the impression determining unit 40 is configured to first refer to the impression label definition table 411 of the impression label relation definition storage unit 41 when determining the final impression label.

  Here, the impression label definition table 411 includes an impression ID column 411a in which identification information (impression ID) for identifying an impression label assigned to each impression label determination unit 11 is stored, and a seal in which the impression label is managed. It consists of a label column 411b and an inconsistent impression label column 411c in which an impression label that contradicts the impression label is stored. For example, the impression label stored as “A” in the impression ID column 411a stores “bright” in the impression label column 411b. As contradictory impression labels corresponding thereto, conflicting words such as “dark” and “blurred” are registered in the contradictory impression label column 411c. In addition, “clear” is registered as an impression label with an impression ID “B”, and words such as “unclear” and “blurred” are registered as impression labels contradicting this. Note that the order of registration in the contradictory impression label column 411c is not particularly limited, but from the left, in order from the most common among the conflicting, contradictory, or recognized opposite words. It is possible to register.

  As described above, the impression determination unit 40 determines the final impression label of the image 20 with reference to the impression label definition table 411 in step S102. That is, the impression determination unit 40 performs a process of eliminating impression labels (conflicting impression labels) having a conflicting relationship among a plurality of impression labels obtained through the determination result output unit 14 of the impression determination processing unit 10. The impression label having the largest value of the degree of proximity of the impression as the impression determination result among the remaining impression labels is determined as the impression label of the image 200. Furthermore, it is also possible to configure so that an impression label having a value equal to or greater than a predetermined threshold is recognized as an impression label of the image without conflicting with the impression label. For example, when the enhancement process is finally performed on the image 100, only the impression label having the largest impression given to the image 200 may be applied to the enhancement process of the image 200, or the impression determination result of a certain threshold value or more. Among the obtained impression labels, the impression label having the greatest impression and the impression label not conflicting with the impression label having the greatest impression may be used in combination for the enhancement processing of the image 200.

  Next, in order to enhance the impression of the image 200 and display it on the display device 3, the impression determination unit 40 performs an impression enhancement process on the image 200 based on the final impression label determined in step S <b> 102. Apply. In this impression enhancement process, a color corresponding to the impression label is selected, and the frame color and pattern of that color are displayed together with the image 200. That is, the impression determination unit 40 selects a frame color / pattern corresponding to the final impression label with reference to the impression frame definition table 412 (step S103 (impression enhanced image processing step)).

  The impression frame definition table 412 of the impression label relation definition storage unit 41 includes an impression ID column 412a for storing an impression ID, an enhanced color column 412b for storing a color for enhancing an impression label corresponding to the impression ID, A contradiction color column 412c in which a color contradicting the enhancement color is stored. Therefore, in this step S103, the impression determination unit 40 acquires from the impression frame definition table 412 an enhanced color for enhancing the final impression label (impression ID) and a color that contradicts the enhanced color. In the above-described step S102, when a plurality of impression labels are finally selected, even if the impression labels are not contradictory, the colors corresponding to the impression labels are incompatible colors (conflicting colors or contradictory colors). ), Such incompatible colors are also excluded. Here, the conflicting color and the contradicting color are, for example, a color having a complementary color relationship with a certain color, a color similar to the complementary color, or a color having an equal hue relationship. In other words, colors other than colors that are similar to a certain color are examples of incompatible colors. Specifically, for example, in the hue circle, if it is red, it is blue-green, blue, patina, green, etc., and when red is classified into two systems, red is classified as a warm color. For example, a color classified as a cold color system can be defined as an incompatible color.

  In step S <b> 103, the impression determination unit 40 can determine a frame pattern as to how to arrange the selected color with respect to the image 200 when the image 200 is displayed. For example, although not shown in FIG. 1, several frame patterns are registered in advance in the impression label relation definition storage unit 41, and the stored patterns are exemplified in the display device 3. . The user can then operate the operation unit such as a push button or a display panel to select one of the illustrated frame patterns and decide to use it when displaying an image. It becomes like this.

  Here, for example, FIGS. 4A to 4C are given as display examples of the frame. The screen 500 includes an image display unit 501 that displays the image 200 and an outer peripheral area 502 around the image display unit 501. FIG. 4A illustrates the image 200 displayed on the image display unit 501. In this example, a color for enhancing the impression label of the image 200 is arranged in the outer peripheral area 502 around the displayed area. FIG. 4B shows an image 200 displayed on the image display unit 501, between the outer peripheral region 502 around the image 200 and the image 200 displayed on the image display unit 501, and the main subject. A color for reinforcing the impression label of the image 200 is arranged in an area 501a where (a person in FIG. 4B) is not displayed. Further, in FIG. 4C, colors are arranged in both of the regions where the color for enhancing the impression label of the image 200 is arranged in FIGS. 4A and 4B. That is, a color for enhancing the color is arranged in both the outer peripheral area 502 and the area 501a where the main subject is not displayed. These can be set in advance by the user, or an optimal display method may be automatically selected at the time of display.

  When the frame pattern is determined as described above, the impression determination unit 40 displays the selected frame color along with the image along any one of the display patterns of FIGS. 4 (a) to 4 (c). (Step S104). In this way, since the color corresponding to the impression (impression label) of the image is displayed together with the image, the user can enjoy visually.

  In the configuration as described above, the relationship between the impression ID in the impression frame definition table 412 and the enhanced color that enhances the impression is set to a color that matches the image of the impression label corresponding to the impression ID. Furthermore, it is good. For example, the impression label stored as “C” in the impression ID column 411 a in the impression label definition table 411 is “settled” as defined in the impression label definition table 411. As for the color corresponding to the impression label whose impression ID is “C”, referring to the impression frame definition table 412, it can be seen that “blue” is registered as an enhanced color. Therefore, the color corresponding to the impression label “settled” is “blue”, and “blue” is used to enhance this impression for images having a “settled” impression label. Become. “Blue” is usually used as a color that stabilizes the mind and is considered to be consistent with the image of the “calm” impression label.

  In this way, when a color that matches the image of the impression label is set, the effect of enhancing the impression of the image by the color corresponding to the determined impression label is enhanced, and the image emphasized by that color is displayed. Then, the image is more easily recognized through the user's vision, and the image display is highly satisfactory. As a result of the above processing, the user can visually extract the impression of the image as a display that can be enjoyed visually, and form an image frame with a color that enhances the extracted impression and display the image. When it does, it is displayed with the frame.

  Next, specific contents of the impression determination process executed in step S101 described above will be described with reference to FIGS. Step S101 shown in FIG. 2 is a process in the impression determination processing unit 10, and is a method of specifically determining the impression label of the image 200. In the impression determination processing unit 10, each of the impression label determination units 11 includes a learning processing unit 11a that performs learning based on a learning image and an impression label registered in advance in the learning image storage unit 21, as shown in FIG. The learning result storage unit 22 includes an evaluation processing unit 11b that evaluates the image 200 based on the stored learning result.

(A) Learning with a learning image The learning processing unit 11a uses a plurality of learning samples (learning images) by the AdaBoost technique to provide one strong classifier for determining the impression of an image according to the processing procedure shown in FIG. H (X) is constructed. When the learning processing unit 11a starts learning, the learning processing unit 11a reads a plurality of learning images stored in the learning image storage unit 21 (step S600). Here, as shown in FIG. 7, the learning image storage unit 21 includes a learning image storage table 211 that stores learning images, and an impression label storage table 212 that associates impression labels with learning images. Configured.

  The learning image storage table 211 stores an image ID column 211a in which an image ID for identifying a learning image is stored, a file column 211b in which a learning image is stored, and a feature amount X (j) of the learning image. And a feature amount column 211c. FIG. 7A shows a case where seven images are registered, and j indicates a learning image number. In addition, the learning image main body may be stored in another area instead of directly storing the learning image in the file column 211b, and the file name (or path name) of the learning image may be stored in this column. Is possible.

  The impression label storage table 212 stores an impression ID column 212a for storing an impression ID for identifying an impression label, an impression label column 212b for storing an impression label, and learning associated with the impression label. And an image ID column 212c in which the image ID of the image is stored. In this image ID column 212c, a plurality of image IDs can be set. For example, an impression label of “bright” with impression ID = “1” has image IDs “0001” and “0002”. Two learning images are set.

  Note that the image feature amount X is defined as an n-dimensional vector using a scalable color or an edge histogram standardized by MPEG7, and in the present embodiment, It is calculated by a predetermined method for each learning image and set in the feature amount column 211c.

Then, the learning processing unit 11a sets “1” as an initial value to the learning step number t (step S610), determines whether the step number t has exceeded a predetermined number of times (Tmax), Steps S630 to S650 are repeated until Tmax is exceeded (step S620). In this iterative process, in step S630, the weak classifier h (x, t) in (t-th) learning for each iterative process is constructed using the learning image read in step S600. Although a detailed construction method of the weak classifier h (x, t) will be described later, the learning processing unit 11a in the present embodiment divides the learning image into predetermined partial areas, and features in the partial areas. Learning is performed for each partial vector of the quantity, and a weak classifier h _{i, k} (x, t) is constructed for each partial vector of the partial area (i is the number of the partial area, and k corresponds to the partial area. Select the partial vector of the feature quantity of the partial area with the strongest impression from the weak classifiers corresponding to the partial vector of the feature quantity in these partial areas, and correspond to this partial vector The weak classifier to be selected is selected as the weak classifier h (x, t) in this learning process (t-th learning process). Therefore, x represents a partial vector in a partial region corresponding to each weak classifier h (x, t) in the feature amount (vector amount) X of the image input in the t-th learning.

In step S640, the coefficient α _t in the t-th learning step for linear combination is calculated from the weak classifier h (x, t) constructed in step S630, and then the next learning step (t + 1-th time). The weight D _{t + 1} (j) of the j-th learning image used in () is updated. In step S650, 1 is added to the step number t.

As described above, by learning Tmax times, a set of Tmax partial areas (i), partial vectors (k) in the partial areas, and weak classifiers h (x, t) is determined. In step S660, one weak classifier H (X) is constructed by linearly combining these weak classifiers h (x, t). The coefficient α _t for linearly combining the weak classifiers h (x, t) in this step S660 is calculated in step S640 as described above. As described later, in the partial region selected in step S630, the coefficient α _t is calculated. Using the weak classifier h (x, t) corresponding to the partial vector and its error rate ε _t , it is determined as the following equation (1). Here, assuming that one learning image is divided into m partial regions, a set of combinations of partial regions is G = {g1, g2,. . . , Gm}, and the k-th partial vector corresponding to the partial area i of the j-th learning image is x _{i, j, k} , the feature quantity vector X is a set of partial vectors (X (j) = {X1 _{, j, 1} , x1 _{, j, 2} , ..., x1 _{, j, k1} , x2 _{, j, 1} , x2 _{, j, 2} , ..., x2 _{, j, k2} , ..., _xm , _{j, 1} , xm _{, j, 2} , ..., xm _{, j, km} }). Note that k1, k2,. . . , Km is the number of feature partial vectors in each partial region i.

This expression (1) represents the reliability of the weak classifier h (x, t). The coefficient α _t increases as the error rate ε _t decreases, and the corresponding weak classifier h (x, t) The influence of the determination in step S660 increases. However, when the error rate epsilon _t exceeds 0.5, the weak classifier h (x, t) it is possible to reduce the error rate epsilon _t If the judgment of the reverse, the error rate from 1 in this case the value obtained by subtracting the ε _t as a new error rate ε _t. Therefore, the worst value of the error rate ε _t is 0.5.

In addition, since the weight D _{t + 1} (j) of the learning image in the _{t +} 1th learning needs to be 1 when the weights for all the learning images are added, the condition value Z _t shown in the following equation (2) Is obtained from the weight D _t (j) in the t-th learning so as to satisfy the following equation (3). In Equations (2) and (3), Jmax represents the number of learning images, and x _{i, j, k} is the _kth of the jth image corresponding to the partial region i for which the weak classifier has been selected. Y _j represents the presence or absence of the impression label of the j-th image, 1 if the impression label assigned to the impression label determination unit performing this learning is 1, and -1 otherwise. It becomes.

  From the above, one strong classifier H (X) constructed in step S660 is expressed by the following equation (4). In Equation (4), X is a feature quantity (vector quantity) of the image evaluated by the strong classifier H, and x is a weak classifier h (x in the t-th learning of the feature quantity X. , T) are partial vectors corresponding to the selected partial region.

  FIG. 9 shows a state where the most frequently selected partial area is identified as an area that affects the impression of the image throughout the learning, and the result of learning performed Tmax times is as follows. It becomes a region (a circled region) that has a high appearance frequency throughout the learning.

  Now, a specific configuration of the process (step S630) for constructing the weak classifier h (x, t) will be described with reference to FIG. First, the learning image is divided into a plurality of predetermined partial areas (step S6301). For example, as shown in FIG. 9, the image is divided into 24 partial areas of 6 × 4 (in this case, m = 24). In addition, here, in order to simplify the description, the case where the vertical and horizontal sides are equally divided is shown, but the size of the partial region can be arbitrarily set. A combination of these divided partial areas may be used as a new partial area. Next, the number of partial vectors in each of the partial regions is k = {k1, k2,. . . , Km} (step S6302). Then, “1” is set as an initial value in the partial area counter i and the partial vector counter k (step S6303), and the following processing is repeated from the partial area g1 to the partial area gm (step S6304).

First, partial vectors of feature amounts in the partial region gi of all the learning images read from the learning image storage unit 21 in step S600 are extracted (step S6305). As described above, when there are seven learning images, the set of partial vectors extracted in the processing of the i-th partial region is expressed as {xi _{, j, 1} , xi _{, j, 2,.} . . , X _{i, j, ki} } (j = 1, 2,..., 7). Then, it is determined whether or not k is larger than ki, that is, whether or not the determination is completed for all the feature vectors corresponding to the i-th partial region (step S6306). Adds 1 to i (step S6307), sets 1 to k (step S6308), and returns to step S6304. On the other hand, if it is determined in step S6306 that the determination has not been completed for all partial vectors, in the t-th learning, the weighted average value of the k-th partial vector of the feature amount of the partial region i of the positive sample of the learning image. Is calculated by the following equation (5), and the weighted average value of the k-th partial vector of the feature quantity of the partial region i of the negative sample of the learning image is calculated by the following equation (6) (step S6309). Here, the positive sample refers to the learning image assigned to the impression label set in the impression label determination unit 11, and is the above-described y _j = 1 sample. It means a learning image that is not assigned to an impression label, and is the above-described sample of y _j = −1. For example, in FIG. 4B, in the learning processing unit 11a of the impression label determination unit 11 in which the impression label “bright” is set, the learning images with the image IDs “0001” and “0002” become positive samples. Learning images other than are negative samples. In Expressions (5) and (6), “x” with “˜” is a weighted average value of the partial vectors, and the character above the subscript indicates that “Yes” is for a positive sample. , “No” indicates a negative sample. Also, m <n.

In these formulas (5) and (6), n ₁ represents the number of positive samples, and n ₂ represents the number of negative samples. Here, n = n ₁ + n ₂ . Λ _j is the reciprocal of the number of impression labels added to the j-th learning image. For example, in FIG. 7B, “bright” and “settled” are added as impression labels to the learning image with the image ID “0001”, and λ ₁ = 0.5. Similarly, since only the impression label “bright” is added to the learning image with the image ID “0002”, λ ₂ = 1.0. That is, λ _j represents the influence on the impression of the j-th learning image, and the influence on the impression of the learning image becomes smaller as the learning image to which more impression labels are added. D _t (j) is the weight of the j-th sample in the t-th learning. The weight mentioned here is a ratio that affects the equations (5) and (6) in which each learning image is a partial vector of an average feature amount, and always satisfies the relationship of the following equation (7). It shall be.

The weight D _t (j) of the positive and negative samples changes with the number of learning steps, but the value at the initial value (t = 1) is expressed by the following equations (8) and (9), respectively. Defined and the weight of each learning sample is the same.

Next, the Euclidean distance between the average value of the partial vector of the feature amount calculated in step S6309 and the partial vector of the feature amount of each learning image is calculated by the following equations (10) and (11) (step S6310). ). In equation (10), f ₁ (x) represents the Euclidean distance between the kth partial vector of the i-th region and the average value of the kth partial vector of the feature quantity of the positive sample. , F ₂ (x) represents the Euclidean distance between the k-th partial vector of the i-th region and the average value of the k-th partial vector of the feature quantity of the negative sample. However, the distance calculation can be substituted by another norm such as 1 norm. Of the subscripts of the average value of the partial vector of feature amounts, the first t indicates the t-th learning step.

Then, using the distances f ₁ (x) and f ₂ (x) calculated in step S6310, the weak classifier h that determines whether the impression is positive or negative in the k-th partial vector of the i-th partial region. _{i, k} (x, t) is constructed by the following equation (12) (step S6311). Here, “1” is positive and “−1” is negative. That is, the weak classifier h _{i, k} (x, t) represented by the equation (12) is the distance between the partial vector of the feature amount of the evaluation image and the weighted average value of the partial vector obtained from the learning image. Is judged as positive (1) when close to the positive sample, and negative (-1) when the distance between the positive and negative samples is equal or close to the negative sample.

Finally, using the weak classifiers h _{i, k} (x, t) constructed in step S6311 described above, the k-th partial vector for the i-th partial region of all learning images is determined, It is examined whether the determination result matches the state (positive sample or negative sample) set in the impression label storage table 212 (step S6312). As a result of this inspection, when the determination result by the weak classifier matches the set state for all learning images, 1 is added to k (step S6315), and the process returns to step S6306 and the next partial vector of the partial area. The above processing is repeated for (Step S6313). On the other hand, if the determination result does not match the set state in any learning image in step S6313, the corresponding weak classifier h _{i, k} (x, t) (in other words, Error rate ε _{i, k} corresponding to the kth partial vector of the partial area i to be calculated (step S6314), 1 is added to k (step S6315), and the process returns to step S6306.

Such a series of processing from steps S6309 to S6312 is repeated by the number of elements of the set G of partial area combinations. Finally, the weak classifier h _{i, k} (x, t) having the lowest error rate ε _{i, k in the i} partial regions is set as the weak classifier h (x, t) in the t-th learning. The learning step t, the partial area number i, the partial vector number k, the coefficient α _t and the k-th partial vector of the feature quantity in the selected partial area for the weak classifier h (x, t). The weighted average value is stored in the feature value table 221 and the determination information table 222 of the learning result storage unit 22 (step S6316). As shown in FIG. 10A, the feature amount table 221 includes an impression ID column 221a and a feature amount column 221b. For each impression ID stored in the impression ID column 221a, the feature amount column 221b. The number i of the partial area selected in the learning from the time t = 1 to the time Tmax and the number k of the partial vector in the partial area are stored. Further, as shown in FIG. 10B, the determination information table 222 includes an impression ID column 222a, a learning frequency column 222b, a coefficient column 222c, and an average value column 222d. A coefficient (α _t ) and a weighted average value of partial vectors in learning from t = 1 to Tmax times are stored. The error rate ε _{i, k} in the partial vector k of the partial region i for which the weak classifier h _{i, k} (x, t) is selected is selected as the error rate ε _t in the t-th learning.

(B) Image Evaluation The evaluation processing unit 11b evaluates an image by Expression (4) based on the learning result described above. Specifically, as shown in FIG. 9, first, when an image to be evaluated (the above-described image 200) is designated by an input device (not shown) in FIG. 1, the evaluation is performed from the display image storage device 2. The image 200 to perform is acquired (step S700). Note that a preprocessing storage data table 223 as shown in FIG. 12 is provided in the learning result storage unit 22, and an image acquired from the display image storage device 2 based on information set in the preprocessing storage data table 223. It is desirable to process 200.

  Specifically, the preprocessing storage data table 223 includes a preprocessing column 223a in which preprocessing information is stored, and a setting value column 223b in which on / off and setting values are stored. As pre-processing of the image 200, “Resize”, “Use EXIF thumbnail image”, “Smoothing”, and the like are stored. “Resize” is a process of resizing the acquired image 200 to the size (number of pixels) set in the setting value column 223b, and “EXIF thumbnail image use” is an EXIF format file. Sometimes the thumbnail image stored in the file is used for impression determination, and “smoothing” is a process of smoothing the image 200. As described above, by performing the pre-processing on the image 200 before the impression label is evaluated, the processing time for calculating the following feature amount can be shortened, and the processing speed can be increased. .

Then, a feature value vector X of the image 200 (or an image after preprocessing) is calculated (step S710). Next, “1” is set as an initial value for the number of repetitions t, and the next processing (step S740 to step S760) is repeated until the number of repetitions t is greater than Tmax (step S730). In step S740, i and k in the t-th learning are acquired from the feature amount table 221 in the learning result storage unit 22, and the weight of the k-th partial vector of the partial region i in the t-th learning is determined from the determination information table 222. Get the average value. In step S750, among the vector X of the feature amount of the image calculated in step S710, the k-th partial vector (x) in the partial area i and the k-th partial vector in the partial area i acquired in step S740. From the weighted average value, the value of the weak discriminator h (x, t) is obtained by the equations (10) to (12) (step S750). In step S760, 1 is added to t. When the results of Tmax weak classifiers h (x, t) are obtained as described above, finally, the value of Tmax weights α _t is read from the determination information table 222 (step S770), and the equation (4) ) To calculate the value of the strong classifier H (X) (step S780).

According to the above configuration, the evaluation processing unit 11b determines whether or not the image 200 has an impression corresponding to the impression label set in the impression label determination unit 11 provided with the evaluation processing unit 11b. Can be calculated as As described above, since the value of “1” or “−1” is output from the weak classifier h (x, t), the determination result output unit 14 determines that the value of the strong classifier H (X) is a positive value. If so, it can be determined that the impression is present, and if the value is less than 0, the impression is not present. Alternatively, a predetermined threshold value is set, and if the value is greater than or equal to this threshold value, the impression is given, and if the value is smaller than the threshold value, it can be determined that the impression is not given. That is, in this impression label determination unit 11, the value of the strong classifier H (X) does not have / does not have an impression (“1” or “−1”), but Tmax weak classifiers and weights. Since it is the total sum (linear combination) of the products with α _t , the determination result output unit 14 can express the degree of proximity of the impression as a numerical value. Therefore, it is possible to compare and rank which impression is the strongest by comparing the respective calculation results (H (X)) of the impression label determination unit 11 in which a plurality of impression labels are set. It is.

  In the impression determination processing unit 10 according to the present embodiment, the determination result output unit 14 determines the determination result (value of the strong classifier H (X)) from the impression label determination unit 11 associated with each of a plurality of impression labels. ) Can be configured to display the largest value as an impression label (impression) of the image, or can be configured to output an impression label equal to or greater than a predetermined threshold. It is also possible to provide a selection unit for the impression label determination unit 11 so that the user can select the impression label of the image from the operation unit or the like. Moreover, in the above description, although the case where the impression label determination part 11 has the learning process part 11a and the evaluation process part 11b was demonstrated, the impression label determination part 11 in this embodiment performs only the impression determination of the image 200. As long as it can be performed, the learning processing unit 11a and the learning image storage unit 21 can be omitted if the learning storage unit 22 and the evaluation processing unit 11b in which the results learned in advance are stored.

  And the determination result output part 14 of the above impression determination process parts 10 outputs the impression label obtained from the determination result to the impression determination part 40 mentioned above. The processing in the impression determination unit 40 is as described above.

  As described above, in the impression determination processing unit 10, the partial region giving the impression of the image and the partial vector of the feature amount are specified by learning, and a discriminator is constructed based on the information. The impression label of the image can be specified by the device. The impression of the image is emphasized with a color that matches the impression label of the image specified in the impression determination processing unit 10, and when the image is displayed on the display device 3, the image is displayed as an image with the impression enhanced. Therefore, it is possible for the user to perform a display that is sufficiently enjoyable visually and highly satisfactory. Further, in the above-described example, when the color corresponding to the impression label is displayed on the image, the case where the color is one color has been described. However, when there are two or more colors, the two or more colors are displayed on the screen. When the image is emphasized by placing it above, it may be displayed in gradation, in a grid pattern, or in a checkered pattern, check pattern, or staggered pattern. Alternatively, in the case of two or more colors, the impression label having the largest value in the determination result (the value of the strong discriminator H (X)) is set as the main base color, and another color is arranged thereon with a pattern or the like. You may do it.

It is a block diagram which shows the structure of the image display apparatus which concerns on this invention. It is a flowchart which shows the process of an image analysis part. It is a data structure figure which shows the data structure of the table memorize | stored in the impression label relation definition memory | storage part, Comprising: (a) shows an impression label definition table, (b) shows an impression frame definition table. It is a schematic diagram of the screen showing the arrangement of the frame color when performing the image enhancement processing, showing (a) an example in which the frame color is arranged in the outer peripheral area of the image, and (b) excluding the outer peripheral area of the image An example in which a frame color is arranged in an area where no image is displayed is shown, and (c) an example in which a frame color is arranged in an outer peripheral area of the image and an area where no image is displayed is shown. It is a block diagram which shows the structure of the determination part in an impression determination process part. It is a flowchart which shows the process in a learning process part among the said determination parts. It is a data structure figure which shows the data structure of a learning image memory | storage part, Comprising: (a) shows the structure of a learning image storage table, (b) shows the structure of an impression label storage table. It is a flowchart which shows the detail of the process which builds a weak discriminator in the said learning process part. It is explanatory drawing which shows the process in which the partial area | region which affects an impression is specified in the process in the above-mentioned learning process part. It is a data structure figure which shows the data structure of a learning result memory | storage part, (a) shows the structure of a feature-value table, (b) shows the structure of a determination information table. It is a flowchart which shows the process in an evaluation process part among the said determination parts. The data structure figure which shows the data structure of the pre-processing storage data table.

Explanation of symbols

1 Computer 3 Display Device 22 Learning Result Storage Unit 30 Image Analysis Unit 41 Impression Label Relationship Definition Storage Unit (412 Impression Frame Definition Table)
100 Image display device

Claims (7)

The image analysis unit is executed by a computer accessible to a learning result storage unit in which a learning result of an image impression is stored for each of a plurality of pre-registered impression labels, the impression of the input image is determined, An image display method for enhancing an impression and outputting to a display device,
By the image analysis unit,
The feature amount of the image is calculated from the image, and the proximity degree of the impression to each of the plurality of impression labels in the image is determined based on the feature amount and the learning result stored in the learning result storage unit. An image impression determination step,
An impression label determining step of selecting an impression label whose degree of proximity of the impression to the impression label in the image impression determination step is equal to or greater than a threshold, and using the impression label as an impression label of the image;
Impression-enhanced image processing step of performing image processing for enhancing the impression of the impression label determined in the impression label determination step when the image is displayed on the display device, and outputting the image.
An image display method comprising:   The image display method according to claim 1, wherein in the image impression determination step, preprocessing is performed on the image and a feature amount of the preprocessed image is calculated.   In the impression label determination step, the impression label of the image is determined by excluding the contradictory impression label from the impression label that is equal to or higher than a threshold value with respect to the impression label having the highest degree of proximity of the impression. The image display method according to claim 1 or 2. The image analysis unit can access an impression frame definition table in which an enhanced color that enhances an impression of the image having the impression label is stored for each impression label,
In the impression enhancement image processing step, the enhancement color for enhancing the impression of the image corresponding to the impression label of the image is acquired from the impression frame definition table, and the enhancement color is arranged around the image. The image display method according to claim 1, wherein the image display method is configured to display on the display device. The impression frame definition table stores a color that contradicts the enhanced color for each impression label,
In the impression enhancing image processing step,
When there are a plurality of impression labels for the image determined in the impression label determination step, the contradictory color corresponding to the impression label having the highest degree of proximity of the impression is acquired from the impression frame definition table,
The image display method according to claim 4, wherein the contradictory color is not arranged in the image in the impression enhancing image processing step.   The program for making a computer perform the image display method as described in any one of Claims 1-5. A learning result storage unit storing a learning result of an image impression for each of a plurality of impression labels registered in advance;
A display device for displaying an image;
An image display apparatus comprising: a computer that executes the program according to claim 6.

Images