JP2018088132A - Sensitivity evaluation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable human visual sensibility to be quantified for evaluation.SOLUTION: For an image to be evaluated with respect to a predetermined sensibility (for example, texture), feature data in plural hierarchies simulating hierarchical visual information processing in the human brain (a plurality of first feature data by frequency processing to be processed in a first hierarchy, a plurality of second feature data by azimuth selection processing to be processed in a second hierarchy and a plurality of third feature data by azimuth signal intensity processing to be processed in a third hierarchy) is calculated. For each feature data, autocorrelation and cross-correlation are calculated. An evaluation model in which the autocorrelation and the cross-correlation of feature data of each hierarchy are set as parameters is stored. The autocorrelation and the cross-correlation calculated for a certain image are checked against the evaluation model to evaluate the predetermined sensitivity.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sensitivity evaluation system.

  Human beings have sensibilities related to vision, for example, texture, aesthetics, and the like. There are Patent Document 1 and Patent Document 2 for evaluating this visual sensitivity. Patent Document 1 displays an evaluation result of an image impression using a sensitivity term representing sensitivity. Japanese Patent Application Laid-Open No. 2004-228561 performs image evaluation reflecting a photographer's preference such as exposure level.

JP-A-9-16797 JP 2006-254107 A

  Conventional evaluation of visual sensibilities is based on the use of sensibility terms and reflects the preference of the photographer. It does not physically evaluate the sensibility itself, and it does not lack concreteness. It was something that I did not get.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a sensitivity evaluation system capable of quantifying and evaluating human visual sensitivity.

In order to achieve the above object, the following solution is adopted in the present invention. That is, as described in claim 1,
Image acquisition means for acquiring an image to be evaluated for a predetermined sensitivity;
Image feature calculation means configured to simulate hierarchical visual information processing in the human brain, and to calculate feature data for each hierarchy of the image acquired by the image acquisition means;
A visual statistic calculating means for calculating auto-correlation and cross-correlation for the feature data calculated by the image feature calculating means;
An evaluation model storage means that stores preset evaluation models for the predetermined sensibilities, which are set in advance as parameters of autocorrelation and cross-correlation of the feature data of each of the layers,
A sensitivity evaluation unit that compares the autocorrelation and cross-correlation calculated by the visual statistic calculation means with the evaluation model, and evaluates the sensitivity of the image to be evaluated,
It is supposed to be equipped with. According to the above solution method, human visual sensitivity can be quantified and evaluated. In particular, since the evaluation method uses data simulating human visual information processing, it is possible to perform an evaluation that appropriately matches the sensibility actually felt by humans.

A preferred mode based on the above solution is as described in claim 2 and the following. That is,
The image feature calculation means includes a plurality of first feature data by frequency processing that is processing in the first hierarchy, a plurality of second feature data by orientation selection processing that is processing in the second hierarchy, and a third hierarchy. A plurality of third feature data is calculated by the azimuth signal intensity process which is the process (corresponding to claim 2). In this case, it is possible to obtain an evaluation that appropriately reproduces the visual information processing performed in the cerebral cortex from the human retina through the outer knee.

  The apparatus further comprises learning means for learning and correcting the evaluation model based on autocorrelation and cross-correlation calculated by the visual statistic calculation means (corresponding to claim 3). In this case, it is preferable to set the evaluation model more appropriately.

A plurality of types of sensibilities to be evaluated are set, and a plurality of the evaluation models are set corresponding to the plurality of sensibilities,
Model selection means for selecting one evaluation model from the plurality of evaluation models according to the predetermined sensitivity to be evaluated,
The Kansei evaluation means collates the autocorrelation and cross-correlation calculated by the visual statistic calculation means against the evaluation model selected by the model selection means,
(Corresponding to claim 4). In this case, evaluation corresponding to various visual sensibilities can be performed.

  The evaluation model is determined by performing a principal component analysis on the autocorrelation and the cross-correlation acquired for a large number of images, and then selecting a parameter for evaluating the predetermined sensitivity. Yes (corresponding to claim 5). In this case, it is preferable for setting the evaluation model as easily and easily as possible.

  The evaluation model is set by the following equation (where Y is a subjective evaluation value of sensitivity, a, b, and c are coefficients, and X is the autocorrelation or the cross-correlation): Item 6). In this case, the sensitivity evaluation value can be obtained by a mathematical expression.

  The predetermined sensitivity is evaluated as a texture evaluation (corresponding to claim 7). In this case, it is possible to evaluate the texture that is often performed widely.

  The evaluation model is configured to set autocorrelation of the third feature data and cross-correlation of the second feature data as parameters (corresponding to claim 8). In this case, the texture can be evaluated in the form of a two-dimensional map while simplifying the parameters used by the evaluation model as much as possible. In addition, since the position on the two-dimensional map can be intuitively understood, it is possible to easily know the orientation for increasing or decreasing the texture.

  According to the present invention, visual sensitivity can be quantified and evaluated.

The block diagram which shows the control system which performs sensitivity evaluation. The figure which simulates the hierarchical visual information processing in a human brain, and shows the calculation of the autocorrelation and the cross correlation of the feature data of each hierarchy, and each feature data. The figure for demonstrating the method of determining an evaluation model. The figure which shows the example of the autocorrelation map of 3rd characteristic data. The figure which shows the example which calculates the cross correlation of 2nd feature data. The figure which shows the example of the map for texture evaluation. The figure which shows the example of an image of the instrument panel part of the motor vehicle used as the object of evaluation of texture evaluation.

  First, with reference to FIG. 2, calculation of feature data in each layer based on processing simulating visual information processing in the human brain, and autocorrelation and cross-correlation for each feature data will be described.

  The visual information processing in the human brain is a process in the cerebral cortex from the retina through the outer knee-like body, such as a process in the first layer, a process in the second layer, and a process in the third layer. Done in stages. The input of visual information input to the retina is indicated by ○ 1 in the figure. In the following explanation, the numbers in FIG. 2 surrounded by circles are displayed with circles 1, 2, etc. (for example, circle 1 is the number 1 surrounded by circles). ).

  The processing in the first layer is a simulation of the outer knee-like body processing after retinal input, and is frequency processing, that is, resolution processing, for example, decomposed into high frequency (high resolution) and low frequency (low resolution). The In the embodiment, the frequency is decomposed in two stages, and the two types of feature data shown as ○ 2 and ○ 3 in FIG. 2 are calculated by the processing in the first layer. . In addition, the process in the first hierarchy is displayed as “LGN” by taking the display of the outer knee (LGN).

  The processing in the second layer simulates the processing of simple cells in the primary visual cortex, and the direction and phase decomposition processing is performed for each frequency processed in the first layer. A total of eight types of feature data of ○ 4 to ○ 11 are calculated by the processing of the second hierarchy. Note that the processing in the second hierarchy is displayed as “V1 Simple” by taking the display of simple cells in the primary visual cortex.

  The processing in the third layer simulates the processing of complex cells in the primary visual cortex, and is a processing that shows the azimuth strength by unifying the phase difference for each frequency. Four types of ○ 12 to ○ 15 Feature data is calculated. The process in the third hierarchy is displayed as “V1 Complex” by taking a display of complex cells in the primary visual cortex.

  Auto-correlation and cross-correlation are calculated for feature data in each layer. In FIG. 2, “Auto” indicates autocorrelation, “Within” indicates cross-correlation between the same frequencies, and “Across” indicates cross-correlation between different frequencies.

  The types of autocorrelation calculated are as follows. First, the autocorrelation for the feature data ○ 2 and ○ 3 of the first layer is calculated as “LGN Auto”.

  For the eight feature data in the second layer, the cross-correlation within the same frequency and the same phase is calculated as “Simple Within”. Further, the cross-correlation between different frequencies among the eight feature data is calculated as “Simple Across”. In the embodiment, “Simple Access” calculates two types of real part and imaginary part by Fourier transform.

  The autocorrelation of the four feature data in the third hierarchy is calculated as “Complex Auto”. Further, the cross-correlation within the same frequency of the four feature data is calculated as “Complex Within”. Further, a cross-correlation between different frequencies of the four feature data is calculated as “Complex Across”.

  In addition to the calculation of autocorrelation and cross-correlation calculated as described above, moment for visual input, moment for feature data in the first layer (LGN moment), moment for feature data in the third layer ( (V1 Complex moment) is calculated (moment is a basic statistic of an image and is defined by one of average, variance, skewness, and kurtosis).

  Next, an example of setting an evaluation model will be described with reference to FIG. In the following description, it is assumed that the sensitivity to be evaluated is the “texture” of an object (article). First, in FIG. 3, G1, G2,... Gn indicate a large number of different images (for example, 500 to 1000) to be subjected to texture evaluation. The visual sensibility evaluation values (for example, the evaluation values in multiple stages regarding the quality of the texture) are associated with the large number of images. The texture evaluation value is raw data or an average value of evaluation values by a plurality of experts who perform texture evaluation.

  For each image, autocorrelation and cross-correlation are calculated as described with reference to FIG. One image has many correlation levels (correlation coefficients as numerical values) for autocorrelation and cross-correlation. For each autocorrelation and each cross-correlation, principal component analysis (for example, five dimensions) is performed, and extraction as shown by the column A in FIG. 3 is performed. Also, as shown by the B column in FIG. 3, parameters that greatly affect the texture evaluation are selected from the A column.

  Based on the parameters shown in the column A acquired by the principal component analysis, the expression (1) indicating the evaluation model is determined as shown in Equation 1.

  In the formula (1), Y is a texture evaluation value for each image, a0 to am, b1 to bm, c (suffix is omitted here) is a coefficient (constant), and X1 to Xm are , The correlation level in the column A shown in FIG. Initially, each coefficient is unknown. By applying the subjective texture evaluation value Y and the correlation levels X1 to Xm for the images G1 to Gn to the equation (1), each coefficient is determined by a regression method. In equation (1), which is a regression model, each coefficient is associated with a specific one of a number of autocorrelations and a number of cross-correlations.

  The expression (1) is determined so as to include the autocorrelation and all cross-correlations indicated by the extracted column A. From this equation (1), only parameters that have a large effect on the texture evaluation as shown in column B in FIG. 3 are left, parameters that do not affect the texture evaluation are deleted, and the evaluation model formula for the texture evaluation is as follows: (2). Therefore, the suffix n in the equation (2) is smaller than the suffix m in the equation (1). The expression (2) is rewritten so that the coefficient suffixes are continued after deleting the coefficients for the unnecessary parameters from the expression (1) (expressions (1) and (2)). And the same suffix coefficient does not necessarily indicate the same content).

  Here, the above equation (1) includes a second-order term and an interaction term (the third term on the right side excluding the initial value a0) for all the first-order terms. Actually, from the viewpoint of simplifying the evaluation model as much as possible, only a specific first-order term that has a predetermined influence or more on the subjective evaluation has a second-order term and an interaction term. It is preferable that the state after such processing is performed is Equation (2).

  When evaluating the texture of a certain object, an image of the certain object is acquired, autocorrelation and cross-correlation are calculated as shown in FIG. 2, and this calculation result is applied to the above equation (2) (autocorrelation, By applying each correlation level of the cross-correlation to X1 to Xn), the texture evaluation value for the certain object is calculated with a numerical value (it is also visualized by quantification). It should be noted that the validity of the evaluation model in the expression (2) is expressed by a plurality of images (autocorrelation and cross-correlation calculated from) other than the images G1 to Gn shown in FIG. It can also be used and verified.

  Next, an example of an overall control system for performing sensitivity evaluation will be described with reference to FIG. In FIG. 1, reference numeral 11 denotes a color image to be evaluated for visual sensitivity, and this image 11 is acquired by a camera. As described above, for the image 11, the feature data in the first to third layers is calculated by the image feature calculation unit 12, and the autocorrelation and mutual correlation are calculated by the visual statistic calculation unit 13 based on the feature data. A correlation is calculated.

  On the other hand, a plurality of evaluation models for visual sensitivity are set. In FIG. 1, the evaluation model A is for texture evaluation, and the evaluation model B is for aesthetic evaluation. Thus, a plurality of evaluation models are set according to the difference in sensitivity to be evaluated. As the evaluation model, other evaluation objects such as facial expression (sensitivity shown) can be set as appropriate.

  In each evaluation model, parameters according to the type of sensitivity to be evaluated are set by selection by the evaluation parameter selection unit P. For example, in texture evaluation, structure / luminance uniformity and luminance gradation are parameters (autocorrelation and cross-correlation are performed to detect structure / luminance uniformity and luminance gradation. ). In the aesthetic evaluation, for example, the symmetry of the luminance distribution is set as a parameter (autocorrelation and cross-correlation are performed so as to detect the symmetry of the luminance distribution). In this way, parameters are selected according to the sensitivity evaluation target, and autocorrelation and cross-correlation are calculated according to the selected parameters.

  In FIG. 1, personal information (for example, gender, age, occupation, etc.) is input in the personal information input unit 21, and the input personal information is pre-processed in the personal information pre-processing unit 22 (for example, age is classified into a plurality of levels). Is recorded (stored) in the personal information recording unit 23.

  The biological information input by the biological sensor input unit 24 and the subjective sensitivity evaluation value input by the subjective evaluation unit 25 are input to the actual measured sensitivity estimation unit 26 to estimate the actual measured sensitivity value. The actual measured sensitivity value is recorded (stored) in the actual measured sensitivity value recording unit 27.

  Among the scenes input by the scene information input unit 28, the evaluation target is specified by the evaluation target specifying unit 29. The evaluation target (image) specified by the evaluation target specifying unit is recorded (stored) in the evaluation target recording unit 30. The calculation result in the visual statistic calculation unit 13 described above is recorded (stored) in the visual statistic recording unit 31.

  Each of the recording units 23, 27, 30, and 31 described above constitutes the database unit D. An evaluation model is set based on the data in the database part D. Specifically, data in the database unit is input to the evaluation parameter selection unit P, and an appropriate parameter (evaluation axis) is selected according to the type of sensitivity to be evaluated (in column B of FIG. 3). The parameter selection corresponds), and an evaluation model is set according to the selected parameter (the evaluation model shown in the equation (2) is set).

The recorded contents in the recording unit 30 are used as a large number of images G1 to Gn shown in FIG. 3, and the recorded contents in the recording unit 27 are used as subjective evaluation values for the images G1 to Gn.
The personal information input unit 21 is for correcting the evaluation value according to personal information (for example, gender or age group), and may be omitted. The recording unit 31 is for learning correction of the evaluation model when a new sensibility evaluation is performed, and may be omitted when the learning correction is not performed.

  The evaluation parameter selection unit selects an evaluation parameter (evaluation axis) according to the content (kind) of the sensitivity to be evaluated. As described above, when the sensitivity to be evaluated is, for example, a texture, the evaluation parameters are, in particular, the structure / luminance uniformity and the luminance gradation (uniformity). In addition, when the sensibility to be evaluated is, for example, aesthetics, the luminance distribution is symmetric. In this way, appropriate parameters are selected according to the content of the sensibility to be evaluated, the evaluation model is set as described above, and evaluation of appropriate sensibilities on the sight such as sensibilities exhibited by other human facial expressions. A model can be set.

  In the control system of FIG. 1, any one evaluation model is set (selected) by the evaluation model setting unit 41. For example, when texture evaluation is performed, the evaluation model setting unit 41 sets an evaluation model A for texture evaluation. Then, the emotion evaluation unit 42 applies the autocorrelation and cross-correlation correlation levels (correlation coefficients) calculated by the visual statistic calculation unit 13 to the evaluation model A to calculate the texture (output as an evaluation value). .

  In order to make the evaluation model A more appropriate, for the image acquired by the evaluation target image acquisition unit 11, when the texture evaluation value by the texture evaluation specialist is different from the texture evaluation value by the sensitivity evaluation unit 42, the above acquisition The evaluation model A (Equation (2) corresponding to this) can be learned and corrected by including the processed image in the image of FIG. 3 and including the texture evaluation value by the texture evaluation specialist.

  Next, autocorrelation and cross-correlation will be described in more detail with reference to the case of texture evaluation. First, the autocorrelation in the third hierarchy used in the texture evaluation will be described. In this case, as the autocorrelation for the feature data of the third hierarchy (for example, ◯ 12 in FIG. 2), the correlation between the original image serving as the feature data of the third hierarchy and the copy image is acquired. Specifically, both images are compared with each other in a state where the copy image is shifted by a predetermined amount in the horizontal direction with respect to the original image, and the correlation is acquired (acquisition of correlation). The same process is repeated by sequentially shifting the copy images in the horizontal direction.

  Next, the two images are compared with each other in a state where the copy image is shifted vertically by a predetermined amount with respect to the original image (acquisition of correlation). The same is repeated by sequentially shifting the copy images in the vertical direction.

  The correlation coefficient indicating the correlation obtained as described above is, for example, an autocorrelation map as shown in FIG. In the map shown in FIG. 4, the average value of the correlation coefficients in the peripheral area including the portion where the correlation coefficient is maximum (the portion indicated by ○ 1 in FIG. 4) is the correlation coefficient for the original image. The

  The same is performed for the images ○ 13 to ○ 15 in the third hierarchy shown in FIG. 2, and autocorrelation correlation coefficients are obtained for all feature data in the third hierarchy. An average value of a plurality of correlation coefficients calculated for each of the images ◯ 12 to ◯ 15 is calculated, and this average value is used as a correlation coefficient indicating autocorrelation for feature data in the third hierarchy.

  Next, details of obtaining the cross-correlation in the second hierarchy will be described. In the following description, an example is described in which cross-correlation between an image of ○ 4 in the second hierarchy in FIG. 2 and an image of ○ 8 having a different frequency with respect to the image of ○ 4 is obtained. In this case, as shown in FIG. 5, two images having different frequencies are overlapped to obtain a correlation image that represents a gradation regarding luminance (the higher the uniformity of luminance change, the higher the correlation is. ) Correlation images that represent gradations of luminance are similarly acquired between images having different orientations of 90 degrees, 45 degrees, 0 degrees, and -45 degrees (in FIG. 5, feature data in the second hierarchy is shown). Is shown in the case of four types). Thereafter, an average value of the correlation coefficients in the peripheral portion including the portion having the largest phase correlation coefficient regarding the uniformity of the luminance change is calculated from the sum of the correlation images for all the directions. The same is performed for other feature data (images) in the second layer, and the average value of all the acquired cross-correlations is calculated, and this average value is calculated for the feature data in the second layer. The correlation coefficient indicates a cross correlation.

  Using the autocorrelation (correlation coefficient) for the feature data in the third hierarchy and the cross-correlation (correlation coefficient) for the feature data in the second hierarchy obtained as described above as parameters, A map showing the level of texture is set, for example, as shown in FIG. 6 (visualized evaluation form). FIG. 6 is a map type evaluation model. In the map shown in FIG. 6, for example, five areas α1, α2, α3, α4, and α5 are set as the area division. The area α1 is the highest quality, the area α2 is the second highest quality, and the area α3 is the middle. This indicates that the region α4 is not fine and the region α5 is least fine.

  In FIG. 6, white circles and black circles indicate the results of the texture evaluation for the instrument panel in an actual vehicle. A vehicle with a white circle is evaluated as a passing level having a texture level of a predetermined level or higher. On the other hand, a vehicle with a black circle is evaluated as having a low texture. The original image (corresponding to the image 1 in FIG. 2) of the instrument panel in various evaluated vehicles is, for example, as shown in FIG. In addition, in FIG. 7, the decoration part with high brightness | luminance is attached | subjected. The auto-correlation of the third feature data and the cross-correlation of the feature data in the second layer are obtained from the original images as shown in FIG. 7 for various vehicles, and the white circles and black circles are collated in FIG. It is a mark.

  By performing a texture evaluation using a map as shown in FIG. 6, it is possible to know at a glance the difference in texture in relation to other vehicles. In addition, using FIG. 6, it is obvious how the texture can be increased (or decreased). That is, it is understood that in the vehicle indicated by the lower circle of the two black circles, the uniformity of the structural luminance is almost the same, and the texture can be enhanced by increasing the luminance gradation. Is done. Further, it is understood that in the vehicle with the black circle on the upper side, the gradation of luminance is almost the same, and the texture can be enhanced by improving the uniformity of the structure and luminance. In particular, the texture evaluation using the map of FIG. 6 may be performed by changing either the structure / luminance uniformity or the luminance gradation in order to increase (or decrease) the texture. Or it becomes possible to judge whether it is necessary to change both.

  Although the embodiments have been described above, the present invention is not limited to the embodiments, and appropriate modifications can be made within the scope of the claims. For example, when evaluating a texture, an evaluation model can be set for each type of object to be evaluated. For example, in a vehicle, the evaluation for the instrument panel portion, the evaluation for the seat, and the evaluation for the door interior can be set as separate evaluation models. It is also possible to set a plurality of evaluation models divided by age, gender, etc. for models that evaluate the texture of the same type of object (for example, evaluation models corresponding to young females, male seniors, etc. individually). It is possible to perform evaluations in various fields such as evaluation of interiors including furniture as well as vehicles. The evaluation model can be set in consideration of various moments shown in FIG. 2 (including values based on the moment, such as the square of the moment). Of course, the object of the present invention is not limited to what is explicitly stated, but also implicitly includes providing what is substantially preferred or expressed as an advantage.

  In the present invention, visual sensitivity can be quantified and evaluated.

11: Evaluation target image acquisition unit 12: Image feature calculation unit 13: Visual statistics calculation unit 25: Subjective evaluation input unit 28: Scene information input unit (input unit of evaluation target image)
41: Evaluation model setting unit 42: Kansei evaluation unit

Claims (8)

Image acquisition means for acquiring an image to be evaluated for a predetermined sensitivity;
Image feature calculation means configured to simulate hierarchical visual information processing in the human brain, and to calculate feature data for each hierarchy of the image acquired by the image acquisition means;
A visual statistic calculating means for calculating auto-correlation and cross-correlation for the feature data calculated by the image feature calculating means;
An evaluation model storage means that stores preset evaluation models for the predetermined sensibilities, which are set in advance as parameters of autocorrelation and cross-correlation of the feature data of each of the layers,
A sensitivity evaluation unit that compares the autocorrelation and cross-correlation calculated by the visual statistic calculation means with the evaluation model, and evaluates the sensitivity of the image to be evaluated,
Kansei evaluation system characterized by having In claim 1,
The image feature calculation means includes a plurality of first feature data by frequency processing that is processing in the first hierarchy, a plurality of second feature data by orientation selection processing that is processing in the second hierarchy, and a third hierarchy. A sensibility evaluation system characterized by calculating a plurality of third feature data by azimuth signal intensity processing which is the processing of. In claim 1 or claim 2,
A sensitivity evaluation system further comprising learning means for learning and correcting the evaluation model based on autocorrelation and cross-correlation calculated by the visual statistic calculation means. In any one of Claim 1 thru | or Claim 3,
A plurality of types of sensibilities to be evaluated are set, and a plurality of the evaluation models are set corresponding to the plurality of sensibilities,
Model selection means for selecting one evaluation model from the plurality of evaluation models according to the predetermined sensitivity to be evaluated,
The Kansei evaluation means collates the autocorrelation and cross-correlation calculated by the visual statistic calculation means against the evaluation model selected by the model selection means,
Kansei evaluation system characterized by that. In any one of Claims 1 thru | or 4,
The evaluation model is determined by performing a principal component analysis on the autocorrelation and the cross-correlation acquired for a large number of images, and then selecting a parameter for evaluating the predetermined sensitivity. Kansei evaluation system. In claim 5,
The evaluation model is set by the following equation (where Y is a subjective evaluation value of sensitivity, a, b, and c are coefficients, and X is the autocorrelation or the cross-correlation). Evaluation system.

In claim 2,
A sensitivity evaluation system, wherein the evaluation of the predetermined sensitivity is a texture evaluation. In claim 7,
The sensitivity evaluation system, wherein the evaluation model is set with parameters of autocorrelation of the third feature data and cross-correlation of the second feature data.

Images