JP2012043357A - Usability evaluation device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which it is impossible to evaluate usability for a visual feature of an image displayed on a screen such as a layout of contents on the screen and a design of an operation object.SOLUTION: A usability evaluation device comprises usability information generation means which generates and outputs usability information for evaluating whether it is possible to attract attention to a specific part in an image based on an element remarkability value indicating remarkability for each unit element of the image.

Description

  The present invention relates to a usability evaluation apparatus, method and program, and more particularly to a usability evaluation apparatus, method and program for evaluating visual usability.

  As a technique for evaluating usability, there is a related technique described in Non-Patent Document 1. The usability evaluation method described in Non-Patent Document 1 uses a means for calculating a semantic similarity of a character written on an operation target object and a user's operation purpose, and the calculated semantic similarity. A means for extracting the problems is provided. The user's operation purpose is, for example, “view a chiffon cake recipe”, assuming a scene where the user searches for a recipe for a chiffon cake in the terminal.

  In the means for calculating the semantic similarity, the semantic similarity between the character string of the label of the operation target object and the character string indicating the user's operation purpose or the label of the operation target object is calculated. In the means for extracting the usability problem, for example, when the value of the semantic similarity between the operation target objects is high, it is determined that the operation target objects are likely to be confused and the usability problem is determined.

  In Non-Patent Document 2, a feature map is created based on each feature quantity such as the color, inclination, and line branch of an input image, and the saliency of these feature maps is integrated, so that the Saliency Map (saliency map) is obtained. Techniques for creating are described. Non-Patent Document 2 describes a technique for predicting a user's gaze position coordinate by detecting a position coordinate having the highest saliency value on a saliency map.

  In Non-Patent Document 3, a feature vector is extracted from a web page to which a non-content area is assigned, and a model for detecting the non-content area by chunking is extracted from the extracted feature vector. Describes a method for automatically detecting.

  Japanese Patent Application Laid-Open No. H10-228561 describes a means for specifying a partial area from an image display area and determining whether or not this satisfies a predetermined condition.

  Patent Document 2 describes a technique for classifying images according to feature amounts.

  Patent Document 3 describes means for calculating the feature amount of image data.

JP 2006-217123 A JP 2007-317034 A JP 09-091430 A

Blackmon, MH and Polson, PG and Kitajima, M. and Lewis, C., "Cognitive walkthrough for the web", Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, 463-470, 2002 , ACM New York, NY, USA Itti, L. and Koch, C. and Niebur, E. and others, "A model of saliency-based visual attention for rapid scene analysis", IEEE Transactions on pattern analysis and machine intelligence, vol.20 (11), pp. 1254-1259, 1998, Citeseer Tatsuya Nakamura and Kiyoaki Shirai, “Detection of non-content areas in web pages”, Proc. Of the 13th Annual Conference of the Language Processing Society of Japan, 234-237, 2007

However, in the technology described in the above-described prior art documents, there is a problem that usability cannot be evaluated with respect to the visual characteristics of the image displayed on the screen, such as the layout of the content on the screen and the design of the operation target object. there were.
(Object of invention)
The objective of this invention is providing the usability evaluation apparatus, method, and program which can solve the problem mentioned above.

  The usability evaluation system of the present invention provides usability information for evaluating whether or not attention can be drawn to a specific portion in the image based on an element saliency value indicating saliency for each unit element of the image. Usability information generating means for generating and outputting.

  According to the method of the present invention, usability information for evaluating whether a processing unit can draw attention to a specific portion in the image based on an element saliency value indicating saliency for each unit element of an image. Is generated.

  The program of the present invention provides usability information for evaluating whether or not the processing unit can draw attention to a specific part in the image based on an element saliency value indicating saliency for each unit element of the image. Causes the computer to execute the process of generating

  The present invention has an effect that it is possible to evaluate usability with respect to a visual feature of an image.

It is a block diagram which shows the structure of 1st Embodiment. It is a figure which shows the example of the evaluation object image of 1st thru | or 6th embodiment. It is a figure which shows an example of the data of the remarkable map in 1st thru | or 6th embodiment. It is an example of the saliency map image which represented the saliency map in 1st thru | or 6th embodiment as an image. It is a flowchart which shows the operation | movement of creation of the saliency map of 1st thru | or 5th embodiment. It is a flowchart which shows operation | movement of the process part of 1st Embodiment. It is a block diagram which shows the structure of 2nd Embodiment. It is a figure which shows an example of the item information acquired in 2nd and 3rd embodiment. It is a figure which shows an example of the data of the remarkable list | wrist produced | generated in 2nd and 4th embodiment. It is a flowchart which shows operation | movement of the process part of 2nd Embodiment. It is a block diagram which shows the structure of 3rd Embodiment. It is a flowchart which shows operation | movement of the process part of 3rd Embodiment. It is a block diagram which shows the structure of 4th Embodiment. It is a figure which shows an example of the item information acquired in 4th Embodiment. It is a figure which shows an example of the data of the semantic similarity list in 4th Embodiment. It is a figure which shows an example of the user operation target object information in 4th Embodiment. It is a flowchart which shows operation | movement of the process part of 4th Embodiment. It is a block diagram which shows the structure of 5th Embodiment. It is a figure which shows an example of the item information produced | generated in 5th Embodiment. It is a figure which shows an example of the data of the content area remarkable list | wrist in 5th Embodiment. It is a figure which shows an example of the data of the non-content area | region remarkable list | wrist in 5th Embodiment. It is a flowchart which shows operation | movement of the process part of 5th Embodiment. It is a block diagram which shows the structure of 6th Embodiment.

[First Embodiment]
FIG. 1 is a block diagram showing the configuration of the first exemplary embodiment of the present invention. As shown in FIG. 1, the usability evaluation apparatus 100 according to the present embodiment includes an input unit 110, a processing unit 120, and an output unit 130.

  The input unit 110 may be, for example, an input device including a keyboard and a mouse, or a data input device including a computer.

  The input unit 110 receives input of the evaluation target image information G10, the saliency threshold t, and the number of pixels (also referred to as unit elements) n, and outputs the image information G20 and information of the saliency threshold t and the number of pixels n. FIG. 2 is a diagram illustrating an example of the evaluation target image G10.

  The image information G20 is data for drawing an evaluation target image G10 as shown in FIG. 2, for example. Specifically, the image information G20 is, for example, drawing data of a bitmap image in which the evaluation target image G10 is divided into pixels and the color and density of each pixel are expressed as numerical values.

  The saliency threshold t indicates a criterion for determining whether or not a pixel in the evaluation target image G10 is visually noticeable, and is a value such as “0.31”, for example. The pixel number n is a pixel having a visual saliency value (corresponding to a saliency value M12 described later, also referred to as an element saliency value) among the pixels in the evaluation target image G10 (hereinafter referred to as a saliency threshold t). , Called a salient pixel), for example, a value such as “8000”. The number of pixels n may be indicated by the upper limit value of the ratio of the salient pixels among the pixels in the evaluation target image G10.

The processing unit 120 includes a saliency map creation unit 121 and a usability information generation unit 122.
The processing unit 120 may be configured by a computer including a central processing unit (CPU), a memory, and a hard disk. In this case, each process in the saliency map creation unit 121 and the usability information generation unit 122 may be executed based on a program stored in a computer storage medium.

  The saliency map creating unit 121 receives the image information G20 output from the input unit 110, calculates a saliency value M12 that is an element saliency value of each pixel of the image information G20, and based on the saliency value M12. Data of the saliency map M10 as shown in FIG. 3 is created. The saliency map creation unit 121 outputs the saliency map M10 to the usability information generation unit 122. The saliency value M12 is information indicating the visual saliency of the pixel. Here, the saliency value is, for example, a value indicating characteristics of luminance, color, and direction of each pixel that is a unit element of the image.

  FIG. 3 is a diagram illustrating an example of data of the saliency map M10. As shown in FIG. 3, the saliency map M10 is data in which the position coordinates (x coordinate, y coordinate) M11 (not shown) of each pixel in the image information G20 are associated with the visual saliency value M12. The saliency map M10 shown in FIG. 3 shows a case where the evaluation target image G10 has “640 × 480” pixels as an example. For example, regarding the position coordinates M11 of the uppermost and leftmost pixels, the x coordinate is “1”, the y coordinate is “1”, and the saliency value M12 is “0.01”. Further, the position coordinate M11 of the lowermost and rightmost pixel has an x coordinate of “640”, a y coordinate of “480”, and a saliency value M12 of “0.21”.

  Note that it is generally considered that the human bottom-up visual attention is guided to a position in the image where the visual saliency value is high.

  FIG. 4 is an example of a saliency map image representing the saliency map M10 of the evaluation target image G10 as an image.

  FIG. 5 is a flowchart showing an operation of creating the saliency map M10 in the saliency map creation unit 121.

  The saliency map creation unit 121 applies an image filter to the input image information G20 to extract each feature such as color, brightness, and direction, and creates a feature map M13 (not shown) of each feature (step S111). ).

  Next, the saliency map creation unit 121 normalizes each feature map M13 (step S112).

  Next, the saliency map creation unit 121 creates a saliency map M10 by integrating the normalized feature maps and outputs the saliency map M10 to the usability information generation unit 122 (step S113).

  As an example of the saliency map creation unit 121, a saliency map creation unit described in Non-Patent Document 2 can be cited.

  The procedure for creating the saliency map M10 of the saliency-based gaze model described in this document is as follows.

  First, the saliency-based gaze model creates a Gaussian pyramid by applying a Gaussian filter to the input evaluation target image G10 (hereinafter, a plurality of images constituting the Gaussian pyramid are referred to as a scale image G30 (not shown)). ).

  Next, the saliency-based gaze model calculates the characteristics (luminance / color / direction) of each scale image G30, and holds the values of the characteristics (luminance / color / direction) of each scale image G30 (not shown). ).

  Next, the saliency-based gaze model calculates a difference between the scales of the image G31 for each feature (brightness, color, direction), and creates a feature map M13 that holds the calculated difference information.

  Next, the saliency-based gaze model normalizes the feature map M13 in order to integrate the maps for each feature, and creates an image G33 (not shown) in which the maps are integrated for each feature.

  Finally, the saliency-based gaze model normalizes the image G33 and integrates it into one map to obtain a saliency map M10.

  This is an example of means for creating the saliency map M10, and the method for creating the saliency map M10 is not limited to this.

  Since it is sufficient that a map representing the visual saliency of the evaluation target image G10 can be created, a part of the process of the flow shown in FIG. 5 may be omitted in the saliency map creation process. Specifically, for example, the step of normalizing the feature map M13 may be omitted.

  The usability information generation unit 122 generates usability information for evaluating whether or not attention can be drawn to a specific portion in the evaluation target image based on the saliency value M12 of each pixel of the saliency map M10.

  Specifically, the usability information generation unit 122 generates usability information based on, for example, whether or not the number of salient pixels is greater than or equal to the number of pixels n. The usability information generation unit 122 determines whether the number of salient pixels is greater than or equal to the pixel number n. When the usability information generation unit 122 determines that the number of remarkable pixels is greater than or equal to the pixel number n, the usability information generation unit 122 generates usability information indicating that there is a problem of excessive pixel excess (a large recognition load on the user). Further, when the usability information generation unit 122 determines that the number of salient pixels is less than the pixel number n, the usability information generation unit 122 generates usability information indicating “no problem of excessive salient pixels” and outputs the usability information to the output unit 130.

  In addition, the usability information generation unit 122, for example, based on whether or not the number of pixels having the saliency value M12 equal to or greater than each saliency threshold t for each of the plurality of saliency thresholds t is greater than or equal to the pixel number n. New usability information may be generated. Specifically, the usability information generation unit 122 determines the saliency threshold t and the saliency value based on the four saliency thresholds t for determining whether the problem of usability is large, during problem, small problem, and no problem, respectively. Compare with M12. Next, the usability information generation unit 122 determines whether the number of pixels having the saliency value M12 equal to or greater than each saliency threshold t is equal to or greater than the number of pixels n (n is different for each saliency threshold). Based on, usability information indicating large usability problems, medium problems, small problems, and no problems is generated and output to the output unit 130.

  In addition, the usability information generation unit 122 may generate usability information based on, for example, the ratio between the number of salient pixels and the number of pixels n. Specifically, the usability information generation unit 122 generates, as usability information, a numerical value representing the ratio of the number of saliency pixels to the number of pixels n in 100 minutes.

  Note that the saliency threshold value t and the pixel number n data are not supplied from the input unit 110 but may be set in the usability information generation unit 122 in advance.

  The output unit 130 includes a display device that displays the processing result of the processing unit 120. The processing result is, for example, a figure (superimposed image) in which the saliency value M12 of the saliency map M10 obtained from the image information G20 is superimposed on the evaluation target image G10 input by the input unit 110 in addition to the above usability information. including. In this case, the usability information generation unit 122 outputs usability information and information of an image obtained by superimposing the saliency map M10 on the image information G20 of the evaluation target image G10 (this information is also called usability information). It is desirable to transmit to 130.

  Further, the output unit 130 may input the image information G20 and the saliency value M12 of the saliency map M10 as the processing result from the processing unit 120, and generate and display an image obtained by superimposing them.

  Next, the operation of this embodiment will be described in detail with reference to FIGS.

  FIG. 6 is a flowchart illustrating an example of the operation of the processing unit 120 of the present embodiment.

  Prior to the operation shown in FIG. 6, the input unit 110 receives an input of the image information G20 and transmits it to the saliency map creation unit 121 of the processing unit 120. In addition, the input unit 110 receives input of values of the saliency threshold t and the number of pixels n, and transmits them to the usability information generation unit 122 of the processing unit 120.

  The saliency map creation unit 121 of the processing unit 120 starts to operate when the image information G20 is received.

  First, the saliency map creation unit 121 of the processing unit 120 calculates the saliency value M12 of each pixel of the image information G20, and creates the saliency map M10 based on the calculated saliency value M12. Subsequently, the saliency map creation unit 121 transmits the created saliency map M10 to the usability information generation unit 122 (step S101).

  The usability information generation unit 122 receives the saliency map M10, and determines whether the number of saliency pixels is greater than or equal to the pixel number n for the saliency map M10 (step S102).

  If the number of salient pixels is greater than or equal to the number of pixels n (YES in step S102), the usability information generation unit 122 generates usability information indicating that there is a problem of excessive pixel count (step S103).

  When the number of remarkable pixels is less than the number of pixels n (NO in step S102), the usability information generation unit 122 generates usability information indicating that there is no problem of excessive pixels (step S104).

  Next, the usability information generation unit 122 is a diagram in which the saliency map M10 is drawn on the usability information generated in either step S103 or step S104 and the evaluation target image G10 (this information is also called usability information). Is transmitted to the output unit 130 (step S105).

  With the above, the operation of the processing unit 120 ends, and following these series of operations, the output unit 130 displays the received usability information and a diagram in which the saliency map M10 is drawn on the evaluation target image G10. .

  By confirming the above-described display, the user of the usability evaluation apparatus 100 is not designed so that there are many positions where the user's visual attention is likely to be induced and the user is subjected to an unnecessary cognitive load. Evaluate usability of whether or not.

  The effect in the present embodiment described above is that it is possible to evaluate the usability with respect to the visual characteristics of the image. That is, according to this embodiment, it is possible to evaluate the usability of the image displayed on the screen, such as the layout of the content on the screen and the design of the operation target object, with respect to the visual features.

The reason is that the saliency value M12 of the evaluation target image G10 is calculated, and based on the calculated saliency value M12, it is determined whether or not the number of saliency pixels is greater than or equal to a predetermined number of pixels. It is because it was made to produce | generate.
[Second Embodiment]
Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 7 is a block diagram showing the configuration of the present embodiment. As shown in FIG. 7, the usability evaluation apparatus 200 according to the present embodiment includes an input unit 210, a processing unit 220, and an output unit 230.

  The input unit 210 is equivalent to the input unit 110 described in the first embodiment.

  The processing unit 220 includes a saliency map creation unit 121, an operation target object detection unit 223 that detects an operation target object on the screen, an operation target object saliency calculation unit 224, and a usability information generation unit 222. The operation target object detection unit 223, the operation target object saliency calculation unit 224, and the usability information generation unit 222 may be configured by a computer including a central processing unit (CPU), a memory, and a hard disk. In this case, each process may be executed based on a program stored in a computer storage medium.

  The saliency map creation unit 121 is the same as that described in the first embodiment.

  The operation target object detection unit 223 includes information of the image information G20 received from the input unit 210 (bitmap image drawing data described in the first embodiment, screen display data described in HTML (HyperText Markup Language), etc.) ) To detect one or a plurality of operation target objects on the screen. Then, the operation target object detection unit 223 acquires at least the position coordinates M11 of each operation target object as the specification information P10 of the detected operation target object as shown in FIG. 8, and further, the width and height of the operation target object. This information is acquired (generated), and the acquired item information P10 is transmitted to the operation target object saliency calculating unit 224. FIG. 8 is a diagram illustrating an example of the specification information P10. As shown in FIG. 8, the specification information P10 includes an operation target object identifier M21, a position coordinate M11, a width, and a height.

  The operation target object detection unit 223 may detect the operation target object based on the input by the usability evaluator and acquire the specification information P10. Further, the operation target object detection unit 223 may automatically detect the operation target object from the image information G20 and acquire the specification information P10.

  In the case of means for manually inputting an operation target object, for example, the input unit 210 and the output unit 230 may have an interface therefor. Specifically, the output unit 230 displays the evaluation target image G10, and the input unit 210 receives information on the button area on the evaluation target image G10 that is designated by the mouse. Next, the operation target object detection unit 223 acquires the specification information P10 including the position coordinates M11, the width, and the height of the operation target object based on the button area information received by the input unit 210.

  In the case of means for automatically detecting an operation target object, the operation target object detection unit 223 detects the operation target object with reference to, for example, screen display data described in HTML. Specifically, the operation target object detection unit 223, for example, describes a start tag <a href=“ “> and an end tag </a> that is a description portion of a link from within screen display data described in HTML. (Character string) is detected. Then, the operation target object detection unit 223 extracts a character string W10 (not shown) described between the detected start tag and end tag, and detects this as an operation target object. Next, the operation target object detection unit 223 acquires and outputs specification information P10 including the position coordinates M11, width, and height of the character string W10 that is the detected operation target object.

  These are examples of the operation of the operation target object detection unit 223, and the method for acquiring the specification information P10 is not limited to this.

  The operation target object saliency calculating unit 224 generates a saliency list M20 as shown in FIG.

  FIG. 9 is a diagram illustrating an example of data of the saliency list M20. As shown in FIG. 9, the saliency list M20 includes one or more sets in which the operation target object identifier M21 and the saliency value M22 are associated with each other. The operation target object saliency calculation unit 224 calculates a saliency value M22 of each operation target object based on the saliency map M10 from the saliency map creation unit 121 and the specification information P10 from the operation target object detection unit 223. The saliency list M20 is generated. For example, the operation target object saliency calculating unit 224 detects the maximum value of the saliency value M12 in each pixel of the operation target object region corresponding to the specification information P10, and uses the maximum value as the saliency value M22 of the operation target object. And

  The usability information generation unit 222 generates usability information based on the saliency value M22 of each operation target object from the operation target object saliency calculation unit 224.

  First, the usability information generation unit 222 compares, for example, the saliency value M22 of the operation target object with the maximum saliency value M12 of the saliency values M12 of the region other than the operation target object. When the saliency value M22 of the operation target object is equal to or less than the maximum saliency value M12 in the region other than the operation target object, the usability information generation unit 222 indicates that “the problem is not the saliency of the operation target object with respect to the region other than the operation target object Usability information indicating “Yes” is generated. When the saliency value M22 of the operation target object exceeds the maximum saliency value M12 among the saliency values M12 of the region other than the operation target object, the usability information generation unit 222 determines that “the operation on the region other than the operation target object Usability information indicating that there is no problem with the saliency of the target object is generated.

  The usability information generation unit 222 replaces the maximum saliency value M12 in the saliency value M12 of the region other than the operation target object, and adds a predetermined value to the maximum saliency value M12. And the saliency value M22 of the operation target object may be compared.

  Further, the usability information generation unit 222 compares the saliency value M22 with each other. When the difference between any of the saliency values M22 is equal to or greater than a predetermined difference threshold d, the usability information generation unit 222 generates usability information indicating that there is a problem with the saliency difference between the operation target objects. When the difference between the saliency values M22 is less than the predetermined difference threshold d, the usability information generation unit 222 generates usability information indicating that there is no problem in the saliency difference between the operation target objects.

  Note that the usability information generation unit 222 may further generate usability information in which the number of significant pixels is based on the number of pixels, similar to the usability information generation unit 122 of the first embodiment.

  The output unit 230 displays the processing result of the processing unit 220. The processing result is a diagram in which the above usability information, the saliency value M22 of each operation target object, and the saliency value M12 of the evaluation target image G10 are superimposed on the evaluation target image G10 (this information is also referred to as usability information). It is.

  Next, the operation of the present embodiment will be described in detail with reference to FIGS. In the following description of the operation of the present embodiment, the operation described in the first embodiment is omitted within a range that does not obscure the description of the operation of the present embodiment.

  FIG. 10 is a flowchart showing the operation of the processing unit 220 of this embodiment.

  Prior to the operation shown in FIG. 10, the input unit 210 receives input of the image information G20 and transmits it to the saliency map creation unit 121 and the operation target object detection unit 223.

  First, the saliency map creating unit 121 creates the saliency map M10 as in the first embodiment. Subsequently, the saliency map creation unit 121 transmits the created saliency map M10 to the usability information generation unit 222 and the operation target object saliency calculation unit 224 (step S201).

  Next, the operation target object detection unit 223 detects the operation target object in the received image information G20, and transmits the detected specification information P10 of each operation target object to the operation target object saliency calculation unit 224 (step S22). S202).

  Next, the operation target object saliency calculating unit 224 receives the saliency map M10 and the specification information P10 of the operation target object, and based on these, calculates the visual saliency value M22 of each operation target object. The saliency list M20 is generated. Then, the operation target object saliency calculating unit 224 transmits the generated saliency list M20 to the usability information generating unit 222 (step S203).

  Next, the usability information generation unit 222 receives the saliency map M10 and the saliency list M20. Subsequently, the usability information generation unit 222 compares the saliency value M22 of the operation target object with the maximum saliency value M12 of the saliency values M12 of the area other than the operation target object (step S204).

  When the saliency value M22 of the operation target object is equal to or smaller than the maximum saliency value M12 among the saliency values M12 of the region other than the operation target object (YES in step S204), the usability information generation unit 222 Usability information indicating that there is a problem in the saliency of the operation target object with respect to the region other than the operation target object is generated and transmitted to the output unit 230 (step S205).

  When the saliency value M22 of the operation target object exceeds the maximum saliency value M12 among the saliency values M12 of the region other than the operation target object (NO in step S204), the usability information generation unit 222 Generates usability information indicating that there is no problem in the saliency of the operation target object with respect to the region other than the operation target object, and transmits this to the output unit 230 (step S206).

  Next, the usability information generation unit 222 compares the saliency values M22 shown in FIG. 9 with each other (step S207).

  When the difference between the saliency values M22 is equal to or greater than a predetermined difference threshold d (YES in step S206), the usability information generation unit 222 displays usability information indicating that there is a problem with the saliency difference between the operation target objects. It is generated and transmitted to the output unit 230 (step S208).

  When the difference between the saliency values M22 is less than the predetermined difference threshold d (NO in step S206), the usability information generation unit 222 displays usability information indicating that there is no problem in the saliency difference between the operation target objects. It is generated and transmitted to the output unit 230 (step S209).

  Next, the usability information generation unit 222 generates a figure (this information is also referred to as usability information) depicting the saliency list M20 and the corresponding saliency map M10 on the evaluation target image G10, and transmits it to the output unit 230 (Step S209).

  With the above, the operation of the processing unit 220 ends, and following these series of operations, the output unit 230 draws the above usability information, the saliency list M20, and the saliency map M10 corresponding to the evaluation target image G10. Display the figure.

  By confirming the above display, the user of the usability evaluation apparatus 200 evaluates whether or not the user's attention is directed to the operation target object.

  The effect of the present embodiment described above is that it is possible to evaluate whether or not the user's visual attention is guided to the operation target object in addition to the effect of the first embodiment. .

The reason is that the usability information is generated by comparing the saliency value M12 of the entire screen with the saliency value M22 of each operation target object on the screen, and further comparing the saliency value M22 between the operation target objects. It was because it did so.
[Third Embodiment]
Next, a third embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 11 is a block diagram showing the configuration of the third exemplary embodiment of the present invention. As shown in FIG. 11, the usability evaluation apparatus 300 according to the present embodiment includes an input unit 310, a processing unit 320, and an output unit 330.
The input unit 310 is equivalent to the input unit 110 described in the first embodiment.

  The processing unit 320 includes a saliency map creation unit 121, a gaze position prediction unit 325, an operation target object detection unit 223, and a usability information generation unit 322. The saliency map creation unit 121 and the operation target object detection unit 223 are equivalent to those described in the first and second embodiments.

  The gaze position prediction unit 325 refers to the saliency map M10, predicts the position coordinates M11 at which the user's gaze is guided, and transmits the prediction result. In general, the user's gaze is guided to a place having the highest saliency value M12. Therefore, first, the gaze position prediction unit 325 sets, for example, the highest saliency value M12 among the saliency values M12 of the saliency map M10 as the gaze saliency value M32 (not shown). Subsequently, the gaze position prediction unit 325 detects the position coordinate M11 of the pixel having the gaze saliency value M32 as the gaze position coordinate C30 (not shown). Then, the gaze position prediction unit 325 transmits the gaze position coordinates C30 as a prediction result. It is assumed that the gaze position coordinate C30 is a position where the user's gaze is guided.

  For example, the gaze position prediction unit 325 may detect the gaze position coordinates C30 as follows. First, the gaze position prediction unit 325 calculates an average value M35 (not shown) of a difference obtained by subtracting the saliency value M12 of each pixel adjacent to the pixel from the saliency value M12 of each pixel. Next, the gaze position prediction unit 325 detects the position coordinate M11 of the pixel having the highest average value M35 among the average values M35 as the gaze position coordinate C30.

  The usability information generation unit 322 extracts usability information based on the user's gaze position coordinates C30 and the specification information P10 received from the operation target object detection unit 223.

Specifically, the usability information generation unit 322 determines whether the gaze position coordinate C30 is within the region of the operation target object based on the specification information P10. If the gaze position coordinate C30 is within the region of the operation target object, the usability information generation unit 322 generates usability information indicating that there is no problem in the operation target object versus the gaze position relationship. That is, usability information indicating no problem indicates that the user's gaze position is designed to be guided to the operation target object. Further, if the gaze position coordinate C30 is outside the region of the operation target object, the usability information generation unit 322 generates usability information indicating that there is a problem in the operation target object versus the gaze position relationship.
Note that the usability information generation unit 322 may determine whether or not the gaze position coordinate C30 is within a predetermined region.
Further, the usability information generation unit 322 may further generate usability information similar to the usability information generation unit 122 of the first embodiment and the usability information generation unit 222 of the second embodiment.

  Next, the operation of this embodiment will be described in detail with reference to FIG. In the following description of the operation of the present embodiment, the operations described in the first and second embodiments are omitted within a range where the description of the operation of the present embodiment is not ambiguous. FIG. 12 is a flowchart showing the operation of the processing unit 320 of the present embodiment.

  Prior to the operation shown in FIG. 12, the input unit 310 inputs the image information G20 and transmits it to the saliency map creation unit 121 and the operation target object detection unit 223.

  First, the saliency map creating unit 121 creates the saliency map M10 as in the first embodiment. Subsequently, the saliency map creation unit 121 transmits the created saliency map M10 to the gaze position prediction unit 325 (step S301).

  Next, the gaze position prediction unit 325 detects a gaze position coordinate C30 (not shown) that holds the maximum saliency value M12 in the received saliency map M10. Subsequently, the gaze position prediction unit 325 transmits the gaze position coordinates C30 and the saliency map M10 to the usability information generation unit 322 (step S302).

  Next, as in the second embodiment, the operation target object detection unit 223 detects the operation target object in the received image information G20, and acquires specification information P10 of each detected operation target object. Subsequently, the operation target object detection unit 223 transmits the specification information P10 to the usability information generation unit 322 (step S303).

  Next, the usability information generation unit 322 checks whether or not the gaze position coordinate C30 is within the region of the operation target object based on the received specification information P10 and the gaze position coordinate C30 (step S311).

  When the gaze position coordinate C30 is within the operation target object area (YES in step S311), the usability information generation unit 322 generates usability information indicating “no problem with the operation target object versus the gaze position relationship”. It transmits to the output part 330 (step S312).

  Further, when the gaze position coordinate C30 is outside the operation target object area (NO in step S311), the usability information generation unit 322 generates usability information indicating that there is a problem with the operation target object versus the gaze position relationship. The data is transmitted to the output unit 330 (step S313).

  Thus, the operation of the processing unit 320 ends. Subsequent to the series of operations, the output unit 330 displays the usability information, the saliency list M20, the gaze position coordinates C30, the gaze saliency value M32, and the saliency map M10 corresponding to the evaluation target image G10 on the evaluation target image G10. The drawn figure (this information is also called usability information) is displayed (step S314).

  By confirming the above-described display, the user of the usability evaluation apparatus 300 evaluates whether or not the user's attention is directed to the operation target object.

In addition to the effects of the first and second embodiments, the effect of the present embodiment described above can evaluate whether or not the user's visual attention is guided to the operation target object. It is a point.
This is because the usability information generation unit 322 determines whether or not the gaze position coordinate C30 is within the region of the operation target object based on the specification information P10.
[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 13 is a block diagram showing a configuration of the present embodiment. As illustrated in FIG. 13, the usability evaluation apparatus 400 according to the present embodiment includes an input unit 410, a processing unit 420, and an output unit 430.

  The input unit 410 has the function of the input unit 110 described in the first embodiment and the function of receiving and transmitting the input character string W40 representing the purpose of the user.

  The processing unit 420 includes a saliency map creation unit 121, an operation target object detection unit 423, a gaze position prediction unit 325, an operation target object saliency calculation unit 224, a semantic similarity calculation unit 426, a selected object prediction unit 427, and usability information generation. Part 422.

  The saliency map creation unit 121, the operation target object saliency calculation unit 224, and the gaze position prediction unit 325 are the same as those described in the first to third embodiments.

The operation target object detection unit 423 detects an operation target object in the received image information G20. Then, the operation target object detection unit 423 acquires at least the position coordinates M11, the width and the height of the detected operation target object, and the character string W41 written in the operation target object as the specification information P40, and transmits it. To do. FIG. 14 is a diagram illustrating an example of the specification information P40. As shown in FIG. 14, the specification information P40 includes an operation target object identifier M21, a position coordinate M11, a width, a height, and a character string W41.
The semantic similarity calculation unit 426 is based on the character string W40 representing the purpose of the user received from the input unit 410 and the character string W41 in the specification information P40 of the operation target object received from the operation target object detection unit 423. Then, the user's purpose and the semantic similarity between the operation target objects are calculated. Then, the semantic similarity calculation unit 426 creates a semantic similarity list S40 as shown in FIG.
FIG. 15 is a diagram illustrating an example of data in the semantic similarity list S40. As shown in FIG. 15, the semantic similarity list S40 includes a character string W40 representing the user's purpose, an operation target object identifier M21, and a character string written in the operation target object indicated by the operation target object identifier M21. One or more sets in which W41 is associated with the calculated semantic similarity value S42 are included.

As an example of the method for calculating the semantic similarity, there is a latent semantic analysis (LSA) described in Non-Patent Document 2. In LSA, the character string W40 and the character string W41 are respectively expressed as vectors in the semantic space, and the semantic similarity between the character string W40 and the character string W41 is obtained by calculating the cosine of the angle formed by these vectors. The value of the semantic similarity becomes higher as the similarity between words of the words included in each of the character string W40 and the character string W41 is higher.
Based on the semantic similarity list S40 created by the semantic similarity calculation unit 426, the selected object prediction unit 427 extracts and transmits an operation target object predicted to be selected by the user. Specifically, the selected object prediction unit 427 allows the user to select the operation target object identifier M21 corresponding to the largest semantic similarity value S42 among the semantic similarity values S42 included in the semantic similarity list S40. Extracted as the operation target object identifier M21 of the operation target object predicted to be selected. Subsequently, the selected object prediction unit 427 determines the operation target object identifier M21, position coordinates M11, height, and width of the operation target object indicated by the extracted operation target object identifier M21 as shown in FIG. Is extracted and transmitted as user operation target object information Q40 predicted to be operated. FIG. 16 is a diagram illustrating an example of user operation target object information Q40 predicted to be operated by the user.

  The usability information generation unit 422 generates usability information based on the saliency list M20 (FIG. 9) from the operation target object saliency calculation unit 224 and the user operation target object information Q40 predicted to be selected by the user. . Specifically, for example, the usability information generation unit 422 is predicted to be selected by the user with the operation target object identifier M21 corresponding to the maximum saliency value M22 in the saliency value M22 included in the saliency list M20. It is determined whether or not the operation target object identifier M21 of the user operation target object information Q40 is the same. If they are the same, the usability information generation unit 422 generates usability information indicating that there is no problem in the consistency of the operation target object. If they are not the same, the usability information generation unit 422 generates usability information indicating that there is a problem in the consistency of the operation target objects.

  Note that the usability information generation unit 422 may further generate usability information similar to the usability information generation unit 122 of the first embodiment and the usability information generation unit 222 of the second embodiment.

  The output unit 430 displays the processing result of the processing unit 420. The processing result is, for example, the above usability information, a figure in which the saliency value M12 obtained from the image information G20 and the user operation target object information Q40 are superimposed on the evaluation target image G10 (this information is also called usability information) ), A saliency list M20 (this information is also called usability information), and a semantic similarity list S40 (this information is also called usability information).

  Next, the operation of this embodiment will be described in detail with reference to FIG. In the following description of the operation of the present embodiment, the operations described in the first to third embodiments are omitted to the extent that the description of the operation of the present embodiment is not ambiguous.

  FIG. 17 is a flowchart showing the operation of the processing unit 420 of this embodiment.

  Prior to the operation shown in FIG. 17, the input unit 410 receives the input of the image information G20 and the character string W40 representing the user's purpose, and the saliency map creation unit 121, the operation target object detection unit 423, and the semantic similarity calculation unit. To 426.

  The operation in step S401 is equivalent to step S301 in the third embodiment.

  Next, the operation target object detection unit 423 detects the operation target object in the received image information G20. Then, the operation target object detection unit 423 acquires the specification information P40 of each detected operation target object, and transmits them to the operation target object saliency calculation unit 224 and the semantic similarity calculation unit 426 (step S403). .

  Next, the operation target object saliency calculating unit 224 generates the saliency list M20 as in the second embodiment. Then, the operation target object saliency calculating unit 224 transmits the generated saliency list M20 to the usability information generating unit 422 (step S404).

  Next, the semantic similarity calculation unit 426 calculates the character string W41 and the character string W40 written in the operation target object based on the received specification information P40 and the character string W40 representing the user's purpose. The semantic similarity is calculated to create a semantic similarity list S40, and is transmitted to the selected object prediction unit 427 (step S405).

  Next, the selected object prediction unit 427 extracts user operation target object information Q40 based on the received semantic similarity list S40. Then, the selected object prediction unit 427 transmits the extracted user operation target object information Q40 and the semantic similarity list S40 to the usability information generation unit 422 (step S406).

  Next, the usability information generation unit 422 receives the saliency map M10, the saliency list M20, the semantic similarity list S40, and the user operation target object information Q40. Subsequently, the usability information generating unit 422 operates corresponding to the maximum saliency value M22 among the operation target object identifier M21 included in the user operation target object information Q40 and the saliency value M22 included in the saliency list M20. It is determined whether or not the two operation target object identifiers M21 and the target object identifier M21 are the same (step S407).

If the two operation target object identifiers M21 are the same (YES in step S407), the usability information generation unit 422 generates usability information indicating that there is no problem in the user operation target object inductivity, and outputs this to the output unit 430. Transmit (step S408).
If the two operation target object identifiers M21 are not the same (NO in step S407), usability information indicating that there is a problem with the user operation target object inductivity is generated and transmitted to the output unit 430 (step S409).

  Thus, the operation of the processing unit 420 ends, and the output unit 430 displays the above usability information following the series of operations. Further, the output unit 430 may display a diagram depicting the saliency map M10 user operation target object information Q40 corresponding to the evaluation target image G10 on the evaluation target image G10. Further, the output unit 430 may display the semantic similarity list S40, the saliency list M20, and the like (step S410).

  By confirming the above display, the user of the usability evaluation apparatus 400 evaluates whether or not the user's attention is directed to the operation target object selected and operated by the user.

In addition to the effects of the first, second, and third embodiments, the effects of the present embodiment described above are such that the user's visual attention is guided to the operation target object that is predicted to be selected and operated by the user. It is possible to evaluate whether or not.
The reason is that the operation target object selected by the user is predicted by calculating the semantic similarity between the character string W41 written in the operation target object and the character string W40 representing the user's purpose, and the predicted operation This is because it is determined whether the target object is the operation target object corresponding to the maximum saliency value M22, and usability information is generated.
[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 18 is a block diagram showing a configuration of the present embodiment. As illustrated in FIG. 18, the usability evaluation apparatus 500 according to the present embodiment includes an input unit 510, a processing unit 520, and an output unit 530.

  The input unit 510 is equivalent to the input unit 110 described in the first embodiment.

  The processing unit 520 includes a saliency map creation unit 121, a non-content region detection unit 528, each region saliency calculation unit 529, and a usability information generation unit 522. The saliency map creation unit 121 is the same as that described in the first embodiment.

  The non-content area detection unit 528 detects a non-content area that does not include useful information in the image information G20 received from the input unit 510. Then, the non-content area detection unit 528 generates specification information P50 that divides the entire evaluation target image G10 into a content area and a non-content area. FIG. 19 is a diagram illustrating an example of the specification information P50. As shown in FIG. 19, the specification information P50 is information in which the position coordinates (x coordinate, y coordinate) M11 of each pixel in the image information G20 are associated with the region identifier P53. The specification information P50 shown in FIG. 19 shows a case where the evaluation target image G10 is “640 × 480” pixels as an example. For example, regarding the position coordinates M11 of the uppermost and leftmost pixels, the x coordinate is “1”, the y coordinate is “1”, and the region identifier P53 is “4”. Further, regarding the position coordinates M11 of the pixel at the lowermost and rightmost end, the x coordinate is “640”, the y coordinate is “480”, and the region identifier P53 is “5”.

  As an example of the non-content region detection method, there is a method described in Non-Patent Document 3. In this non-content area detection method, first, a web page is decomposed into text units, and a person determines whether each unit is a non-content area and inputs it to create teacher data for learning. In this non-content area detection method, supervised learning is performed using the created teacher data, and a non-content area is automatically detected by generating a chunking model.

  This is an example of the non-content area detection unit 528, and means for detecting the non-content area is not limited to this.

  The area saliency calculation unit 529 calculates the saliency values (content area saliency value M53 and non-content area saliency value M55 described later) of each area based on the saliency map M10 and the specification information P50. For example, the region saliency calculating unit 529 calculates the saliency value of each region by setting the sum of the saliency values M12 held by each pixel in the region as the saliency value of the region. Then, the region saliency calculation unit 529 creates a content region saliency list M50 as shown in FIG. 20 and a non-content region saliency list M51 as shown in FIG.

  FIG. 20 is a diagram illustrating an example of data in the content area salient list M50. As shown in FIG. 20, the content area saliency list M50 includes one or more sets in which the content area identifier M52 and the content area saliency value M53 are associated with each other. FIG. 21 is a diagram illustrating an example of data of the non-content area saliency list M51. As shown in FIG. 21, the non-content area saliency list M51 includes one or more sets in which the non-content area identifier M54 and the non-content area saliency value M55 are associated with each other.

  The usability information generation unit 522 generates usability information based on the content area saliency list M50 and the non-content area saliency list M51.

First, the usability information generation unit 522 compares each content area saliency value M53 with each non-content area saliency value M55, for example. If any of the non-content area saliency values M55 is greater than or equal to any of the content area saliency values M53, the usability information generation unit 522 generates usability information indicating that there is a problem in the different area saliency difference. Further, when all the non-content area saliency values M55 are less than any content area saliency value M53, the usability information generation unit 522 generates usability information indicating that there is no problem in the different area saliency difference.
Note that the usability information generation unit 522 may compare a predetermined numerical value with a non-content area instead of the content area saliency value M53. Similarly, the usability information generation unit 522 may compare a predetermined numerical value with the content region saliency value M53 instead of the non-content region saliency value M55.

  Furthermore, the usability information generation unit 522 compares the content area saliency values M53 with each other. As a result of the comparison, if any of the differences between the content area saliency values M53 is equal to or greater than a predetermined difference threshold d, the usability information generation unit 522 generates usability information indicating that there is a problem with the content area saliency difference. When all of the differences between the content area saliency values M53 are less than a predetermined difference threshold d, the usability information generation unit 522 generates usability information indicating that there is no problem in the content area saliency difference.

  Note that the usability information generation unit 522 may further generate usability information similar to that of the usability information generation unit 122.

  Further, the present embodiment may include any of the second embodiment, the third embodiment, and the fourth embodiment.

  The output unit 530 displays the processing result of the processing unit 520. The processing results include the above usability information, a diagram depicting the content area and non-content area of the evaluation target image G10 on the evaluation target image G10 (this information is also called usability information), and a content area salient list M50 ( This information is also called usability information), a non-content area remarkable list M51 (this information is also called usability information), and the like.

  Next, the operation of the present embodiment will be described in detail with reference to FIGS. In the following description of the operation of the present embodiment, the operation described in the first embodiment is omitted within a range that does not obscure the description of the operation of the present embodiment.

  FIG. 22 is a flowchart showing the operation of the processing unit 520 of the present embodiment.

  Prior to the operation illustrated in FIG. 22, the input unit 510 receives input of the image information G20 and transmits it to the saliency map creation unit 121 and the non-content area detection unit 528.

  First, the saliency map creating unit 121 creates the saliency map M10 as in the first embodiment. Subsequently, the saliency map creation unit 121 transmits the created saliency map M10 to the usability information generation unit 522 and the region saliency calculation unit 529 (step S501).

  Next, the non-content area detection unit 528 detects the non-content area in the received image information G20 and transmits the specification information P50 of the non-content area and the other content areas to the operation target object saliency calculation unit 224. (Step S502).

  Next, the region saliency calculation unit 529 calculates the content region saliency value M53 and the non-content region saliency value M55 based on the saliency map M10 and the specification information P50, and thereby calculates the content region saliency list M50 and the non-contents. The region saliency list M51 is created and transmitted to the usability information generation unit 522 (step S503).

  Next, the usability information generation unit 522 receives the saliency map M10, the content area saliency list M50, and the non-content area saliency list M51. Subsequently, the usability information generation unit 522 compares the content area saliency value M53 and the non-content area saliency value M55 (step S504).

  If any of the non-content area saliency values M55 is greater than or equal to any of the content area saliency values M53 (YES in step S504), the usability information generation unit 522 indicates usability indicating that there is a problem in the different area saliency difference. Information is generated and transmitted to the output unit 530 (step S505).

  When all the non-content area saliency values M55 are less than any of the content area saliency values M53 (NO in step S504), the usability information generation unit 522 displays usability information indicating that there is no problem in the different area saliency difference. It is generated and transmitted to the output unit 530 (step S506).

  Next, the usability information generation unit 522 compares the content area saliency values M53 with each other (step S507).

  When the difference between the content area saliency values M53 is greater than or equal to a predetermined difference threshold d (YES in step S506), the usability information generation unit 522 generates usability information indicating that there is a problem with the content area saliency difference. This is transmitted to the output unit 530 (step S508).

  When the difference between the content area saliency values M53 is less than the predetermined difference threshold d (NO in step S506), the usability information generation unit 522 generates usability information indicating that there is no problem in the content area saliency difference. This is transmitted to the output unit 530 (step S509).

  Next, the usability information generation unit 522 generates a diagram depicting the saliency map M10, the content area, and the non-content area corresponding to the evaluation target image G10. Subsequently, the usability information generation unit 522 transmits the content area saliency list M50, the non-content area saliency list M51, and the generated diagram to the output unit 530 (step S510).

  Thus, the operation of the processing unit 520 is completed, and the output unit 530 displays the received usability information following the series of operations. Further, the output unit 530 displays the received content area saliency list M50, non-content area saliency list M51, and a diagram in which the content area and the non-content area of the evaluation target image G10 are drawn on the evaluation target image G10. .

  By confirming the above-described display, the user of the usability evaluation apparatus 500 evaluates whether or not the user's attention is not guided to the non-content area.

  The effect in the present embodiment described above is that it is possible to evaluate whether or not the user's attention is guided to the non-content area in addition to the effect of the first embodiment.

This is because the non-content area in the entire screen is detected, and the usability information is generated by comparing the saliency between the content area and the non-content area.
[Sixth Embodiment]
Next, a sixth embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 23 is a block diagram showing a configuration of the present embodiment. As shown in FIG. 23, the usability evaluation device 600 according to this embodiment includes a usability information generation unit 122.

  The usability information generation unit 122, for example, based on the saliency value M12 for each unit element (for example, pixel) included in the saliency map M10 as shown in FIG. 3 of the evaluation target image G10 as shown in FIG. Usability information of the evaluation target image G10 is generated. For example, the saliency map M10 shown in FIG. 3 may be developed on a memory (not shown) by means not shown.

  The effect in the present embodiment described above is that it is possible to evaluate the usability with respect to the visual characteristics of the image.

This is because usability information for evaluating whether or not attention should be directed to the evaluation target image is generated based on the saliency value M12 of the evaluation target image G10.
[Modification of Embodiment]
Each of the above embodiments may be modified by combining some or all of them. For example, the operation target object saliency calculation unit 224 is added to the processing unit 320 of the third embodiment, and the usability information generation unit 322 generates the same usability information as the usability information generation unit 222 of the second embodiment. May be. For example, a gaze position prediction unit 325 is added to the processing unit 420 of the fourth embodiment, and the usability information generation unit 422 generates usability information similar to the usability information generation unit 322 of the third embodiment. May be.

  Each component described in each of the above embodiments may be configured by, for example, a computer including a CPU (Central Processing Unit) and a storage medium. In this case, the storage medium may store a program for causing each component to function by a computer.

  Each component described in each of the above embodiments does not necessarily have to be individually independent. For example, for each component, a plurality of components may be realized as one module, or one component may be realized as a plurality of modules. Each component is configured such that a component is a part of another component, or a part of a component overlaps a part of another component. Also good.

  Further, in each of the embodiments described above, a plurality of operations are described in order in the form of a flowchart, but the described order does not limit the order in which the plurality of operations are executed. For this reason, when each embodiment is implemented, the order of the plurality of operations can be changed within a range that does not hinder the contents.

  Furthermore, in each embodiment described above, a plurality of operations are not limited to being executed at different timings. For example, another operation may occur during the execution of a certain operation, or the execution timing of a certain operation and another operation may partially or entirely overlap.

  Furthermore, in each of the embodiments described above, a certain operation is described as a trigger for another operation, but the description does not limit all relationships between the certain operation and the other operations. For this reason, when each embodiment is implemented, the relationship between the plurality of operations can be changed within a range that does not hinder the contents. The specific description of each operation of each component does not limit each operation of each component. For this reason, each specific operation | movement of each component may be changed in the range which does not cause trouble with respect to a functional, performance, and other characteristic in implementing each embodiment.

  Each component in each embodiment described above may be realized by hardware, software, or a mixture of hardware and software, if necessary. May be.

  Further, the physical configuration of each component is not limited to the description of the above embodiment, and may exist independently, may exist in combination, or may be configured separately. May be.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
Usability information generation for generating and outputting usability information for evaluating whether or not attention can be drawn to a specific portion in the image based on an element saliency value indicating saliency for each unit element of the image Usability evaluation apparatus having a unit.

(Appendix 2)
An operation target object detection unit that detects an operation target object in the image and outputs specification information including information indicating a region of the detected operation target object;
An operation target object saliency calculation unit that calculates and outputs an operation target object saliency value corresponding to the saliency value of the operation target object based on the specification information and the element saliency value;
The usability information generation unit generates and outputs usability information based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other. The usability evaluation apparatus according to attachment 1.

(Appendix 3)
An operation target object detection unit that detects an operation target object in the image and outputs specification information including information indicating a region of the detected operation target object;
A gaze position prediction unit that predicts the gaze position of the user and detects gaze position coordinates;
The usability evaluation device according to attachment 1 or 2, wherein the usability information generation unit generates and outputs usability information based on the gaze position coordinates and the region of the operation target object.

(Appendix 4)
The specification information further includes character string information included in the operation target object,
The operation target object saliency calculating unit associates and outputs the operation target object saliency value and the identifier of the operation target object corresponding to the operation target object saliency value,
A semantic similarity calculator that calculates the semantic similarity between the character string information and the character string information representing the purpose of the user;
A selection object determination unit that predicts a selection target object to be selected by the user from among the operation target objects based on the semantic similarity and outputs selection object information including an identifier of the selection target object; The usability evaluation device according to claim 2, wherein the usability information generation unit generates and outputs usability information based on the operation target object saliency value, the identifier of the operation target object corresponding thereto, and the selected object information. .

(Appendix 5)
An area detection unit for detecting a content area and a non-content area in the image;
A region saliency calculating unit that calculates and outputs a region saliency value for each of the content regions and each of the non-content regions;
The usability evaluation device according to any one of appendices 1 to 4, wherein the usability information generation unit generates and outputs usability information based on the saliency values of the content areas and the non-content areas.

(Appendix 6)
The usability information generation unit generates usability information based on a result of comparing the number of unit elements having the element saliency value that satisfies a predetermined saliency threshold with a predetermined number of unit elements. The usability evaluation apparatus according to appendices 1 to 5.

(Appendix 7)
A processing unit for evaluating whether or not attention can be drawn to a specific part in the image based on an element saliency value for each unit element indicating saliency of the image developed on the memory is usability information. How to generate.

(Appendix 8)
Detecting an operation target object in the image, outputting specification information including information indicating a region of the detected operation target object;
Based on the specification information and the element saliency value, calculate and output an operation target object saliency value corresponding to the saliency value of the operation target object,
The method according to claim 7, wherein usability information is generated based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other.

(Appendix 9)
Detecting an operation target object in the image, outputting specification information including information indicating a region of the detected operation target object;
Predict the user's gaze position, detect gaze position coordinates,
The method according to claim 7 or 8, wherein usability information is generated and output based on the gaze position coordinates and the region of the operation target object.

(Appendix 10)
The specification information further includes character string information included in the operation target object,
The operation target object saliency value and the corresponding identifier of the operation target object are output in association with each other,
Calculating the semantic similarity between the character string information and the character string information representing the purpose of the user;
Based on the semantic similarity, predict a selection target object selected by the user from among the operation target objects, and output selection object information including an identifier of the selection target object,
The method according to claim 8, wherein usability information is generated and output based on the operation target object saliency value, the identifier of the operation target object corresponding to the operation target object saliency value, and the selected object information.

(Appendix 11)
Detecting a content area and a non-content area in the image;
Calculate and output a region saliency value for each content region and each non-content region;
The method according to any one of appendices 7 to 10, wherein usability information is generated and output based on the saliency value of each content area and each non-content area.

(Appendix 12)
Based on the result of comparing the number of unit elements having the element saliency value satisfying a predetermined saliency threshold with the predetermined number of unit elements, usability information is generated and output. The method described.

(Appendix 13)
A processing unit for evaluating whether or not attention can be drawn to a specific portion in the image based on an element saliency value indicating saliency for each unit element of the image developed on the memory is usability information. A program that causes a computer to execute the process of generating

(Appendix 14)
Detecting an operation target object in the image, outputting specification information including information indicating a region of the detected operation target object;
Based on the specification information and the element saliency value, calculate and output an operation target object saliency value corresponding to the saliency value of the operation target object,
The supplementary note 13 causes the computer to execute a process of generating usability information based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other. Program.

(Appendix 15)
Detecting an operation target object in the image, outputting specification information including information indicating a region of the detected operation target object;
Predict the user's gaze position, detect gaze position coordinates,
The program according to appendix 13 or 14, which is executed by a computer that generates and outputs usability information based on the gaze position coordinate and the region of the operation target object.

(Appendix 16)
The specification information further includes character string information included in the operation target object,
The operation target object saliency value and the corresponding identifier of the operation target object are output in association with each other,
Calculating the semantic similarity between the character string information and the character string information representing the purpose of the user;
Based on the semantic similarity, a selection target object to be selected by the user from among the operation target objects is predicted, and selection object information including an identifier of the selection target object is output, and the operation target object saliency value and this 15. The program according to appendix 14, which is executed by a computer that generates and outputs usability information based on the identifier of the operation target object corresponding to the selected object information and the selected object information.

(Appendix 17)
Detecting a content area and a non-content area in the image;
Calculate and output a region saliency value for each content region and each non-content region;
The program according to any one of appendices 13 to 16, which is executed by a computer that generates and outputs usability information based on the saliency values of the content areas and the non-content areas.

(Appendix 18)
Based on the result of comparing the number of unit elements having the element saliency value that satisfies a predetermined saliency threshold with the predetermined number of unit elements, the usability information is generated and output to a computer. The program according to appendixes 13 to 17.

(Appendix 19)
Usability information generation for generating and outputting usability information for evaluating whether or not attention can be drawn to a specific portion in the image based on an element saliency value indicating saliency for each unit element of the image Usability evaluation apparatus having a unit.

(Appendix 20)
An operation target object detection unit that detects an operation target object in the image and outputs specification information including information indicating a region of the detected operation target object;
An operation target object saliency calculation unit that calculates and outputs an operation target object saliency value corresponding to the saliency value of the operation target object based on the specification information and the element saliency value;
The usability information generation unit generates and outputs usability information based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other. The usability evaluation apparatus according to appendix 19.

(Appendix 21)
An operation target object detection unit that detects an operation target object in the image and outputs specification information including information indicating a region of the detected operation target object;
A gaze position prediction unit that predicts the gaze position of the user and detects gaze position coordinates;
The usability evaluation apparatus according to appendix 19 or 20, wherein the usability information generation unit generates and outputs usability information based on the gaze position coordinates and the region of the operation target object.

(Appendix 22)
The specification information further includes character string information included in the operation target object,
The operation target object saliency calculating unit associates and outputs the operation target object saliency value and an identifier of the operation target object corresponding to the operation target object saliency value,
A semantic similarity calculator that calculates the semantic similarity between the character string information and the character string information representing the purpose of the user;
A selection object determination unit that predicts a selection target object to be selected by the user from among the operation target objects based on the semantic similarity and outputs selection object information including an identifier of the selection target object; The usability evaluation apparatus according to claim 20, wherein the usability information generation unit generates and outputs usability information based on the operation object saliency value, an identifier of the operation object corresponding to the operation object object, and the selected object information. .

(Appendix 23)
An area detection unit for detecting a content area and a non-content area in the image;
A region saliency calculating unit that calculates and outputs a region saliency value for each of the content regions and each of the non-content regions;
The usability evaluation apparatus according to any one of appendices 19 to 22, wherein the usability information generation unit generates and outputs usability information based on the saliency values of the content areas and the non-content areas.

(Appendix 24)
The usability information generation unit generates usability information based on a result of comparing the number of unit elements having the element saliency value that satisfies a predetermined saliency threshold with a predetermined number of unit elements. The usability evaluation device according to appendices 19 to 23.

DESCRIPTION OF SYMBOLS 100 Usability evaluation apparatus 110 Input part 120 Processing part 121 Remarkable map creation part 122 Usability information generation part 130 Output part 200 Usability evaluation apparatus 210 Input part 220 Processing part 222 Usability information generation part 223 Operation target object detection part 224 Operation target object saliency Calculation unit 230 Output unit 300 Usability evaluation device 310 Input unit 320 Processing unit 322 Usability information generation unit 325 Gaze position prediction unit 330 Output unit 400 Usability evaluation device 410 Input unit 420 Processing unit 422 Usability information generation unit 423 Operation target object detection unit 426 Semantic similarity calculation unit 427 selected object prediction unit 430 output unit 500 usability evaluation device 510 input unit 520 processing unit 5 2 Usability information generation unit 528 Non-content region detection unit 529 Region saliency calculation unit 530 Output unit 600 Usability evaluation apparatus G10 Evaluation target image M10 Saliency map M11 Position coordinates M12 Saliency value M20 Saliency list M21 Operation target object identifier M22 Saliency value M50 Content area saliency list M51 Non-content area saliency list M52 Content area identifier M53 Content area saliency value M54 Non-content area identifier M55 Non-content area saliency value P10 Specification information P40 Specification information P50 Specification information P53 Area identifier Q40 User Operation target object information S40 Semantic similarity list S42 Semantic similarity value W40 Character string W41 Character string

Claims (10)

Usability information generation for generating and outputting usability information for evaluating whether or not attention can be drawn to a specific portion in the image based on an element saliency value indicating saliency for each unit element of the image Usability evaluation apparatus having a unit. An operation target object detection unit that detects an operation target object in the image and outputs specification information including information indicating a region of the detected operation target object;
An operation target object saliency calculation unit that calculates and outputs an operation target object saliency value corresponding to the saliency value of the operation target object based on the specification information and the element saliency value;
The usability information generation unit generates and outputs usability information based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other. The usability evaluation apparatus according to claim 1.
An operation target object detection unit that detects an operation target object in the image and outputs specification information including information indicating a region of the detected operation target object;
A gaze position prediction unit that predicts the gaze position of the user and detects gaze position coordinates;
The usability evaluation device according to claim 1, wherein the usability information generation unit generates and outputs usability information based on the gaze position coordinates and the region of the operation target object. The specification information further includes character string information included in the operation target object,
The operation target object saliency calculating unit associates and outputs the operation target object saliency value and the identifier of the operation target object corresponding to the operation target object saliency value,
A semantic similarity calculator that calculates the semantic similarity between the character string information and the character string information representing the purpose of the user;
A selection object determination unit that predicts a selection target object to be selected by the user from among the operation target objects based on the semantic similarity and outputs selection object information including an identifier of the selection target object; 3. The usability evaluation according to claim 2, wherein the usability information generation unit generates and outputs usability information based on the operation object saliency value, the identifier of the operation object corresponding to the operation object object, and the selected object information. apparatus. An area detection unit for detecting a content area and a non-content area in the image;
A region saliency calculating unit that calculates and outputs a region saliency value for each of the content regions and each of the non-content regions;
The usability evaluation apparatus according to claim 1, wherein the usability information generation unit generates and outputs usability information based on a saliency value of each content area and each non-content area. The usability information generation unit generates usability information based on a result of comparing the number of unit elements having the element saliency value that satisfies a predetermined saliency threshold with a predetermined number of unit elements. The usability evaluation apparatus according to any one of claims 1 to 5. A method for generating usability information by a processing unit for evaluating whether or not attention can be drawn to a specific portion in the image based on an element saliency value indicating saliency for each unit element. Detecting an operation target object in the image, outputting specification information including information indicating a region of the detected operation target object;
Based on the specification information and the element saliency value, calculate and output an operation target object saliency value corresponding to the saliency value of the operation target object,
The method according to claim 7, wherein usability information is generated based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other. Based on an element saliency value indicating saliency for each unit element, a processing unit for evaluating whether or not attention can be drawn to a specific part in the image executes processing for generating usability information on a computer Program to make. Detecting an operation target object in the image, outputting specification information including information indicating a region of the detected operation target object;
Based on the specification information and the element saliency value, calculate and output an operation target object saliency value corresponding to the saliency value of the operation target object,
The computer executes a process of generating usability information based on a result of comparing the element saliency value and the operation target object saliency value and a result of comparing the operation target object saliency values with each other. The listed program.

Images