JP2015125543A - Line-of-sight prediction system, line-of-sight prediction method, and line-of-sight prediction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To clearly present a difference between a plurality of designs.SOLUTION: A line-of-sight prediction system includes one or a plurality of processors. At least one processor (a) receives a reference image having a common or similar design and a comparison image, (b) applies line-of-sight prediction for predicting an area which is likely to be visually attractive, to both reference image and the comparison image, to calculate a line-of-sight distribution of the reference image and a line-of-sight distribution of the comparison image, (c) calculates the amount of change between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparison image, (d) and outputs the amount of change as a change in line of sight distribution.

Description

  One aspect of the present invention relates to a line-of-sight prediction system, a line-of-sight prediction method, and a line-of-sight prediction program.

  Conventionally, a technique for evaluating the visual effect of a design is known. For example, Patent Document 1 below receives an input scene, applies a visual attention model to the input scene, and predicts an area in the input scene that tends to attract visual attention. And a visual attention module that functions to interact with the visual attention module to determine the degree to which at least one of the identified regions is robust (prominent) or the scene is robust A computer system is described that includes a robustness (saliency) evaluation module. The following Patent Documents 2 to 4 also describe a mechanism for evaluating the design.

Special table 2012-504827 gazette Japanese Patent No. 4661398 Japanese Patent No. 4997443 Japanese Patent No. 4208614

  It is often the case that a design is finally determined by repeatedly comparing a design plan with another design plan. If the technique of the above-mentioned patent documents 1 to 4 is adopted in the work, the user decides the design while comparing the line-of-sight prediction results of the individual designs. When trying to compare designs having common parts, it may be difficult to compare designs by visual observation alone. Therefore, it is desirable to clearly show the differences between multiple designs.

  The line-of-sight prediction system according to one aspect of the present invention includes one or more processors, and at least one processor tends to accept a reference image and a comparative image having a common or similar design and attract visual attention. By applying gaze prediction for predicting a certain area to both the reference image and the comparison image, the gaze distribution of the reference image and the gaze distribution of the comparison image are calculated, and the gaze distribution of the reference image and the gaze distribution of the comparison image Is calculated as a change in the gaze distribution.

  In such an aspect, since the amount of change in the line-of-sight distribution of two images having a common or similar design is calculated and the result is output, differences between a plurality of designs can be clearly presented.

  According to one aspect of the present invention, a plurality of design differences can be clearly presented.

It is a figure showing the whole line-of-sight prediction system composition concerning an embodiment. It is a figure which shows the hardware constitutions of the analysis server which concerns on embodiment. It is a block diagram which shows the function structure of the gaze prediction system which concerns on embodiment. It is a figure which shows the example of a reference | standard image and a comparison image. It is a figure which shows the concept of a unit area | region and a visibility score. It is a figure which shows the example which visualized the visibility score. It is a figure which shows the concept of the variation | change_quantity of a visibility score. It is a figure which shows the example of a response | compatibility with the variation | change_quantity of a visibility score, and a color contour. It is a figure which shows an example of a difference image. It is a figure which shows another example of a difference image. It is a figure which shows another example of a difference image. It is a flowchart which shows operation | movement of the gaze prediction system which concerns on embodiment. It is a figure which shows the example of a reference | standard image and two comparison images. It is a figure which shows the example of the difference image corresponding to the comparison image shown in FIG. It is a figure which shows the structure of the gaze prediction program which concerns on embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

  The function and configuration of the line-of-sight prediction system 1 according to the embodiment will be described with reference to FIGS. The line-of-sight prediction system 1 is a computer system that predicts an area in an image (scene) that tends to attract visual attention. In particular, the line-of-sight prediction system 1 obtains the amount of change in the prediction result obtained for two images having a common or similar design and presents the amount of change to the user. Users can see the degree of change and determine how the visual effect changes by processing the image, and can determine which image exhibits the desired visual effect. .

  Any design or subject may be shown in the images herein. For example, the design shown in the image may be a label, business card, postcard, poster, wrapping paper, advertisement, in-store POP, or sign. Alternatively, the subject shown in the image may be an urban area or a store where advertisements or billboards exist, or a product or a product package itself. Alternatively, the image may be a captured image of a web page. These images may be photographs or computer graphics.

As used herein, “common or similar design” means that at least some of the attributes of the design are the same between the two images. For example, “common or similar design” includes the following aspects.
・ Completely the same design (design, shape, pattern, dimensions, aspect ratio, color, transparency, etc.).
A design in which at least some of the attributes such as shape, pattern, dimensions, aspect ratio, color, and transparency are different from each other, but the others are the same.

  For example, it can be said that an image showing only the background portion of a business card and an image in which business card data (person name, company name, telephone number, e-mail address, etc.) is embedded in the background have a common or similar design. In addition, it can be said that images of a plurality of signboards having the same outer shape, pattern, and characters but having different background colors or character colors have a common or similar design. Of course, examples of images having a common or similar design are not limited to these.

  As shown in FIG. 1, the line-of-sight prediction system 1 includes a user terminal T and an analysis server 10. The user terminal T and the analysis server 10 can communicate with each other via the network N. The specific configuration of the network N is not limited, and may be configured from, for example, at least one of the Internet, a dedicated line, and a LAN (Local Area Network). The number of user terminals T in the line-of-sight prediction system 1 is not limited.

  The line-of-sight prediction system 1 in the present embodiment is a so-called client-server system, but the specific construction method is not limited at all. For example, the line-of-sight prediction system 1 may function on the assumption that dedicated client software is installed in the user terminal T, and may be a cloud type or browser-based that does not require such software. May be. If dedicated client software is required to use the line-of-sight prediction system 1, the user installs the software in the user terminal T. The software may be provided to the user via a network or may be provided via a storage medium such as a DVD-ROM. If dedicated client software is not required, the user can use the line-of-sight prediction system 1 if the user terminal T includes general-purpose software (for example, a web browser) that is necessary.

  The type of the user terminal T is not limited, and may be, for example, a stationary or portable personal computer (PC), or a portable terminal such as a high-function mobile phone (smart phone), a mobile phone, or a personal digital assistant (PDA).

  The analysis server 10 is a computer that executes line-of-sight prediction for each of two images having a common or similar design and obtains the degree of change between images from the prediction result. Gaze prediction is a process of simulating the movement of a person's gaze, and is also called a visual attention model. By this processing, it is possible to estimate where in the image (scene) attracts human visual attention.

  The analysis server 10 has a hardware configuration as shown in FIG. That is, the analysis server 10 includes a CPU 101 that executes an operating system, application programs, and the like, a main storage unit 102 that includes a ROM and a RAM, an auxiliary storage unit 103 that includes a hard disk and a flash memory, and a network card. Alternatively, the communication control unit 104 includes a wireless communication module, an input device 105 such as a keyboard and a mouse, and an output device 106 such as a display.

  Each functional component of the analysis server 10 described later reads predetermined software on the CPU 101 or the main storage unit 102, and operates the communication control unit 104, the input device 105, the output device 106, and the like under the control of the CPU 101. This is realized by reading and writing data in the main storage unit 102 or the auxiliary storage unit 103. Data and a database necessary for processing are stored in the main storage unit 102 or the auxiliary storage unit 103.

  The analysis server 10 may be configured with one computer or may be configured with a plurality of computers.

  As illustrated in FIG. 3, the analysis server 10 includes a reception unit 11, an analysis unit 12, a change amount calculation unit 13, an image generation unit 14, and a transmission unit 15 as functional components. These functions are realized by the operation of at least one processor (CPU 101).

  The accepting unit 11 is a functional element that accepts two images designated by the user. In the present specification, the two images are distinguished as a reference image and a comparative image. In the present embodiment, the line-of-sight prediction system 1 indicates to the user how the line-of-sight prediction result of the comparative image changes compared to that of the reference image.

  The reception unit 11 receives a reference image and a comparison image having a common or similar design, and outputs these two images to the analysis unit 12. The receiving unit 11 may directly receive these images from the user terminal T, or may read those images from a predetermined storage unit (for example, a memory or a database) based on an instruction from the user terminal T. . Examples of the reference image and the comparison image are shown in FIG. In this example, the reference image 201 is a flower picture, and the comparison image 202 is an image in which the character string “ABCDEFG” is added to the reference image 201. That is, the reference image 201 and the comparison image 202 have the same flower portion, and thus these two images have a common design.

  The analysis unit 12 is a functional element that calculates the line-of-sight distribution of each image by applying a process (line-of-sight prediction) that predicts a region that tends to attract visual attention to both the reference image and the comparative image. The line-of-sight prediction is described in, for example, Patent Document 1 described above, and the process will be described below.

  There have been many visual attention models. In general, a visual attention model accepts an image as a scene and generates predictions about where attention is assigned in the scene. Many mathematical models have been developed that attempt to predict attention through simulation. One technique is that of Itti, L. et al. & Koch, C.I. (2000) "A saliency-based search mechanism for over and cover shifts of visual attention" (Vision Research, vol. 40, pages 1489 to 1506). This approach predicts visual attention by evaluating bottom-up features (eg, color, motion, brightness, contrast, edges, etc.). Specifically, the computer analyzes the input image for color, intensity, orientation, or other scene connections (eg, shapes obtained from motion, relay, terminator, stereo parallax, or shading). Subsequently, the computer generates a plurality of feature maps, and generates a saliency map by combining these feature maps. In the case of the Koch model, the saliency map is provided to the user as a rendering of the original image with the brightest object that is predicted to be assigned the next visual attention. The predicted object is identified as visually prominent in the winner winning algorithm. These series of processes are repeated until a plurality of objects are identified by the model.

  The Itti & Koch model is representative of a bottom-up visual attention model, which makes its prediction based on an analysis of a particular thing in the scene. Other examples of bottom-up visual attention models are described in Gao, Mahadevan, and Vesconcelos (2008).

  Apart from the bottom-up model, there is another model called the visual attention top-down model. In contrast to the bottom-up model, this model affects the scene and the obvious tasks (eg avoiding obstacles and collecting objects) or where attention is assigned during a particular search task. It starts with any prior knowledge of the world (eg, the chair is on the floor, not on the ceiling). This knowledge based on tasks and scenes is used in conjunction with bottom-up features to draw attention to objects in the observed scene. Some representative top-down models are Rothkopf, C .; A. Ballard, D .; H. & Hayhoe, M .; M.M. "Task and context Determine Where You Look" (2007, Journal of Vision 7 (14): 16, 1-20), and Torralba, A. et al. "Contextual Modulation of Target Saliency" (Adv. In Neural Information Processing Systems 14 (NIPS) (2001) (MIT Press, 2001), for example, the type of visual attention Torralba. It has prior knowledge about features including objects and information about the absolute and relative positions of these objects in the scene, which has a top-down influence on the search for specific targets in the scene.

  There are also hybrid visual attention models that have both bottom-up and top-down features.

  The analysis unit 12 obtains the gaze distribution of each image by processing the reference image and the comparative image using such a gaze prediction (visual attention model) mechanism. The line-of-sight distribution in one image is a visibility score distribution indicating how much line-of-sight gathers in which part of the image. This line-of-sight distribution is obtained by dividing an image into a plurality of unit areas and obtaining a visibility score of each unit area. Therefore, the line-of-sight distribution is a set of visibility scores.

  First, the analysis unit 12 divides each of the reference image and the comparison image into m × n unit areas. In addition, m = n may be sufficient. The size of the unit region is not limited. For example, the unit region may be composed of only one pixel, or may be composed of n × n (n> 1) pixels. When the number of pixels, the size, or the resolution is different between the reference image and the comparison image, the analysis unit 12 sets both the images to m by changing the size of the unit area between the two images. What is necessary is just to divide into * n unit area. Subsequently, the analysis unit 12 performs line-of-sight prediction on each image, thereby obtaining a visibility score V (i, j) of each unit region (i, j) in the image (where 1 ≦ i ≦ m). , 1 ≦ j ≦ n). Then, the analysis unit 12 outputs the line-of-sight distributions of both the reference image and the comparison image to the change amount calculation unit 13.

When the reference image 201 and the comparison image 202 shown in FIG. 4 are input, the analysis unit 12 divides each of the two images into m × n as shown in FIG. Then, the analysis unit 12 calculates a visibility score V _b for each unit region of the reference image 201 and calculates a visibility score V _m for each unit region of the comparative image 202. FIG. 6 schematically shows an example in which the line-of-sight distribution of the reference image 201 and the comparison image 202 obtained by the calculation is visualized with a color contour. In FIG. 6, the color contour 201a of the reference image 201 and the color contour 202a of the comparative image 202 both indicate that the visibility score is higher as the shading is thinner. Since the main purpose of the analysis server 10 is to present the difference in visual effect between the reference image and the comparison image to the user, it is essential for the analysis server 10 to output the line-of-sight distribution of both the reference image and the comparison image. is not.

  The change amount calculation unit 13 is a functional element that calculates the change amount of the line-of-sight distribution between the reference image and the comparison image. The change amount calculation unit 13 performs a process for obtaining a change amount δV (i, j) of the visibility score between the unit region (i, j) of the reference image and the corresponding unit region (i, j) of the comparison image. Execute for all unit areas. The “change amount” in this specification means an index indicating how much the line-of-sight distribution differs between the reference image and the comparative image, and the value is not limited to a difference obtained by simple subtraction. Below, three examples about the calculation method of variation | change_quantity (delta) V are shown.

[First method]
The change amount calculation unit 13 may obtain the change amount δV by the following equation. That is, the change amount calculation unit 13 may obtain a value obtained by subtracting the visibility score of the reference image from the visibility score of the comparison image as the change amount.
δV (i, j) = V _m (i, j) −V _b (i, j)
However, if the δV (i, j) is less than 0, the change amount calculation unit 13 corrects the value to 0. This correction is done to show only how much the comparison image has changed compared to the reference image.

[Second method]
The change amount calculation unit 13 may obtain the change amount δV by the following equation.
_{δV (i, j) = [} V m (i, j) × {V max -V b (i, j)}] / P max × V max
Here, the value V _max is a maximum value of a predetermined visibility score. For example, if the range of the visibility score is 0 to 100, V _max is 100. Of course, the maximum value of the actual visibility score may be lower than V _max in at least one of the reference image and the comparative image. The value P _max is the _maximum value [V _m (i, j) × {V _max −V _b (i, j)}] obtained for all unit regions.

This second method is a method for obtaining the change amount δV using the product of the visibility score of the comparative image and the visibility score of the reference image. Also, this technique is also a method to normalize the amount of change in each unit region between 0 and V _max.

[Third method]
The change amount calculation unit 13 may obtain the change amount δV by the following equation.
_{δV (i, j) = [} {V m (i, j) -V b (i, j)} - P min] / (P max -P min) × V max
Here, the values P _max and P _min are the maximum value and the minimum value of the values [V _m (i, j) × {V _max −V _b (i, j)}] obtained for all the unit areas, respectively. is there.

Similar to the first method, the third method is a method of obtaining the change amount δV using a value obtained by subtracting the visibility score of the reference image from the visibility score of the comparative image. This method is also a method for normalizing the amount of change in each unit region between 0 and V _max , as in the second method.

  As described above, there are various methods for calculating the change amount δV, but in any case, the change amount calculation unit 13 calculates the change amount δV (i, j) for all the unit regions (i, j) and changes the change. A set of the amount δV is output to the image generation unit 14. A set of the variation δV is schematically shown in FIG. The set of change amounts δV is data indicating changes in the line-of-sight distribution.

  The image generation unit 14 is a functional element that generates an image (difference image) for showing the distribution of the change amount δV of the visibility score in an easily understandable manner. Although the representation method of a difference image is not limited at all, in the present embodiment, the difference image is represented by a heat map. The heat map is a technique for expressing the magnitude of a numerical value by color shading or color change, and is obtained by superimposing the color shading or color change on an original image. For example, a heat map in which the range from black indicating the minimum value to white indicating the maximum value is expressed in 256 levels of gray scale, or the range from black indicating the minimum value to red indicating the maximum value is expressed as black → blue → light blue. It is possible to use a heat map expressed by a color contour that changes smoothly in the order of green, yellow, red.

First, the image generation unit 14 acquires a minimum value V _min and a maximum value V _max of a visibility score determined in advance, and associates a range from V _min to V _{max with} a scale of the heat map. For example, if V _min = 0 and V _max = 100, the image generation unit 14 associates the range of 0 to 100 with the scale of the heat map. Since the change amount δV takes a value between V _min and V _max , the change amount δV can be expressed by this scale. If the change amount δV is expressed in gray scale, the image generation unit 14 associates V _min and V _max with black and white, respectively. In this case, a unit region having a small change amount δV is shown in dark or dark gray, and a unit region having a large change amount δV is shown in light or light gray. If the amount of change δV is expressed by a color contour, the image generation unit 14 sets V _min and V _max to black (RGB values are (0, 0, 0)) and red (RGB values are (127, 0, 0)). ). In this case, a unit region having a small change amount δV is indicated by blue or light blue, a unit region having a large change amount δV is indicated by yellow or red, and a unit region having a change amount δV close to an intermediate value is green. Indicated.

An example of the correspondence between the change amount δV and the color contour is shown in FIG. In this example, the minimum value V _min and the maximum value V _{max of} the visibility score determined in advance are 0 and 100, respectively, and therefore the change amount δV takes a value between 0 and 100. The color (RGB value) indicating the change amount δV is black (0,0,0) → ... → blue (0,0,255) → ... → light blue (0,255,255) → ... → green (0,255) , 0) → ... → yellow (255,255,0) → ... → red (255,0,0) → ... → dark red (127,0,0). Note that the reference values (18, 36, 55, 73, 91) of δV in FIG. 8 are roughly the values.

  Subsequently, the image generation unit 14 generates a grayscale image or a color contour image by setting the color of each unit region by applying the input change amount δV of each unit region to the scale of the heat map.

  Subsequently, the image generation unit 14 generates a difference image (heat map) by synthesizing the grayscale image or the color contour image and the comparison image. This difference image is an image that indicates which part of the comparison image has attracted more visual attention than the reference image, or conversely, which part of the comparison image has no longer attracted visual attention. is there. The image generation unit 14 outputs the difference image to the transmission unit 15.

  Examples of color contour images and difference images (heat maps) corresponding to the first to third methods are shown in FIGS. FIG. 9 is a diagram showing the color contour image 211 and the difference image 212 of the change amount δV obtained by the first method. FIG. 10 is a diagram showing the color contour image 221 and the difference image 222 of the change amount δV obtained by the second method. FIG. 11 is a diagram showing a color contour image 231 and a difference image 232 of the change amount δV obtained by the third method.

  In the first method, the difference in the visibility score is obtained as it is as the change amount δV, so that the change amount becomes a low value as a whole. Accordingly, as can be seen from the images 211 and 212 in FIG. 9, when the change amount δV is shown in gray scale, the image tends to be dark overall, and when the change amount δV is shown in color contour, the image is entirely black, blue Or tend to be shown in light blue. On the other hand, since the change amount δV is normalized according to the scale range of the heat map in the second or third method described above, light gray or white is also included as shown in the images 221, 222, 231, 232. A gray scale image or a color contour image including colors such as green, yellow, and red can be generated.

  The transmission unit 15 is a functional element that transmits the difference image to the user terminal T. This transmission is equivalent to outputting the amount of change in the line-of-sight distribution. The user terminal T receives and displays the difference image, so that the user can know how the line-of-sight distribution of the comparison image has changed compared to the reference image.

  Next, the operation of the line-of-sight prediction system 1 (analysis server 10) will be described using FIG. 12 and the line-of-sight analysis method according to the present embodiment will be described.

  When the reception unit 11 receives the reference image and the comparison image from the user terminal T (step S11, reception step), the analysis unit 12 applies the line-of-sight prediction to each of the two images, thereby determining the line-of-sight distribution of each image. Calculate (step S12, first calculation step). Subsequently, the change amount calculation unit 13 calculates a change amount between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image (step S13, second calculation step). As described above, there are various calculation methods for the change amount, and the change amount calculation unit 13 obtains the change amount δV in each unit region using an arbitrary method. Subsequently, the image generation unit 14 generates a difference image based on the amount of change (step S14), and the transmission unit 15 transmits the difference image to the user terminal T (step S15, output step). The difference image is displayed on the user terminal T by this series of processes, and the user can determine how the visual effect of the comparison image changes compared to the reference image.

  The user can view the difference image any number of times while changing at least one of the reference image and the comparison image. Each time the analysis server 10 receives the reference image and the comparison image from the user terminal T, the analysis server 10 repeatedly executes the processes of steps S11 to S15. The user can determine whether the image exhibits a desired visual effect while comparing the difference images. For example, as illustrated in FIG. 13, the user can compare only the comparison image 202 in which the character string “ABCDEFG” having a uniform color as a whole is added to the reference image 201 and a part of the character string (“DE”). It is also possible to compare the enhanced comparative image 203. FIG. 14 shows a difference image 301 obtained from the comparison image 202 and the reference image 201, and a difference image 302 obtained from the comparison image 203 and the reference image 201. Looking at the difference image 301, the change amount δV is uniformly high in almost the entire character string. On the other hand, when the difference image 302 is viewed, the amount of change δV of the “DE” portion of the character string protrudes and is high. Therefore, it can be seen that the comparative image 202 is more advantageous than the comparative image 203 when the entire character string attracts visual attention and the entire character string needs attention.

  When determining the visual effect of the character string region for the comparison image 202 and the comparison image 203, if it is determined only by the total sum (integration score) of the amount of change δV in that region, the integration score is the direction of the comparison image 203. May be larger than the comparison image 202. If such a calculation result is obtained, the user may determine that the comparison image 203 attracts visual attention to the entire character string more than the comparison image 202. In this embodiment, since the change amount δV for each unit area is output, the user can compare the images while observing the tendency of the change amount δV in the entire image or the entire area of interest (Area of Interest (AOI)). . Here, the region of interest is a part in which the user particularly wants to know whether or not the other person's visual attention can be attracted in the entire image.

  Next, a line-of-sight prediction program P1 for causing a computer to function as the line-of-sight prediction system 1 (analysis server 10) will be described with reference to FIG.

  The line-of-sight prediction program P1 includes a main module P10, a reception module P11, an analysis module P12, a change amount calculation module P13, an image generation module P14, and a transmission module P15.

  The main module P10 is a part that comprehensively controls the line-of-sight prediction function. The functions realized by executing the reception module P11, the analysis module P12, the variation calculation module P13, the image generation module P14, and the transmission module P15 are the reception unit 11, the analysis unit 12, and the variation calculation unit 13, respectively. The functions of the image generation unit 14 and the transmission unit 15 are the same.

  The line-of-sight prediction program P1 may be provided after being fixedly recorded on a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, the line-of-sight prediction program P1 may be provided via a communication network as a data signal superimposed on a carrier wave.

  As described above, the line-of-sight prediction system according to one aspect of the present invention includes one or more processors, and at least one processor receives a reference image and a comparative image having a common or similar design, and visually By applying line-of-sight prediction that predicts a region that tends to attract attention to both the reference image and the comparison image, the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparison image are calculated, and the line-of-sight distribution of the reference image And the line of sight distribution of the comparison image are calculated, and the amount of change is output as the line of sight distribution change.

  Also, in the gaze prediction method according to one aspect of the present invention, at least one processor receives a reference image and a comparative image having a common or similar design, and at least one processor attracts visual attention. A first calculation step of calculating a gaze distribution of the reference image and a gaze distribution of the comparison image by applying gaze prediction that predicts a region tending to be applied to both the reference image and the comparison image; and at least one The processor includes a second calculation step of calculating a change amount between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image, and an output step in which at least one processor outputs the change amount as a change in the line-of-sight distribution. .

  The eye gaze prediction program according to one aspect of the present invention is based on a reception step for receiving a reference image and a comparative image having a common or similar design, and eye gaze prediction for predicting an area that tends to attract visual attention. A first calculation step of calculating the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image by applying to both the image and the comparative image; and between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image The computer executes a second calculation step for calculating the amount of change and an output step for outputting the amount of change as a change in the line-of-sight distribution.

  In such an aspect, since the amount of change in the line-of-sight distribution of two images having a common or similar design is calculated and the result is output, differences between a plurality of designs can be clearly presented.

  In another aspect of the present invention, at least one processor divides each of the reference image and the comparison image into a plurality of unit regions, and applies visibility prediction to the reference image to obtain a plurality of unit region visibility scores. Is obtained as a gaze distribution of the reference image, and visibility scores of a plurality of unit areas obtained by applying gaze prediction to the comparison image are obtained as the gaze distribution of the comparison image, and the visibility in the unit area of the reference image is obtained. The process of calculating the amount of change between the score and the visibility score in the unit region of the corresponding comparison image may be executed for all unit regions, and the amount of change in each unit region may be output as a change in line-of-sight distribution. . By dividing the reference image and the comparison image into a plurality of unit regions and obtaining the amount of change for each unit region, it is possible to present a detailed change in the line-of-sight distribution.

  In another aspect of the present invention, at least one processor may output a change in the line-of-sight distribution in different colors. In this case, the change in the line-of-sight distribution can be shown in an easily understandable manner.

  In another aspect of the present invention, at least one processor may generate a difference image by superimposing a change in color-coded line-of-sight distribution and a comparison image, and output the difference image. In this case, it is possible to easily show how much the visual effect of which part of the image has changed.

  In another aspect of the present invention, at least one processor may normalize a change amount of each unit region between 0 and a maximum value in the visibility scores of the unit regions of the reference image and the comparison image. . By normalizing the amount of change in the unit area, the magnitude of the change in the line-of-sight distribution can be shown in an easily understandable manner.

  The present invention has been described in detail based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be variously modified without departing from the gist thereof.

  In the embodiment described above, the line-of-sight prediction system 1 shows the change in the line-of-sight distribution using a heat map, but the expression method is not limited to this. For example, the line-of-sight prediction system may output a region map indicating the change amount in several stages (for example, three stages). This region map is a technique for indicating a region where the line of sight gathers with a frame and a numerical value indicating the degree of concentration of the line of sight for each region. Alternatively, the line-of-sight prediction system 1 may represent changes in the line-of-sight distribution using only a color contour image or a grayscale image.

  The output destination of the change in the line-of-sight distribution is not limited. For example, the line-of-sight prediction system may store line-of-sight distribution change data in an arbitrary storage unit (for example, a database). In this case, only the change amount raw data may be stored without generating a difference image. .

  The present invention can also be applied to a single computer. That is, you may mount the function of the whole gaze prediction system in a user terminal.

  DESCRIPTION OF SYMBOLS 1 ... Gaze prediction system, 10 ... Analysis server, 11 ... Reception part, 12 ... Analysis part, 13 ... Change amount calculation part, 14 ... Image generation part, 15 ... Transmission part, P1 ... Gaze prediction program, P10 ... Main module, P11: reception module, P12: analysis module, P13: change amount calculation module, P14: image generation module, P15: transmission module, T: user terminal.

Claims (7)

With one or more processors,
At least one of the processors is
Accept reference and comparison images with a common or similar design,
By applying line-of-sight prediction that predicts a region that tends to attract visual attention to both the reference image and the comparison image, the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparison image are calculated,
Calculating the amount of change between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image;
Outputting the amount of change as a change in gaze distribution;
Gaze prediction system. The at least one processor comprises:
Dividing each of the reference image and the comparative image into a plurality of unit regions;
Obtaining a visibility score of the plurality of unit regions obtained by applying the line-of-sight prediction to the reference image as a line-of-sight distribution of the reference image;
Obtaining a visibility score of the plurality of unit regions obtained by applying the line-of-sight prediction to the comparison image as a line-of-sight distribution of the comparison image;
A process of calculating the amount of change between the visibility score in the unit area of the reference image and the visibility score in the corresponding unit area of the comparison image is executed for all unit areas.
The amount of change in each unit area is output as a change in the line-of-sight distribution.
The line-of-sight prediction system according to claim 1. The at least one processor outputs the change in the line-of-sight distribution in different colors;
The line-of-sight prediction system according to claim 1 or 2. The at least one processor generates a difference image by superimposing the change in the line-of-sight distribution color-coded and the comparison image, and outputs the difference image;
The gaze prediction system according to claim 3. The at least one processor normalizes a change amount of each unit region between 0 and a maximum value in the visibility scores of the unit regions of the reference image and the comparison image;
The line-of-sight prediction system according to any one of claims 2 to 4. An accepting step in which at least one processor receives a reference image and a comparative image having a common or similar design;
The at least one processor applies a line-of-sight prediction that predicts a region that tends to attract visual attention to both the reference image and the comparative image, so that the line-of-sight distribution of the reference image and the line of sight of the comparative image A first calculation step of calculating a distribution;
A second calculating step in which the at least one processor calculates a change amount between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image;
An eye gaze prediction method including: an output step in which the at least one processor outputs the amount of change as a gaze distribution change. Receiving a reference image and a comparative image having a common or similar design;
By applying gaze prediction that predicts a region that tends to attract visual attention to both the reference image and the comparison image, a first gaze distribution of the reference image and a gaze distribution of the comparison image are calculated. A calculation step;
A second calculation step of calculating an amount of change between the line-of-sight distribution of the reference image and the line-of-sight distribution of the comparative image;
A gaze prediction program for causing a computer to execute an output step of outputting the change amount as a gaze distribution change.

Images