JP2016162234A - Attention area detection device, attention area detection method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an attention area detection device, an attention area detection method and a program, capable of appropriately selecting an attention area with regard to an input image.SOLUTION: An attention area detection device includes: acquisition means 11 configured to acquire information on a sight line position of a user to an input image; setting means 12 configured to set a plurality of candidate areas in the input image; and detection means 13 configured to detect an attention area according to the plurality of candidate areas and a statistical distribution of the acquired sight line position of the user.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for detecting a region of interest from an image based on information about a user's line of sight.

  2. Description of the Related Art Conventionally, various methods for detecting a user's line of sight and using the user's line of sight information to detect a region that the user is paying attention to are known. In the method described in Patent Document 1, in order to detect which position on the finder surface of the camera is focused, a plurality of local regions are set on the finder surface, and the user's gaze stop time is measured for each local region. To do. And the local area | region where a stop time becomes the maximum is set as an attention area which the user has paid attention to.

JP 2000-131599 A JP-A-1-241511

Ken Takegami, Toshiyuki Goto, Gen Oyama, "Highly accurate pupil detection algorithm for gaze direction measurement", IEICE Transactions. D-II, information / system, II-pattern processing. Dong Hyun Yoo and Myung Jin Chung, “Non-intrusive Eye Gaze Estimate with Knowledge of Eye Pose”, Proc. IEEE FG 2004. R. Achanta, S .; Hemimi, F.A. Estrada, and S.E. Susstrunk, “Frequency-tuned salient region detection”, CVPR2009. Nobuyuki Otsu, “Automatic threshold selection method based on discrimination and least squares”, IEICE Transactions, J63-D-4 (1980-4), 349.356. L. Itti, C.I. Koch, and E.M. Niebur, “A model of saliency-based visual attention for rapid scene analysis”, TPAMI 1998. D. Klein and S.K. Frintrop, “Center-surround diversity of feature statistics for saliency object detection”, ICCV2011. M.M. Yamanaka, M .; Matsusu, and M.M. Sugiyama, “Salient object detection based on direct density-ratio estimation”, IPSJ2013. Paul Viola and Michael J. et al. Jones. “Rapid Object Detection using a Boosted Cascade of Simple Features”, CVPR2001. Yusuke Mitarai, Katsuhiko Mori, Yukazu Masashi “Face Detection System with Selective Module Activation and Robust against Variations” FIT2003

  However, in the method described in Non-Patent Document 1, since a plurality of local regions for detecting the user's line of sight are determined at predetermined positions, when a region of interest is not included in the plurality of local regions, There was a problem that the attention area could not be selected properly.

  In order to solve the above-described problems, the present invention provides an acquisition unit that acquires information about a user's line-of-sight position with respect to an input image, a setting unit that sets a plurality of candidate areas in the input image, and the set plurality of Detecting means for detecting a region of interest based on the acquired candidate region and the statistical distribution of the acquired gaze position of the user.

  According to the above configuration, the present invention can accurately detect a region of interest to which the user is interested based on information on the user's line of sight.

1 is a schematic block diagram illustrating a configuration of a region of interest detection apparatus according to a first embodiment. The figure explaining the function of a candidate area | region setting part in 1st Embodiment. The schematic block diagram which shows the function structure of the attention area detection part in connection with 1st Embodiment. The figure which shows the example of a process of the candidate area | region setting part in 1st Embodiment. The figure which shows the graph of the m-order function k (t ') with respect to time t' in 1st Embodiment. The figure which shows an example of the setting process of the attention area by the area | region setting part in 1st Embodiment. The figure which shows the change of the attention area | region with progress of time in 1st Embodiment. The flowchart of the attention area detection method in connection with 1st Embodiment. The schematic block diagram which shows the structure of the attention area detection apparatus in connection with 2nd Embodiment. The figure explaining the process of the face area detection by a face detection part in 2nd Embodiment. The figure explaining the setting process of the candidate area | region of an attention area in 2nd Embodiment. The figure which shows the setting process of the attention area by the area | region setting part in 2nd Embodiment. The flowchart of the attention area detection method in connection with 2nd Embodiment.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of the attention area detection device according to the present embodiment. The attention area detection device 1 includes a line-of-sight information acquisition unit 11, a candidate area setting unit 12, and an attention area detection unit 13.

  The attention area detection apparatus 1 according to the present embodiment is realized using a semiconductor integrated circuit (LSI). Alternatively, the attention area detection device 1 may include a hardware configuration such as a CPU, a ROM, a RAM, and an HDD. In that case, the CPU executes a program stored in the ROM, HD, or the like, thereby realizing, for example, each functional configuration and flowchart processing described below. The RAM has a storage area that functions as a work area where the CPU develops and executes the program. The ROM has a storage area for storing programs executed by the CPU. The HD has a storage area for storing various types of data including various programs necessary for the CPU to execute processing, data on threshold values, and the like.

  The line-of-sight information acquisition unit 11 acquires information related to the user's line-of-sight position (x (t), y (t)) with respect to the input image at time t. For the user's line-of-sight position (x (t), y (t)), for example, a method using a high-precision pupil detection algorithm described in Non-Patent Document 1 or an image process described in Non-Patent Document 2 is used. It can be detected by a non-contact gaze detection method. Using these techniques, the line-of-sight information acquisition unit 11 itself may detect and acquire the user's line-of-sight position from a photograph (image) shown by the user.

The candidate area setting unit 12 acquires an input image input from the outside, and sets the candidate area R _n (n = 1, 2,..., N) of the attention area in the input image. As the method, for example, a method described in Non-Patent Document 3 that uses a saliency obtained from a low-order feature amount (luminance value, edge strength, texture, etc.) can be employed. FIG. 2 is a diagram for explaining processing in which the candidate area setting unit 12 sets the candidate area R _n (n = 1, 2,..., N) of the attention area by the method of Non-Patent Document 3. If the input image as shown in FIG. 2A is acquired, the candidate area setting unit 12 calculates the saliency S at the point (x, y) at each position in the input image, and the saliency image is obtained. Create (FIG. 2B). In the saliency image, all the areas of the saliency S that are equal to or greater than the predetermined threshold Th are extracted (FIG. 2C). Further, a rectangular area including each extracted area is set as a candidate area R _n (n = 1, 2,..., N) of the attention area (FIG. 2D).

Note that the threshold Th for the saliency S is, for example, the threshold value processing method described in Non-Patent Document 4 (the threshold is adaptively set by minimizing intra-class variance and maximizing inter-class variance). The calculation may be performed by the above method. N represents the number of candidate areas for the attention area, and is given by N = 2 in the case of FIG. In addition, as a method for calculating the saliency S, the methods described in Non-Patent Documents 5 to 7 may be used. The candidate region R _n (n = 1, 2,..., N) set in this way is subjected to the statistical distribution of the line-of-sight position (of the stopping time) in the attention region setting processing by the attention region detection unit 13 at the subsequent stage. It is used as an initial value of a range (position and size) in the image space when calculating a total value or the like.

  The attention area detection unit 13 detects the attention area based on the information regarding the user's line-of-sight position acquired by the line-of-sight information acquisition unit 11 and the candidate area R of the attention area set by the candidate area setting unit 12. FIG. 3 is a diagram illustrating a functional configuration of the attention area detection unit 13. As shown in the figure, the attention area detection unit 13 includes an area selection unit 131, a statistic calculation unit 132, an area centroid calculation unit 133, an area size calculation unit 134, and an area setting unit 135.

The region selection unit 131 sets the user's line-of-sight position (x (t (t)) from time t = 0 to T for each of the candidate regions R _n (n = 1, 2,..., N) set by the candidate region setting unit 12. The total value of the stop times of t) and y (t)) is calculated. Then, the region selection unit 131 selects the candidate region RεR _n (n = 1, 2,..., N) having the maximum value, and the barycentric position (X, Y) and size (W, H). Here, (X, Y) represents the horizontal coordinate and the vertical coordinate of the center of gravity of the selected candidate region R. (W, H) represents the horizontal size and vertical size of the selected candidate region R.

FIG. 4 is a diagram illustrating an example of processing of the region selection unit 131. In the example of the figure, among all the candidate regions R _n (n = 1, 2,..., N), the candidate region R ₂ is the stop of the user's line-of-sight position (x (t), y (t)). Since the total value of time is the maximum, the region selection unit 131 acquires the center of gravity (X, Y) and the size (W, H). When the region selection unit 131 acquires the centroid (X, Y) and size (W, H) of the selected candidate region R, the information is used as the statistic calculation unit 132, the region centroid calculation unit 133, and the region size calculation unit 134. Output to.

The statistic calculation unit 132, in the candidate region R selected by the region selection unit 131, the time average position (g _x ) of the user's line-of-sight position (x (t), y (t)) from time t = 0 to T. , G _y ) is calculated from equation (1).

Further, the statistic calculator 132 calculates the variance (σ _x ) of the user's line-of-sight position (x (t), y (t)) from time t = 0 to T in the candidate region R selected by the region selector 131. ² , σ _y ² ) is calculated from equation ( ² ).

The area centroid calculation unit 133 calculates the time based on the centroid (X, Y) of the candidate area R and the time average position (g _x , g _y ) of the user's line-of-sight position (x (t), y (t)). The center of gravity (X ′, Y ′) of the attention area R ′ at t ′ (> T) is calculated. The center of gravity (X ′, Y ′) of the attention area R ′ is gradually set as the center of gravity (X, Y) of the candidate area R and the center of gravity of the user's line-of-sight position (x (t), y (t)) as time passes. Is set to a position approaching Specifically, the area centroid calculating unit 133 calculates the centroid (X ′, Y ′) of the attention area R ′ based on the following equation (3).

Here, k _g (t ′) is given as an m-th order function with respect to time t ′, and is expressed as the following equation (4). FIG. 5 shows a graph of an m-th order function k (t ′) with respect to time t ′. FIGS. 5A to 5D show the function k (t ′) when m = 1 to 4. It is a graph. The function k _g (t ′) in this embodiment is given as an m-order function with respect to time t ′ as shown in FIG.

Here, the coefficient a _g in Formula 4 are predetermined real value, is predetermined in accordance with the processing speed and visual axis detection accuracy of the system. Note that k _g (t ′) may be given as another function as long as it is a multidimensional function with respect to time t ′.

The region size calculation unit 134 is based on the size (W, H) of the candidate region R and the variance (σ _x ² , σ _y ² ) of the user's line-of-sight position (x (t), y (t)). Then, the size (W ′, H ′) of the attention area R ′ at time t ′ (> T) is calculated. The size (W ′, H ′) of the attention region R ′ is a spatial distribution of the user's line-of-sight position (x (t), y (t)) with respect to the size (W, H) of the candidate region R. When the width is large, the size (W, H) of the candidate region R is set to gradually increase as time passes. On the other hand, when the width of the spatial distribution of the user's line-of-sight position (x (t), y (t)) is small with respect to the size (W, H) of the candidate region R, the candidate region R The size (W, H) is set to be gradually reduced. Specifically, the region size calculation unit 134 calculates the size (W ′, H ′) of the attention region R ′ at time t ′ (> T) by the following equation (5).

Here, k _s (t ′) is given as an m-th order function with respect to time t ′, and is expressed as the following Expression 6. This function k _s (t ′) is also an m-th order function for time t ′ as shown in FIGS.

Here, the coefficient a _s in equation (6) is a predetermined real value, is predetermined in accordance with the processing speed and visual axis detection accuracy of the system. Note that k _s (t ′) may be given as another function as long as it is a multidimensional function with respect to time t ′.

  The region setting unit 135 includes the center of gravity (X ′, Y ′) of the attention region R ′ calculated at the time t ′ calculated by the region gravity center calculation unit 133 and the attention region R at the time t ′ calculated by the region size calculation unit 134. Based on the size of 'W', H ', the attention area R' at time t 'is set. FIG. 6 is a diagram illustrating an example of processing for setting the attention area R ′ by the area setting unit 135. As shown in the figure, in this embodiment, the coordinates (X ′, Y ′) are set as the center of gravity, and the rectangular area having the size (W ′, H ′) is set as the attention area R ′. Note that the attention area R ′ set in this way may be displayed superimposed on the input image on display means (such as a display) provided outside the attention area detection device.

FIG. 7 is a diagram illustrating an example of a change in the attention area R ′ over time. As shown in the figure, the attention area R ′ is continuously enlarged, reduced, and translated in time according to the values of k _g (t ′) and k _s (t ′) at time t ′. When an interrupt operation (for example, a button is pressed) from the user is performed at time t ′ = T _int , the attention area detection device acquires the information, and the area setting unit 135 _{'= T int} after the enlargement, reduction, to stop the movement parallel. The region setting unit 135 also determines the final region of interest R based on the center of gravity (X ′, Y ′) and the size (W ′, H ′) of the region of interest R ′ at time t ′ = _Tint. Set '. On the other hand, when the user's interruption operation is not performed until time t ′ = T _end where k _g (t ′) = k _s (t ′) = 1 is satisfied, the attention area R ′ at time t ′ = T _end . The final region of interest R ′ is set based on the center of gravity (X ′, Y ′) and the size (W ′, H ′).

  The attention area detected in this way is output to a device that performs processing using the attention area detection result. For example, in an imaging apparatus such as a digital camera, the detected region of interest is focused and subjected to processing for improving the image quality of the region. In addition, a semiconductor integrated circuit provided in an imaging apparatus such as a digital camera may realize the function as the above-described attention area detection device. In this case, the imaging apparatus itself is the attention area detection device of this embodiment. It corresponds to. Further, when the imaging device itself functions as the attention area detection device in this way, information on the user's line of sight is measured by the attention area detection device itself from the viewfinder, for example, by a method disclosed in Patent Document 2. You may make it acquire.

FIG. 8 shows a flowchart of the attention area detection method according to the present embodiment. When this process is started, first, in step S101, the line-of-sight information acquisition unit 11 acquires information about the user's line-of-sight position (x (t), y (t)). Next, in step S102, the candidate area setting unit 12 sets candidate areas R _n (n = 1, 2,..., N) of the attention area in the input image.

Subsequent steps S103-1 to S103-5 are steps in which the attention area is detected by the attention area detection unit 13. First, in step S103-1, the region selection unit 131 sets the user's line-of-sight position (x (t), y (t) for each of the set candidate regions R _n (n = 1, 2,..., N). )) The total stop time is calculated. In step S103-2, the statistic calculator 132 calculates the time average position (g _x , g _y ) of the user's line-of-sight position (x (t), y (t)) at time t = 0 to T, The variance (σ _x ² , σ _y ² ) of the user's line-of-sight position (x (t), y (t)) at time t = 0 to T is calculated.

In step S103-3, the area centroid calculation unit 133 determines the time t ′ (> T) based on the centroid (X, Y) of the candidate area R and the time average position (g _x , g _y ) of the user's line-of-sight position. ) To calculate the center of gravity (X ′, Y ′) of the attention area R ′. In step S103-4, the region size calculation unit 134 determines the time t based on the size (W, H) of the candidate region R and the variance (σ _x ² , σ _y ² ) of the user's line-of-sight position. The size (W ′, H ′) of the attention area R ′ in “(> T) is calculated.

  In step S103-5, the area setting unit 135 calculates the center of gravity (X ′, Y ′) of the attention area R ′ at time t ′ and the size (W ′, H) of the attention area R ′ at time t ′. Based on “), the attention area R ′ at time t ′ is set.

  As described above, according to the present embodiment, the center of gravity (X ′) of the attention area R ′ is based on the stop time of the user's gaze position, which is a statistical distribution of the user's gaze position (x (t), y (t)). , Y ′) and size (W ′, H ′). With this configuration, in the present embodiment, it is possible to accurately detect a region of interest to which the user pays attention.

[Second Embodiment]
Next, as a second embodiment of the present invention, a description will be given of a configuration for detecting a region of interest to which the mth person pays attention based on the detected face region of the person. In addition, about the structure already demonstrated in 1st Embodiment, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

  FIG. 9 is a schematic block diagram illustrating the configuration of the attention area detection device according to the present embodiment. The attention area detection device 1 according to the present embodiment includes a face detection unit 21 in addition to the functional units described in the first embodiment. First, the face detection unit 21 acquires an image for detecting the face of a person who pays attention to a display unit that displays an input image to be processed by the attention area detection device. Here, for example, an image taken by a camera attached to a digital signage that displays an input image to be processed is acquired.

Then, the face detection unit 21 detects the face regions F _m (m = 1, 2,..., M) of all persons from the acquired image. FIG. 10 shows an example of detection of the face region F _m (m = 1, 2,..., M) by the face detection unit 21. M represents the number of face areas of a person (in the case of FIG. 10, M = 4). For example, the methods disclosed in Non-Patent Documents 8 and 9 may be used as a method for detecting a human face area. Further, the user detection method for detecting the user who pays attention to the input image is not limited to the method based on the face area as described above, and may be performed based on other human body parts.

  The line-of-sight information acquisition unit 11 acquires information regarding the line-of-sight position (x (t), y (t)) of the mth person (user) at time t. As described in the first embodiment, the method of detecting the user's line-of-sight position can employ methods such as Non-Patent Documents 1 and 2. The line-of-sight information acquisition unit 11 itself may detect and acquire the user's line-of-sight position.

The candidate area setting unit 12 acquires an input image input from the outside, and sets the candidate area R _n (n = 1, 2,..., N) of the attention area in the input image. FIG. 11 is a diagram illustrating processing for setting a candidate region R _n (n = 1, 2,..., N) of a region of interest by the candidate region setting unit 12 in the present embodiment. FIG. 5A shows an example of an input image of the present embodiment, and here is an advertisement (digital signage) composed of images, texts, and the like targeted at an unspecified number of people. As shown in FIG. 11 (b), the candidate area setting unit 12 assigns each of a group of areas (position and size are known) as meaning areas to the input image. It is set as a candidate area R _n (n = 1, 2,..., N).

  The attention area detection unit 13 includes functional units similar to those in the first embodiment, and includes an area selection unit 131, a statistic calculation unit 132, an area centroid calculation unit 133, an area size calculation unit 134, and an area setting unit 135. Have

The area selection unit 131 displays the line-of-sight position of the mth person from time t = 0 to T in all candidate areas R _n (n = 1, 2,..., N) set by the candidate area setting unit 12. The total value of the stop times of (x _m (t), y _m (t)) is calculated. Then, the candidate region R _m εR _n (n = 1, 2,..., N) having the maximum total value is selected.

The statistic calculation unit 132 selects the line-of-sight position (x _m (t), y) of the m-th person from time t = 0 to T in the m-th person candidate region R _m selected by the region selection unit 131. The time average position (g _x , g _y ) of _m (t)) is calculated from Equation 7.

Further, the statistic calculation unit 132, in the m th person candidate region R _m selected by the region selection unit 131, the line-of-sight position (x _m (t), The variance (σ _x ² , σ _y ² ) of y _m (t)) is calculated from Equation 8.

The area center-of-gravity calculation unit 133 calculates the time-average position (g _x , g _y ) of the center of gravity (X _m , Y _m ) and the line-of-sight position (x _m (t), y _m (t)) of the m-th person candidate area R. ) And the center of gravity (X _m ′, Y _m ′) of the attention area R _m ′ of the m-th person at time t ′ (> T). Specifically, the area centroid calculation unit 133 calculates the centroid (X _m ′, Y _m ′) of the attention area R ′ of the m-th person based on the following equation (9).

Further, the area size calculation unit 134 is based on the mth person candidate area R _m and the variance (σ _x ² , σ _y ² ) of the line-of-sight position (x _m (t), y _m (t)). The size (W _m ′, H _m ′) of the attention area R _m ′ of the m-th person at time t ′ (> T) is calculated from Equation 10.

The size of the area setting unit 135, 'm th person of the region of interest R _m' in the center of gravity of the time t (X _m ', Y _m') and 'attention area of the m-th person in R _m' at time t Based on (W _m ′, H _m ′), the attention area R _m ′ of the m-th person at time t ′ is set.

FIG. 12 is a diagram illustrating an example of processing for setting the attention area R _m ′ by the area setting unit 135. As shown in the figure, in the present embodiment, the coordinate (X _m ′, Y _m ′) is set as the center of gravity, and the rectangular area having the size (W _m ′, H _m ′) is set as the attention area R _m ′. . Note that the attention area R _m ′ set in this way may be displayed superimposed on the input image on a display means (display or the like) provided outside the attention area detection device. As shown in FIG. 10, when a plurality of face areas are detected, only the attention area corresponding to the person of the face area detected as having the highest importance may be displayed. May be displayed.

  The attention area detected in this way is output to a device that performs processing using the attention area detection result. For example, in an electronic display device such as digital signage, only the detected region of interest is subjected to processing such as increasing the bit rate to improve image quality. In addition, a semiconductor integrated circuit provided in an imaging device such as an electronic display device may realize the function as the attention area detection device described above. In this case, the electronic display device itself is the attention area of the present embodiment. It corresponds to a detection device. Further, the electronic display device may be provided with a camera (imaging means), and an image of a person who pays attention to the electronic display device may be taken.

FIG. 13 is a flowchart of the attention area detection method according to the present embodiment. When this process is started, first, in step S201, the face detection unit 21 detects the face regions F _m (m = 1, 2,..., M) of all persons from the image. In step S101, the line-of-sight information acquisition unit 11 acquires information regarding the line-of-sight position (x (t), y (t)) of the mth person. In subsequent steps S102 to S103-5, processing similar to the processing described in the first embodiment is executed based on the line-of-sight position information of the mth person. Further, if the attention area is detected for all detected face areas F _m (m = 1, 2,..., M), this flow may be repeatedly executed.

  As described above, according to the present embodiment, it is possible to accurately detect the attention area focused on by a specific person by extracting the face area of the person and detecting the attention area focused on by the specific person. It becomes.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the present embodiment, based on the kurtosis (S _x , S _y ) and the skewness (K _x , K _y ) representing the characteristics of the spatial distribution of the user's line-of-sight position (x (t), y (t)). The center of gravity (X ′, Y ′) and size (W ′, H ′) of the attention area R ′ are set. In addition, about the structure already demonstrated in 1st, 2nd embodiment, the same code | symbol is attached | subjected and the description is abbreviate | omitted. The attention area detection device 1 and its processing flow of this embodiment are the same as those of the first embodiment, but the contents of the processing by the statistic calculation unit 132, the region centroid calculation unit 133, and the region size calculation unit 134 are the first. This is different from the embodiment.

First, in the present embodiment, the statistic calculator 132 calculates the user's line-of-sight position (x (t), y (t)) from time t = 0 to T in the candidate region R selected by the region selector 131. The time average position (g _x , g _y ) is calculated by the equation (11).

In addition, the statistic calculation unit 132 distributes (σ _x ² ) the line-of-sight position (x (t), y (t)) of the user from time t = 0 to T in the candidate region R selected by the region selection unit 131. , Σ _y ² ) is calculated from equation (12).

In addition, the statistic calculation unit 132, in the candidate region R selected by the region selection unit 131, the skewness (S _x of the user's line-of-sight position (x (t), y (t)) from time t = 0 to T. , S _y ) is calculated from equation (13). Here, the skewness (S _x , S _y ) represents the characteristics of the statistical distribution of the user's line-of-sight position (x (t), y (t)) from time t = 0 to T. When is positive (negative), it means that the statistical distribution is biased in the positive (negative) direction.

Further, the statistic calculation unit 132, in the candidate region R selected by the region selection unit 131, the kurtosis (K _x ) of the user's line-of-sight position (x (t), y (t)) from time t = 0 to T. , K _y ) is calculated from equation (14). Here, the kurtosis (K _x , K _y ) represents the characteristics of the statistical distribution of the user's line-of-sight position (x (t), y (t)) from time t = 0 to T. When is positive (negative), the statistical distribution is sharp (flat).

Based on the centroid (X, Y) of the candidate region R and the skewness (S _x , S _y ) of the user's line-of-sight position (x (t), y (t)), the region centroid calculation unit 133 The center of gravity (X ′, Y ′) of the attention area R ′ at “(> T) is calculated. Specifically, the area centroid calculation unit 133 calculates the centroid (X ′, Y ′) of the attention area R ′ based on Expression 15. As a result, the center of gravity (X ′, Y ′) of the attention area R ′ is the same as the center of gravity (X, Y) of the candidate area R and the center of gravity of the user's line-of-sight position (x (t), y (t)). And it is set as a position that gradually approaches.

Further, the region size calculation unit 134 is based on the size (W, H) of the candidate region R and the kurtosis (K _x , K _y ) of the user's line-of-sight position (x (t), y (t)). Then, the size (W ′, H ′) of the attention area R ′ at time t ′ (> T) is calculated. Specifically, the region size calculation unit 134 calculates the size (W ′, H ′) of the attention region R ′ based on Expression 16. The size (W ′, H ′) of the attention area R ′ is a statistical distribution of the user's line-of-sight position (x (t), y (t)) with respect to the size (W, H) of the candidate area R. When the width is large (small), the size (W, H) of the candidate region R is set to gradually expand (reduce) with time.

As described above, according to the present embodiment, the kurtosis (S _x , S _y ) and the skewness (K _x , K _y ) representing the characteristics of the spatial distribution of the user's line-of-sight position (x (t), y (t)). ), The center of gravity (X ′, Y ′) and the size (W ′, H ′) of the attention area R ′ are set. With this configuration, in the present embodiment, it is possible to accurately detect a region of interest to which the user pays attention.

[Other Embodiments]
In addition, the present invention supplies software (program) for realizing the functions of the above-described embodiments to a system or apparatus via a network or various storage media, and the computer of the system or apparatus (or CPU, MPU, etc.) programs Is read and executed. Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. The present invention is not limited to the above embodiments, and various modifications (including organic combinations of the embodiments) are possible based on the spirit of the present invention, and these are excluded from the scope of the present invention. is not. That is, the present invention includes all the combinations of the above-described embodiments and modifications thereof.

DESCRIPTION OF SYMBOLS 1 Attention area detection apparatus 11 Gaze information acquisition part 12 Candidate area setting part 13 Attention area detection part

Claims (14)

Acquisition means for acquiring information on the user's line-of-sight position with respect to the input image;
Setting means for setting a plurality of candidate areas in the input image;
Detecting means for detecting a region of interest based on the plurality of set candidate regions and a statistical distribution of the acquired gaze position of the user;
An attention area detecting device characterized by comprising:   The attention area detection apparatus according to claim 1, wherein the detection unit selects one candidate area from the plurality of set candidate areas based on a stop time of the user's line-of-sight position. .   The attention area detection according to claim 2, wherein the detection unit determines the position of the attention area based on a gravity center position of the selected candidate area and a time average position of the user's line-of-sight position. apparatus.   The attention area detection apparatus according to claim 2, wherein the detection unit determines a position in the attention area based on a gravity center position of the selected candidate area and a skewness of the user's line-of-sight position. .   The said detection means determines the magnitude | size of the said attention area | region based on the magnitude | size of the said selected candidate area | region and dispersion | distribution of the said user's gaze position, The said any one of Claim 2 to 4 characterized by the above-mentioned. The attention area detection device described.   The said detection means determines the magnitude | size of the said attention area | region based on the magnitude | size of the selected candidate area | region and the kurtosis of the said user's eyes | visual_axis position, The one of Claim 2 to 4 characterized by the above-mentioned. The attention area detection device described in 1. A user detecting unit for detecting a user who pays attention to the input image;
7. The attention area detection apparatus according to claim 1, wherein the detection unit detects the attention area that a specific user pays attention to among the detected users. 8.   The attention area detection apparatus according to claim 7, wherein the user detection unit detects a user who pays attention to an input image by detecting a face area.   The said setting part calculates saliency from the low-order feature-value in each position of the said input image, and sets several candidate area | region based on this saliency. The attention area detection device according to item.   The attention area detection apparatus according to claim 1, wherein the setting unit sets an area forming one content included in the input image as one candidate area.   The attention area detection apparatus according to claim 1, wherein the attention area detected by the detection unit is displayed superimposed on an input image.   The attention area detection apparatus according to claim 1, wherein the attention area detected by the detection unit is displayed with high image quality. Obtaining information about a user's line-of-sight position with respect to the input image;
Setting a plurality of candidate regions in the input image;
Detecting a region of interest based on the plurality of set candidate regions and the statistical distribution of the acquired gaze position of the user;
A region of interest detection method characterized by comprising:   A program for causing a computer to function as the attention area detection device according to any one of claims 1 to 12.