KR20130043369A - Apparatus for deciding user's intention and method therof - Google Patents

Abstract

PURPOSE: A user intention determination device and a method thereof are provided to determine a user intention change point by determining information maintenance intention, information extinction intention, and information occurrence intention. CONSTITUTION: A capturing unit(110) captures a user. A detecting unit(120) detects gaze information by using an image. A determination unit(130) generates an intention index from gaze information. A determination unit determines the intention of a user by using the intention index. An intention index generation unit(131) generates the intention index by collecting the gaze information. An intention index conversion unit(132) normalizes the intention index. [Reference numerals] (110) Capturing unit; (120) Detecting unit; (131) Intention index generation unit; (132) Intention index conversion unit; (133) Intention determination unit;

Description

Apparatus and method for determining user intention {APPARATUS FOR DECIDING USER'S INTENTION AND METHOD THEROF}

The present invention relates to an apparatus and a method for determining a user's intention, and more particularly, to an apparatus and a method for determining a user's intention by analyzing the user's gaze information.

Psychologically, it is very common to judge a person's psychological state or intention based on his or her specific behavior or words. However, in the case of a person who is unable to express his or her intention (for example, a language impaired person), in addition to this psychological approach, an effort to judge the intention by using a specific biological response of these people is required.

In this case, it is common to take a psychological approach based on the response of the autonomic nervous system to make a specific response irrespective of one's will. In particular, the human eye not only identifies things but also exhibits many changes in response to specific mental states and intentions of the human being. In addition, since the human eye is exposed to the outside and thus it is relatively easy to measure the reaction, a technique for determining the intention of the human being has been developed based on the reaction in the human eye.

Furthermore, in the past, electronic devices have been operated by simply inputting a user directly, but in order to provide a more convenient and user-adaptive service to a user, a technology for identifying a user's intention and providing a service suitable for a user is required. ought.

The present invention is in accordance with the above-described needs, an object of the present invention is to provide a device and method for determining the user's intention by detecting and analyzing the gaze information of the user.

In order to achieve the above object, a user intention determination device according to an embodiment of the present invention includes an imaging unit for photographing a user, a detection unit for detecting gaze information using the captured image, and an intention from the detected gaze information. And a determination unit configured to generate an indicator and determine the intention of the user using the intention indicator.

The gaze information may be at least one of a gaze point, a gaze time, a gaze path, a pupil size, and a tendency of a pupil size change, and the intention indicator is a pupil size, a slope of a pupil size change, and a gaze of a region of interest. At least one of a fixed time, an eye fixation frequency for an area of interest (AOI), and an eye fixation frequency for an area other than the ROI.

The determination unit may further include an intention indicator generator configured to collect the eyeline information at predetermined time intervals to generate the intention indicator, an intention indicator converter to normalize the generated intention indicator, and a user using the normalized intention indicator. It may include an intention determination unit for determining the intention of the.

The intention of the user may be any one of a navigation intention, an information generation intention, an information destruction intention, and an information maintenance intention.

In addition, the user intention determination apparatus according to an embodiment of the present invention may further include a storage unit for storing the intention indicator data of the user's intention analysis, the intention determination unit, the intention normalized by the intention indicator conversion unit The intention of the user may be determined by comparing the indicator with the stored intention indicator data.

Here, the intention indicator converting unit may normalize the generated intention indicator using the following equation.

Where dn is the value of the normalized intention indicator, di is the value of the intention indicator corresponding to the time interval to be normalized, max (d) and min (d) are the maximum and minimum values of the intention indicator within the entire time interval, and n Is a constant.

The intention determination unit may further determine a change in point of intention of the user using the determined user intention.

On the other hand, the user intention determination method according to an embodiment of the present invention, imaging the user, detecting the gaze information by using the captured image and generates an intention indicator from the detected gaze information and the intention indicator Determining the intention of the user by using a.

The gaze information may be at least one of a gaze point, a gaze time, a gaze path, a pupil size, and a tendency of a pupil size change, and the intention indicator is a pupil size, a slope of a pupil size change, and a gaze of a region of interest. It may be at least one of a fixed time, a gaze fixation number for the ROI, and a gaze fixation number for an area other than the ROI.

The determining may include generating the intention indicator by collecting the gaze information at predetermined time intervals, normalizing the generated intention indicator, and determining a user's intention using the normalized intention indicator. It may include the step.

The intention of the user may be any one of a navigation intention, an information generation intention, an information destruction intention, and an information maintenance intention.

The determining of the intention may include determining the intention of the user by comparing the intention indicator normalized by the intention indicator conversion unit with the intention of the user.

In the normalizing step, the generated intention index may be normalized using the following equation.

Where dn is the value of the normalized intention indicator, di is the value of the intention indicator corresponding to the time interval to be normalized, max (d) and min (d) are the maximum and minimum values of the intention indicator within the entire time interval, and n Is a constant.

In addition, the determining of the intention may further determine a time point of changing the intention of the user by using the determined user intention.

According to various embodiments of the present disclosure as described above, the user's intention may be determined by detecting and analyzing the gaze information of the user.

1 is a block diagram illustrating a configuration of an apparatus for determining a user's intention according to an exemplary embodiment.
2 is a block diagram illustrating a configuration of an apparatus for determining a user's intention according to an exemplary embodiment.
3 shows data obtained by continuously detecting the pupil size.
4 shows the result of collecting continuous data on the detected pupil size at regular time intervals.
FIG. 5A illustrates a result of generating an intention index value for a slope of a pupil size change using pupil size data corresponding to each time interval.
5B and 5C illustrate intention indicators corresponding to the number of gaze fixing times and gaze fixing times for regions of interest and regions other than the region of interest, respectively, for each time interval.
6 is a table showing the result of normalizing the intention index.
7 illustrates a graph in which the intention index is normalized using Equation 1.
8 shows the intention determined according to the pentagonal graph and graph characteristics.
9 is a flowchart illustrating a method of determining an intention according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to the present invention.

1 is a block diagram illustrating a configuration of an apparatus for determining a user's intention according to an exemplary embodiment. According to FIG. 1, the apparatus 100 for determining user intention includes an image capturing unit 110, a detecting unit 120, and a determining unit 130.

The imaging unit 110 photographs a user. In detail, the image capturing unit 110 may image the user with the eyes of the user as the center. Meanwhile, in the embodiment of the present invention, although the imaging unit 110 is expressed as being embedded in the intention determination device 100, the imaging unit 110 may be implemented as an external device connected to the intention determination device 100.

The detector 120 detects the user's gaze information about the image captured by the imaging unit 110. In detail, the detector 120 may detect a pupil size, a pupil size change, a gaze point, a gaze time, and a gaze path on an image captured by the imaging unit 110. The gaze information of the user detects the pupil size on the image captured by the imaging unit 110 to determine the pupil size, detects the gaze point of the user based on the detected pupil position, and the gaze detection operation May be continuously performed to detect a user's gaze time, gaze path, and pupil size change. Specifically, the pupil size detection may be continuously performed to detect the tendency of the pupil size change, and the gaze point detection may be continuously performed to detect the gaze time and the gaze path.

3 shows data obtained by continuously detecting the pupil size. The three graphs of FIG. 3 each show data obtained by detection in different environments. The tendency (+,-or 0) of pupil size change can be detected from the data for pupil size in FIG. Here, only the process of detecting the pupil size and the tendency of the pupil size has been described, but the gaze point, the gaze time, and the gaze path may also be detected by the same process.

Meanwhile, in an exemplary embodiment of the present invention, the imaging unit 110 and the detection unit 120 are described as having separate configurations. However, the image pickup unit 110 and the detection unit 120 may be implemented in one configuration using an eye tracking apparatus.

The determination unit 130 determines the intention of the user by using the detected gaze information of the user. In detail, the determination unit 130 may generate the intention indicator from the detected user's eye information and determine the intention of the user by using the generated intention indicator. To this end, the determination unit 130 may be composed of the intention indicator generation unit 131, the intention indicator conversion unit 132, the intention determination unit 133.

In this case, the intention indicator means the pupil size, the slope of the pupil size change, the eye fixation time for the region of interest, the number of eyes fixed for the region of interest, and the number of eyes fixed for the region other than the region of interest. May be at least one of '.

Here, 'P Pupil Size (PS)' means the absolute size of the pupil, 'PG (Puppil Gradient)' is the amount of change in the pupil size with respect to time, and 'AOI' Fixation Time AOI (FTA) is the time for the user's gaze to stay in the area of interest, and 'Fixation Frequency AOI (FFA) for the area of interest' means that the user's gaze stays in the area of interest. 'Fixation Frequency Non AOI (FFNA)' refers to the number of times a user's gaze stays in an area outside the region of interest.

In addition, the region of interest refers to a region determined according to a user's intention. In more detail, the ROI refers to an area including a specific object or information on a user's field of view corresponding to a user's purpose.

The intention indicator generator 131 generates the intention indicator using the gaze information detected by the detector 120. In detail, the intention indicator generation unit 131 collects the pupil size, the pupil size change, the gaze point, the gaze time, and the gaze path which are the gaze information detected by the detector 120 at predetermined time intervals, and uses the gaze information to time Create an intent indicator for the interval. A process of generating the intention indicator by the intention indicator generation unit 131 will be described in detail with reference to FIGS. 4 and 5A to 5C.

4 shows the result of collecting continuous data on the detected pupil size at regular time intervals.

Referring to FIG. 4, the intention indicator generator 131 simplifies the graph by connecting the pupil sizes corresponding to the beginning and the end of each time interval in a straight line. Here, the time interval may be set in advance, and the time interval may be adjusted by the user. In addition, when the intention indicators have similar values in each successive time interval, two or more time intervals may be combined into one time interval.

FIG. 5A illustrates a result of generating an intention index value of a slope of a pupil size change using pupil size data corresponding to each time interval on the x-axis.

Referring to FIG. 5A, the intention index generator 131 may generate an intention index and an intention index value for a slope of a pupil size change from a linear graph corresponding to each time interval. Here, although only the process of generating the intention index for the slope of the pupil size change by using the pupil size and the tendency of the pupil size change will be described with reference to FIGS. 4 and 5A, the values of other intention indices may be generated by the same process. have.

 5B and 5C illustrate intention indicators corresponding to the number of gaze fixation times and gaze fixation times for regions of interest and regions other than the region of interest, respectively, for each time interval on the x-axis. Specifically, the bar graph 0 on the left graph of FIGS. 5B and 5C is a graph showing the number of gaze fixing times and the gaze fixing time for regions other than the ROI. In addition, the bar graphs 1 to 3 are graphs showing the number of gaze fixing times and the gaze fixing times for the first to third regions of interest, respectively.

Here, referring to the left graph of FIG. 5B, it can be seen that the gaze fixing time for regions other than the region of interest is high in the first time interval and the third time interval. In addition, it can be seen that the number of gaze fixings for the ROI is higher than the number of gaze fixing for the regions other than the ROI in the second time interval, the fourth time interval, and the fifth time interval.

Also, referring to the graph on the left side of FIG. 5C, it can be seen that the gaze fixing time for each region is generally equal in the first time interval. It can be seen that the gaze fixing time for the third ROI is the highest in the interval and the fifth time interval.

As shown in the graphs on the right side of FIGS. 5B and 5C, the graphs of the first to third regions of interest may be represented as one graph by calculating an arithmetic mean value. 5B and 5C, graph 0 is a graph for regions other than the ROI, and graph 1 is a graph showing average values of the first ROI to the third ROI. Here, the average value may be represented by not only an arithmetic mean but also an average value such as a geometric mean and a harmonic mean.

The intention indicator conversion unit 132 normalizes the generated intention indicator. In detail, the intention indicator conversion unit 132 may normalize the intention indicator corresponding to the specific time interval by using the intention indicator for one or more time intervals (total time intervals). Here, the total time interval may be preset and changed by the user.

Specifically, the intention indicator generated is normalized using Equation 1 below.

Where dn is the value of the normalized intention indicator, di is the value of the intention indicator corresponding to the time interval to be normalized, and max (d) and min (d) are the maximum and minimum values of the intention indicator within the entire time interval. , n is a constant.

6 is a table showing the result of normalizing the intention index.

The normalization process will be described in detail with reference to FIG. 6. For example, in the normalization process for the pupil size PS, the intention index values of the pupil size are 8.81, 17.52, 14.30, 18.60, and 20.03. Here, if the value of 8.81 is normalized, the maximum value of the pupil size is 20.03 and the minimum value is 8.81 over the entire time interval. And, di is a pupil size value for the time interval to be normalized is 8.81. When n = 50 and substituted in Equation 1, the intention index value of the normalized pupil size becomes zero. When the value of 17.52 is normalized, the minimum value of the maximum value is 20.03 and the minimum value is 8.81, and di is set to 17.52, n = 50, and substituted into Equation 1, and the intention index of the normalized pupil size is 38.81. Although the normalization process has been described using an example in which n = 50, n can be variously set such as 30 100. In this way, normalization of pupil size and other intent indicators for the remaining time intervals is possible.

N is the maximum range to be normalized. n may be preset and changed by the user, and the value dn of the normalized intention indicator may not be greater than n.

7 shows a graph in which the intention index is normalized using Equation 1. Specifically, FIG. 7 shows the result of normalization by setting n to 50 in Equation 1. Referring to FIG.

The intention determiner 133 determines the intention of the user by using the normalized intention indicator. In detail, the intention determiner 133 may combine the normalized five intention indicators corresponding to one or more time intervals, and express the intention as a pentagonal graph, and determine the intention of the user according to the shape of the pentagonal graph.

In this case, the user's intention may be classified into navigation intention and information intention, and the information intention may be further classified into information generation intention, information destruction intention, and information maintenance intention.

Navigational intent is the intention to navigate information without the expectation of a specific location, information, or outcome of an action (eg, browsing an article on a web page). An informational intent is an intention created for the purpose of obtaining information at a specific location. In addition, the information generating intention, the information destruction intention, and the information retention intention mean the occurrence of the information intention, the disappearance of the generated information intention, and the maintenance of the generated information intention, respectively.

8 shows the intention determined according to the pentagonal graph and graph characteristics.

Referring to FIG. 8, PS and FTA may be determined as navigation intentions when the minimum value, FFA and PG values are intermediate values, and FFNA has a maximum value, FFA is the minimum value, PS FFNA FTA is the middle value, and PG is the large value. In the case of, it can be determined as the intention to generate information.If FFNA has the maximum value and FTA and PG have the minimum value, and PS and FFA have the median value, the information is determined to be destroyed and PS, FTA and FFA determine the maximum value. In addition, if PC and FFNA have a small value, it can be judged as information retention intention.

In addition, the intention determiner 133 may determine the intention change time point using the determined user intention. In detail, the intention determiner 133 may determine a user's intention and determine a time point at which the determined intention changes.

Accordingly, the user intention determination device according to the present embodiment may determine the navigation intention, the information generation intention, the information destruction intention and the information maintenance intention which are the user's intentions by using the user's gaze information, Can be judged.

2 is a block diagram illustrating a configuration of an apparatus for determining a user's intention according to an exemplary embodiment. According to FIG. 2, the user intention determination apparatus 200 includes an imaging unit 210, a detection unit 220, and a determination unit 230. The storage unit 240 is included.

The imaging unit 210 photographs a user. In detail, the image capturing unit 110 may image the user with the eyes of the user as the center. The imaging unit 210 corresponds to the imaging unit 110 of the intention determination device 100.

The detector 220 detects gaze information of the user with respect to the image captured by the imaging unit 210. In detail, the detector 220 may detect a pupil size, a pupil size change, a gaze point, a gaze time, and a gaze path on an image captured by the imaging unit 210.

Here, the detection unit 220 may correspond to the detection unit 120 of the intention determination device 100, and detect the gaze information in the same process as the detection unit 120.

Meanwhile, in an exemplary embodiment of the present invention, the imaging unit 110 and the detection unit 120 are described as having separate configurations. However, the image pickup unit 110 and the detection unit 120 may be implemented in one configuration using an eye tracking apparatus.

The determination unit 230 determines the intention of the user by using the detected gaze information of the user. In detail, the determination unit 230 may generate an intention indicator from the detected user's gaze information and determine the intention of the user by using the generated intention indicator. To this end, the determination unit 230 may be composed of an intention indicator generation unit 231, an intention indicator conversion unit 232, and an intention determination unit 233.

The intention indicator generator 231 and the intention indicator converter 232 correspond to the intention indicator generator 131 and the intention indicator converter 132 of the intention determination device 100, respectively, and generate the intention indicator through the same process. Can be normalized.

The intention determiner 233 determines the intention of the user by using the normalized intention indicator. In detail, the intention determination unit 233 may determine the intention of the user by comparing the intention indicator normalized by the intention indicator conversion unit with the intention indicator data stored in the storage unit 240.

In this case, the intention determiner 233 may express the normalized intention indicator as a graph of a pentagon, and compare the pentagon graph and features of the intention indicator that have previously analyzed the intention of the user previously stored in the storage unit 240. The intention of the user can be determined.

The storage unit 240 stores intention indicators that have previously analyzed the intention of the user. In detail, the storage unit 240 may store characteristic data of an intention indicator corresponding to navigation intention, information generation intention, information destruction intention, and information maintenance intention. The stored intention indicator may be an intention indicator to be compared when the intention determiner 233 determines the intention of the user. Here, the user may be given a command corresponding to a specific intention, collect eyeball information when the user performs the command, and generate and normalize an intention indicator to obtain characteristic data of the intention indicator corresponding to each intention.

Accordingly, the user intention determination device according to the present embodiment may determine the navigation intention, the information generation intention, the information destruction intention and the information maintenance intention which are the user's intentions by using the user's eye information. You can judge.

On the other hand, according to an embodiment of the present invention, the storage unit 240 is expressed as embedded in the user intention determination device 200, the storage unit 240 is an external HDD or connected to the user intention determination device 200 or It may be implemented as a memory card, for example, a flash memory (M / S, XD, SD, etc.) or a USB memory.

9 is a flowchart illustrating a method of determining an intention according to an embodiment of the present invention.

According to FIG. 9, the intention determination method first captures a user (S910). The imaging unit 110 may photograph the user with the eyes of the user as the center.

Then, the gaze information is detected using the captured image (S920). In detail, the pupil size, the pupil size change, the gaze point, the gaze time, and the gaze path may be detected on the captured image. The gaze information of the user detects the pupil size on the captured image to determine the pupil size, detects the gaze point of the user based on the detected pupil position, and continuously performs the gaze detection operation to the user. The gaze time, gaze path, and pupil size change can be detected.

The intention indicator is generated from the detected gaze information, and the intention of the user is determined using the generated intention indicator (S930). To this end, the step of determining the user's intention (S930) is to collect the gaze information at a predetermined time interval to generate the intention indicator, normalizing the generated intention indicator and the user's intention using the normalized intention indicator It may be configured to determine.

In detail, the detected gaze information, the pupil size, the pupil size change, the gaze point, the gaze time, and the gaze path are collected at predetermined time intervals, and the intention index for each time interval is generated using these gaze information. Then, using Equation 1, the generated intention index is normalized. In addition, the normalized five intention indicators corresponding to one or more time intervals may be combined and expressed as a pentagonal graph, and the intention of the user may be determined according to the shape of the pentagonal graph.

Meanwhile, in another exemplary embodiment, the determining of the intention of the user (S930) may generate and normalize the intention indicator using the gaze information. In addition, the intention of the user may be determined by comparing the normalized intention indicator with the user's intention and comparing the stored intention indicator data.

Therefore, the user intention determination method according to the present embodiment, by using the user's gaze information to determine the navigation intention, information generation intention, information destruction intention and information maintenance intention which is the user's intention, it is determined to change the user intention Can be.

The user intention determination method as shown in FIG. 9 may be executed by a user intention determination apparatus having the configuration of FIGS. 1 and 2, and may also be executed by a user intention determination apparatus having another configuration.

Although the above has been illustrated and described with respect to the preferred embodiments of the present invention, the present invention is not limited to the above-described specific embodiments, it is usually in the art without departing from the spirit of the invention claimed in the claims. Anyone skilled in the art can make various modifications, as well as such changes are within the scope of the claims.

100: intention determination device 110: imaging unit
120: detection unit 130: determination unit
131: intention indicator generation unit 132: intention indicator conversion unit
133: intention determination unit 200: intention determination device
210: imaging unit 220: detection unit
230: Determination unit 231: Intention indicator generation unit
232: intention indicator conversion unit 233: intention determination unit
240: storage unit

Claims (14)

An imaging unit for photographing a user;
A detector for detecting gaze information by using the captured image; And
And a determination unit which generates an intention indicator from the detected gaze information and determines the intention of the user using the intention indicator. The method of claim 1,
The gaze information,
At least one of a gaze point, a gaze time, a gaze path, a pupil size, a pupil size change,
The intention indicator,
And at least one of a pupil size, a slope of a pupil size change, a gaze fixation time for the ROI, a gaze fixation number for the ROI, and a gaze fixation for an area other than the ROI. The method of claim 2,
The determination unit,
An intention indicator generation unit configured to generate the intention indicator by collecting the eyeline information at predetermined time intervals;
An intention indicator conversion unit which normalizes the generated intention indicator; And
And an intention determiner configured to determine the intention of the user by using the normalized intention indicator. The method of claim 3,
The intention of the user is,
And a navigation intention, an information generation intention, an information destruction intention, and an information maintenance intention. 5. The method of claim 4,
The apparatus may further include a storage configured to store intention index data based on the user's intention analysis.
The intention determination unit,
And an intention indicator of the user by comparing the intention indicator normalized by the intention indicator conversion unit with the stored intention indicator data. The method of claim 3,
The intention indicator conversion unit,
An apparatus for determining intention of a user, wherein the generated intention index is normalized using the following equation:

Where dn is the value of the normalized intention indicator, di is the value of the intention indicator corresponding to the time interval to be normalized, max (d) and min (d) are the maximum and minimum values of the intention indicator within the entire time interval, and n Is a constant. 6. The method according to any one of claims 1 to 6,
The intention determination unit,
And a user's intention change time point is further determined using the determined user intention. Imaging the user;
Detecting gaze information by using the captured image; And
And generating an intention indicator from the detected gaze information and determining the intention of the user by using the intention indicator. 9. The method of claim 8,
The gaze information,
At least one of a gaze point, a gaze time, a gaze path, a pupil size, a pupil size change,
The intention indicator,
And at least one of a pupil size, a slope of a pupil size change, a gaze fixation time with respect to the ROI, a gaze fixation time with respect to the ROI, and a gaze fixation with respect to an area other than the ROI. 10. The method of claim 9,
The determining step,
Generating the intention indicator by collecting the gaze information at predetermined time intervals;
Normalizing the generated intention indicators; And
And determining the intention of the user by using the normalized intention indicator. The method of claim 10,
The intention of the user is,
A user intention determination method, characterized in that the navigation intention, information generation intention, information destruction intention, information maintenance intention. The method of claim 11,
Determining the intention,
And analyzing the pre-analyzed intention indicators and the intentions of the user in the intention indicator conversion unit to compare the stored intention indicator data to determine the intention of the user. The method of claim 10,
The normalizing step is
The user intention determination method characterized in that to normalize the generated intention indicator using the following formula:

Where dn is the value of the normalized intention indicator, di is the value of the intention indicator corresponding to the time interval to be normalized, max (d) and min (d) are the maximum and minimum values of the intention indicator within the entire time interval, and n Is a constant. The method of claim 8 to 13.
Determining the intention,
And determining the user's intention change time point using the determined user intention.

Images