KR20170140655A - Context Aware POMDP based Adaptive Eye Tracking for Robust Human Computer Interaction - Google Patents

Abstract

The present invention relates to an eye tracking device using CA-POMDP for robust human computer interaction. Also, according to the present invention, the eye tracking device includes: an eye tracking module binarizing an eye image by a binary threshold after extracting an eye area when a face image is input and tracking eyes by tracking the position of pupils in the eye image binarized, wherein the eye tracking module also outputs a compensation value on a control action of a system control parameter; an image quality evaluating module providing an image quality label by evaluating image quality by receiving the face image; and a CA-POMDP module defining the system control parameter in a state and forming a word context model by combination of various objects by the each image quality label. The CA-POMDP module also determines a current state based on information on the word context model expressed by using properties of the various objects and property values in accordance with the image quality label provided from the image quality evaluating module. The CA-POMDP module adjusts the system control parameter which can be executed based on the determined current state and renews an action function value based on the compensation value of the control action of the system control parameter. The CA-POMDP selects the optimal control action of the system control parameter in accordance with the renewed action function value.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an eye tracking apparatus using CAPOMDP for robust human computer interaction,

The present invention relates to an eye tracking apparatus using CA-POMDP for robust human computer interaction.

Recently, many eye-tracking processing methods have been proposed for various goals and applications.

Automatic line-of-sight tracking has been used in many application areas because it is strongly connected to the state of human perception, attention and cognition.

Eye-tracking technology is an effective tool for interaction and can be used for human-computer interaction (HCI) and computer-to-human communication that can be used without interference and without hands.

Nevertheless, there are many problems in technology development and many problems such as illumination, viewing angle, and scale, and there is a problem due to the change of operating environment according to various shapes of eyes and jitter of individual.

The most successful eye tracking techniques in the commercial domain require very costly image capture devices (e.g., high cost cameras and lenses) or very limited operations in strongly controlled situations.

The greatest interest in recent line-of-sight technology is being generated in the HCI field for web usability, advertising, smart TV and mobile applications.

Conventional line-of-sight tracking technology, however, has the problem of high cost and limitation for use in low cost industrial applications.

Without control of lighting and control of the situation, more available and more general eye tracking techniques are needed for successful HCI applications, as well as reducing costs and simplifying complex initial user settings.

Published Patent No. 2015-002741 Registration number 10-138775

1. L. J. G. Vazquez, et. al .: Low Cost Human Computer Interface Voluntary Eye Movement as Communication Systems for Disabled People with Limited Movements. PAHCE, pp. 165-170. (2011) 2. J. S. Agustin, et. al .: Low-Cost Gaze Interaction: Ready to Deliver the Promises. CHI 2009 - Spotlight on Works in Progress - Session 2, Boston, MA, USA (2009) 3. A. Sein et al. al .: Eyeing a Real-Time Human-Computer Interface to Assist Those with Motor Disabilities. IEEE Potentials, pp. 19-25 (2008) 4. R. S. Sutton, A. G. Barto: Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, (1998) 5. S. Ross et. al.: A Bayesian Approach for Learning and Planning in Partially Observable Markov Decision Processes. Journal of Machine Learning Research 12, pp.1729-1770. (2011) 6. G. Shani: A survey of point-based POMDP solvers. Auton Agent Multi-Agent Syst, Published online, Springer (2012) 7. Atrash et al. al .: Development and validation of a robust speech interface for improved human-robot interaction. International Journal of Social Robotics, 1, pp. 345-356. (2009) 8. G. Shani, et. an MDP-based recommender system. Journal of Machine Learning Research, 6, pp. 1265-1295. (2005) 9. M. Hauskrecht, H. S. F. Fraser: Planning treatment of ischemic heart disease with partial observable Markov decision processes, Artificial Intelligence in Medicine, 18 (3), pp. 221-244. (2000) 10. J. Hoey, et. al .: Automated hand washing assistance for persons with dementia using video and a partially observable Markov decision process. Computer Vision and Image Understanding, 114 (5), pp. 503-519. (2010) 11. J. D. Williams, S. Young: Partially observable Markov decision processes for spoken dialog systems. Computer Speech & Language, 21 (2), pp. 393-422. (2007) 12. L. P. Kaelbling et. al .: Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101, pp. 99-134. (1998) 13. J. Pineau, et. , "Point-based value iteration: An anytime algorithm for POMDPs," In International Joint Conference on Artificial Intelligence, pp. 1025-1032. (2003) 14. T. Smith and R. G. Simmons, "Point-based POMDP algorithms: Improved analysis and implementation," In Conference on uncertainty in artificial intelligence (UAI), pp. 542-547. (2005) 15. S. Paquet: Distributed Decision-Making and Task Coordination in Dynamic, Uncertain and Real-Time Multiagent Environments. Ph.D. thesis, Laval University (2006) 16. S. Ross, et. al .: Online Planning Algorithms for POMDPs. Journal of Artificial Intelligence Research, 32, pp. 663-704. (2008) 17. M. L. Littman et. al .: Learning policies for scaling up. International Conference on Machine Learning (1995) 18. G. Shani, et. Al: Forward search value iteration for POMDPs. In Joint Conference on Artificial Intelligence, (IJCAI), (2007) 19. T. Smith, and R. Simmons, "Heuristic search value iteration for POMDPs," In Conference on uncertainty in artificial intelligence (UAI), 2003. 20. H. Kurniawati, et. al. "SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces," Proc. Robotics: Science & Systems, 2008. 21. A. Barto, et. al .: Learning to act using realtime dynamic programming. AIJ, Vol. 72, pp. 81-138. (1995) 22. S. Paquet, et. al, "An online POMDP algorithm for complex multiagent environments, AAMAS (2005) 23. J. Wang, et. JA Luo, volume 137 of Advances in Intelligent and Soft Computing, Springer Berlin / Heidelberg, 2012, p. 69-76. 24. B. Bonet and H. Geffner, "Solving POMDPs: RTDP-bel vs. point based algorithms," Proceedings of the 21st International Joint Conference on Artificial Intelligence, IJCAI, 2009, p. 1641-1646. 25. S. Ross and B. Chaib-draa et. al., "Aems: An anytime online search algorithm for approximate policy refinement in large POMDPs," Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007, pp. 2592-2598. 26. C. A. P. Chanel, "POMDP-based online target detection and recognition for autonomous UAVs," ECAI, 2012, pp. 955-960. 27. S. Paquet, et. al., "Hybrid POMDP algorithms," Proceedings of the Workshop on Multi-Agent Sequential Decision Making in Uncertain Domains, 2006. 28. S. Ross, et. al., "Theoretical analysis of heuristic search methods for online POMDPs," Advances in Neural Information Processing Systems, 2008. 29. H. Geffner and B. Bonet, "Solving large POMDPs using real time dynamic programming," in Working Notes Fall AAAI Symposium on POMDPs, 1998, pp. 61-68. 30. R. Washington, "BI-POMDP: Bounded, Incremental Partially-Observable Markov-Model Planning," Proceedings of the 4th European Conference on Planning, volume 1348 of Lecture Notes in Computer Science, Toulouse, France, Springer, 1997, pp. . 440-451. 31. T. Wang, et. , "A Bayesian Sparse Sampling for On-line Reward Optimization," ICML'05: Proceedings of the 22nd International Conference on Machine Learning, New York, NY, USA: ACM, 2005, pp. 956-963. 32. J. Asmuth, et. "A Bayesian sampling approach to exploration in reinforcement learning," Conference on Uncertainty in Artificial Intelligence (2009) 33. F. Doshi-Velez, et. al., "Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs," Artificial Intelligence, 2012, pp. 187-188. 34. M. Strens, "A Bayesian framework for reinforcement learning," International Conference on Machine Learning, 2000. 35. R. Jaulmes, "Active Learning in Markov Decision Processes," ECML, 2005. 36. S. Ross, et. al., "Bayes-adaptive POMDPs. In Neural Information Processing Systems," 2008. 37. F. Doshi-Velez, "The infinite partial observable Markov decision process," Advances in Neural Information Processing Systems, 22, pp. 477-485 (2009) 38. P. Poupart and N. Vlassis, "Model-based Bayesian reinforcement learning in partially observable domains," ISAIM, 2008. 39. A. Krause, et. al., "Trading off prediction accuracy and power consumption for context-aware wearable computing, " Proc. 9th IEEE Int. Symp. Wearable Computers, Oct. 2005, pp. 20-26. 40. L. Au, et. al, "Episodic sampling: Towards energy-efficient patient monitoring with wearable sensors," Proc. IEEE Annu. Int. Conf. Engineering in Medicine and Biology Society, Sep. 2009, pp. 6901-6905. 41. B. Bhanu and J. Peng, "Adaptive integrated image segmentation and object recognition," IEEE Trans. on Systems, Man, and Cybernetics-PART C: Applications and Reviews, Vol. 30, No. 4, Nov. 2000, pp. 427-441. 42. R. D. Smallwood and E. J. Sondik. "Optimal control of partially observable Markov processes over a finite horizon," Operations Research, 21 (5): Sep / Oct, 1973, pp. 1071-1088. 43. K. Murphy, "Dynamic Bayesian Networks: Representation, Inference and Learning," PhD Thesis, UC Berkeley, 2002. 44. M. Abdel-Mottaleb and M. H. Mahoor, "Application notes - Algorithms for assessing the quality of facial images ", IEEE Computational Intelligence Magazine, Volume 2, Issue 2, May 2007, pp. 10-17. 45. H. Sellahewa and S. A. Jassim, "Image-quality-based adaptive face recognition," IEEE Trans. on Instrumentation and Measurement, Vol. 59, No. 4, April, 2010, pp. 805-813. 46. Q. Li and Z. Wang, "Reduced-reference image quality assessment using divisive normalization-based image representation," IEEE Journal of Selected Topic in Signal Processing, Vol. 3, No. 2, Apr. 2009, pp. 202-211. 47. C. J. C. H. Watkins, Learning from delayed rewards, Ph.D. Thesis, Cambridge University, Cambridge, England, 1989. 48. Z., Wang, A. C., Bovik: A universal image quality index. IEEE Signal Process. Lett., Vol. 9, No. 3, pp. 81-84. (2002) 49. P. K., Rhee, et. al .: Context-aware evolvable system framework for identifying systems. Part II, pp. 270-283. KES (2005) 50. E.J., Koh, M.Y., Nam, P.K., Rhee: A context-driven Bayesian classification method for eye location. ICANNGA, 2, pp.517-524 (2007) 51. R. Dearden, N. Friedman, D. Andre: Model based Bayesian exploration. Proceedings UAI (1999) 52. P. K. Rhee, et. al .: Pupil location and movement measurement for efficient emotional sensibility analysis. ISSPIT (2011) 53. Z. Kadal, et. al .: Tracking-Learning-Detection. IEEE Trans on PAMI, Vol. 34, No. 7, pp. 1409-1422. (2012) 54. Y. Shen, et al .: Evolutionary adaptive eye tracking for low-cost human computer interaction applications. Journal of Electronic Imaging, 22.1, 013031-013031 (2013)

The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a robust human-computer interaction capable of optimizing system control parameters using CA-POMDP instead of using a high- CA-POMDP. ≪ / RTI >

According to one aspect of the present invention, an eye region is extracted, a binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binarized binary binarized binarized binarized binarized binary binarized binary binarized binary binarized binary binarized An eye tracking module for outputting a compensation value for an action; An image quality evaluation module that receives a face image and evaluates image quality to provide an image quality label; And the system control parameter is defined as a state, a world context model is configured by a combination of various objects according to image quality labels, and a world context model expressed by attribute and attribute values of various objects according to an image quality label provided by the image quality evaluation module Determines an existing state based on information on the system control parameter adjustment operation, performs an executable system control parameter adjustment operation based on the determined current state, updates the behavior function value based on the compensation value of the system control parameter adjustment operation, And a CA-POMDP module for selecting an optimal system control parameter adjustment action according to a function value.

Further, the system control parameter of an aspect of the present invention includes a binarization threshold, an angle parameter of the partial Huff transform, and a noise parameter of the Kalman filter.

According to an aspect of the present invention, the image quality evaluation module includes an image quality learning unit that learns an image quality index from an image quality index prototype using the collected training image; And an image quality labeling unit for extracting a pixel of interest from the scanning window by dividing the region of interest into rectangular patches, loading the prototype of the image quality learning unit, and calculating the image quality label.

The image quality label portion according to an aspect of the present invention is characterized by calculating a simplified image quality label in consideration of illumination change in calculating an image quality label.

In addition, the CA-POMDP module according to an aspect of the present invention defines the system control parameter as a state, and determines the current state of each state variable based on information on a world context model represented by attributes and attribute values of various objects And performing an executable action based on the updated behavior function value; A world context modeling unit for constructing a world context model based on a combination of various objects for each image quality label; And a real-time Q-learning unit for updating the action function value based on the compensation value of the action for the current state and storing the updated action function value.

Further, the real-time Q-learning unit according to an aspect of the present invention is characterized by acquiring an action, calculating an internal compensation, and observing a new internal state based on the immediate compensation to update an action function value of the world context model .

The present invention can ensure performance by optimizing system control parameters using CA-POMDP instead of using expensive image capture devices or very limited circumstances.

1 is a block diagram of an eye tracking apparatus using POMDP for robust human computer interaction according to an embodiment of the present invention.
FIG. 2 is a diagram showing partial circles representing the outer boundary of the iris according to the degree of eye shedding.
3 is a view for explaining a process for labeling the image quality in the image quality label portion.
Referring to FIG. 4, a comparison between a general POMDP and a CA-POMDP of the present invention is shown.
5 is a flowchart for explaining the operation of the eye tracking apparatus of FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

First, the terminology used in the present application is used only to describe a specific embodiment, and is not intended to limit the present invention, and the singular expressions may include plural expressions unless the context clearly indicates otherwise. Also, in this application, the terms "comprise", "having", and the like are intended to specify that there are stated features, integers, steps, operations, elements, parts or combinations thereof, But do not preclude the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a block diagram of an eye tracking apparatus using POMDP for robust human computer interaction according to an embodiment of the present invention.

Referring to FIG. 1, an eye tracking apparatus using POMDP for robust human computer interaction according to an embodiment of the present invention includes an eye tracking module 100, an image quality evaluation module 200, and a CA-POMDP module 300 .

The eye tracking module 100 binarizes the eye image at the binarization threshold value after extracting the eye region at the time of inputting the face image, and tracks the pupil position in the binarized eye image to perform eye tracking.

The performance of the eye tracking method performed in such an eye tracking module 100 is highly dependent on the selection of the algorithm and the thresholds and parameters (collectively referred to as system control parameters) associated with the selected algorithm in each step.

The eye tracking method can be best structured in the eye tracking algorithm structure and in the critical dynamic control problem of determining the thresholds and parameters associated with the algorithm structure.

The threshold values and parameters are greatly influenced by changes in the environment in consideration of the fact that the performance of the eye tracking method is deteriorated due to changes in image quality due to illumination, pose, and change in shape of the image. In the present invention, system control parameters of the eye tracking module 100 are adaptively controlled by image-quality labels, observations, and compensation values (rewards).

The eye region can be located using the already well known face position and eye region location method, and eye tracking can also be processed based on the well known eye tracking method.

The center of the initial eye can be estimated by preprocessing, feature extraction and partial Hough transformation.

Preprocessing may be performed using an adaptively selected algorithmic infrastructure and a threshold associated with the selected algorithmic infrastructure, and feature extraction may be performed to generate an outline or edge image.

The contour or edge image may be processed by a partial Huff transform using the adjusted whistle parameter of the iris boundary to determine the optimal tracking point.

Finally, a Kalman filter can be applied and the next eye center can be predicted. The above steps can be repeated until the eye tracking is finished.

The eye tracking control space can be determined based on previous knowledge or experience, for example, considering possible algorithmic structures and thresholds or a variety of parameters, taking into account tradeoffs between tracking accuracy and runtime constraints.

In such an eye tracking method, the preprocessing step may include histogram smoothing, Retinex using one threshold value, and end-in contrast stretching using two parameters.

For example, possible algorithmic structures of the preprocessing step may be retinex or end-in-contrast stretch using histogram equalization (HE), retinex, histogram smoothing in a serial combination.

Next, the feature extraction step is performed in a parallel combination using binarization and canny or (4) OR operation in parallel combination using (1) binarization, (2) canny edge search algorithm, (3) AND operation, And can be. Here, in a series combination, binarization and contouring can be represented by binarization for simplicity.

Feature extraction may be selectively performed using one of the algorithm structures of (1) to (4) described above. The binarization may have one threshold (X _BN ), and the Kanny algorithm may have two thresholds for edge detection. Here,

One of the two thresholds for the algorithm can be related to edge linking and the other can be related to finding the initial segment of the strong edge.

On the other hand, in the eye tracking method, the iris boundary can be approximated to a circle because it aims at providing real-time performance using a low resolution image.

The algorithm can detect the center of the eye (iris) by calculating the outer boundary of the iris image. At this time, a modified Hough transform method (referred to as partial Hub transform) may be used to extract characteristics less affected by noise and eyelid occlusion.

Instead of a complete circle, the two parts of the circle (partial circles) can be connected to minimize the effect of occlusion of the upper and lower eyelids.

The four-dimensional control space for center tracking of the eye may include an angle value representing an iris boundary that is a partial interest (e.g., partial circles).

FIG. 2 is a diagram showing partial circles representing the outer boundary of the iris according to the degree of eye shedding. Fig. 2 shows a state in which, instead of all the iris boundaries, the angles of 陸₁ and 陸₂ and the iris outer boundaries between angles of 陸₃ and 陸₄ are considered in order to avoid the eyelid occlusion effect.

The partial Huff transformation algorithm for circle fitting can be described as follows. First, the iris outer circle can be expressed by the following equation (1).

(1)

(xa ₀ ) ² + (yb ₀ ) ² = r ²

Here, (a ₀ , b ₀ ) may mean the center of the eye (the center of the outer boundary of the iris), and r may mean the radius of the circle. At this time, the following two stage algorithms can be used. The first stage finds the center of the eye circle, and the second stage predicts the normal direction of the outer boundary point of the iris in the partial circles intersecting the eye center. (x, y) is a gradient direction point at the outer boundary of the iris. (x, y, X _PTH ) The mapping from the triple to the center parameter (a, b) is straight. Here, X _PTH may mean an angle in an oblique direction. Many intersections of these straight lines can identify the coordinates at the center of the eye. (x, y, X _PTH ) and (a, b) can be given by the following equation (2).

(2)

X _PTH = arctan ((yb) / (xa))

In the partial Huff transformation algorithm, the range of the iris outer boundary can be limited by four angular parameters X _PTH1 , X _PTH2 , X _PTH3 , and X _PTH4 to avoid the effect of the eyelid occlusion as in the example of Equation (3).

(3)

X _PTH = [X _PTH1 , X _PTH2 ] U [X _PTH2 , X _PTH4 ]

On the other hand, a Kalman filter (KF) can be an iterative approach to the linear filter problem of discontinuity by estimation of the state process minimizing the squared error.

The Kalman filter can be used to calculate an approximation of tracking centered eye as a Bayesian tracking problem.

Suppose that the motion at the center of the eye has a constant velocity F _t , a covariance matrix of the state transition model, and a process noise model, which can be determined by the following equations (4) and (5)

(4)

(5)

Where DELTA T is the time period between adjacent frames (usually very short), and q may be a parameter associated with process noise, respectively. Assuming that H _t is a constant, the matrices of the measurement model and the measured noise covariance can be determined as shown in the following equations (6) and (7), respectively.

(6)

(7)

Here, r may mean a parameter related to the measurement noise. The Kalman filter approach often has difficulties in estimating the noise covariance matrix Q _t-1 and R _t . In many applications, Q _t-1 and R _t of the noise covariance can be quickly stabilized and invariant. The parameters r and q can be pre-computed by operating the filter off-line or by determining the state value. However, noise is not invariant in eye tracking due to uncertainty in the dynamically changing environment. The parameters r and q may be adjusted to achieve optimal performance in an eye tracking scheme according to embodiments of the present invention.

Here, the system control parameter is represented by X _KF1 for the unification in the notation, and q is represented by X _KF2 .

In conclusion, the system control parameter space X ^? Is represented by the following equation (8) with parameters as an inner set.

(8)

Here, each parameter is included in the corresponding discrete area.

The eye tracking module 100 may perform reinforcement learning of the real-time Q-learning unit 330 when the system control parameter is defined as a state s by the CA-POMDP module 300 and the system control parameter is changed by the action a Evaluates the execution result of each action, determines the compensation value, and provides the compensation value to the real-time Q-learning unit 330. At this time, the compensation value is also provided to the world context modeling unit 320 to update the world context model.

On the other hand, given an external environment, an optimal algorithm structure with associated threshold values and parameters may be determined according to the perceived image context.

In general, a context can be any information that affects system performance of interest. The image context may be influenced by the direction, brightness, contrast and spectral composition of the illumination. The concept of image context using image quality analysis technique can improve system performance.

Recently, image quality analysis methods have been successfully applied to various applications such as image storage, compression, communication, display, segmentation and recognition, and adaptive

Can be used to determine the preprocessing for adaptive light normalization using the intermediate parameter.

)로 CA-POMDP 모듈(300)로 알려준다.For this, the image quality evaluation module 200 analyzes the image quality and outputs the analyzed result to the image quality label (

) To the CA-POMDP module 300.

For this, the image quality evaluation module 200 includes an image quality label unit 210 and an image quality learning unit 220.

Here, the image quality label portion 210 extracts a region of interest (ROI) 210-1 as shown in FIG. 3 when the image frame I is input.

The grid Θ is made up of n grid cells and is thus defined as Θ = {ω ₁ , ..., ω _n }, and the ROI is defined as I (Θ).

Then, the image quality label portion 210 divides the region of interest into a rectangular patch 210-2 and creates an index of the rectangular patch 210-2.

I (Θ) = I (ω ₁ ) ∪ ... ∪ I (ω _n ) when the square patch is I (ω) and each ω is the center grid cell.

^j _i를 Λ⁺ _i의 j번째 라벨이라 하면, Λ_i={

¹ _i,...

^l _i}이 된다. 이에 따라, 관심 영역의 화질 라벨 공간은 Φ(I(Θ))∈

로 정의될때

=Λ⁺ _ix…xΛ⁺ _n 에 속한다.Let Λ _i (i = 1, ..., n) be the label set of n grid cells, and let Λ ⁺ _i = {0} ∪Λ _i (where {0} box),

^{Let j} _{i be} the jth label of Λ ⁺ _i , then Λ _i = {

¹ _i , ...

^l _i }. Accordingly, the image quality label space of the region of interest is Φ (I (Θ)) ∈

When defined as

= Λ ⁺ _i x ... belongs to xΛ ⁺ _n .

Next, the image quality labeling unit 210 loads the pixels of the prototype image, extracts the pixels of the W scanning window of the rectangular patch, calculates the image quality by comparing the pixels of the scanning window with the pixels of the prototype image, And outputs it.

Here, the W scanning window is performed from left to right and top to bottom of the rectangular patch.

^j _i를 갖는 클러스터 프로로타입(cluster prototype)의 픽셀은 z^k(

^j)={z^k ₁,...,z^k _m}으로 표현된다. 그리고, 스캐닝 윈도우 k의 테스트 픽셀의 화질값은 x^k={x^k ₁,...,x^k _m}이라 하며, W 스캐닝 윈도우에서 예측값 Δ(

^j)는 아래 수학식9와 10으로 정해진다.In detail, when the scanning window is represented by k,

The pixels of the cluster prototype with ^j _i are z ^k (

^j = {z ^k ₁ , ..., z ^k _m }. Then, the image quality value of the test pixel of the scanning window k is x ^k = {x ^k ₁ , ..., x ^k _m }, and the predicted value Δ (

^j ) is defined by the following equations (9) and (10).

 (9)

,

,

,

및

이다.From here,

,

,

,

And

to be.

(10)

^j)∈{-1, 1}이며, Δ(

^j)의 최대값은 모든 i=1,...,m일때 z^k(

^j)=x^k 즉, z^k _i=x^k _i,이면 1.0이다.Here,? (

^j ) ∈ {-1,1} and Δ (

The maximum value of ^j is z ^k (i = 1, ..., m)

^j ) = x ^k, that is, z ^k _i = x ^k _i , and 1.0.

In the image patch I (?), The image quality label is determined by the following equation (11).

(11)

In the region of interest I (?), The image quality label is obtained by the following equation (12).

(12)

(I(Θ))=

^* ₁

^* ₂…

^* _n

(I (?)) =

^* ₁

^* ₂ ...

^* _n

^j _i의 화질값은 x^k는 아래 수학식 13과 같이 상관 관계, 조명 왜곡 및 컨트라스트 왜곡의 3개부분으로 나누어진다.each

quality value of the ^j x ^k _i is is divided into three parts of the correlation, lighting contrast distortion and distortion as shown in Equation 13 below.

(13)

Here, when the illumination effect is most important in consideration of the main factors of the performance of the eye tracking and the real-time constraints, the image quality value is simplified by the following equation (14).

(14)

Accordingly, the image quality of the ROI is represented by the following equation (15).

(15)

이다.From here,

to be.

Consequently, the image quality label unit 210 calculates the image quality by reflecting the illumination change.

Next, the image quality learning unit 220 learns the image quality index from the image quality index prototype using the collected training image.

The CA-POMDP module 300 includes a control unit 310, a world context modeling unit 320, and a real-time Q-learning unit 330.

The CA-POMDP module 300 defines a system control parameter as a state s (the controller performs this role).

The CA-POMDP module 300 constructs a world context model by combining various objects according to image quality labels (a world context modeling unit), and based on the information on the world context model represented by the attributes and attribute values of various objects The current state is determined by the control unit by determining the Boolean value of each state variable.

The current state determined by the CA-POMDP module 300 is an important basis for the behavior and the Q-learning.

The CA-POMDP module 300 updates the Q value, which is a value function of the action, based on the compensation value of the action for the current state, stores the updated Q value (real-time Q-learning unit performs) And performs an executable action based on the value (performed by the control unit). The eye tracking module 100 receives the compensation value and updates the Q value, and selects an action having the best Q value (the controller and the real- Added).

In general, the partially observable Markov decision process (POMDP) is a general decision framework for partially observable problems.

It is an appropriate model for real problems in which uncertainty exists, assumes an accurate model for the real world, and finds the optimal action policy in a given model.

POMDP basically has the following components {S, A, O, T, Ω, R, γ}, taking into account the uncertainties that can be observed in part.

Where S is a set of states s, A is a set of actions a that the controller can take, T is a variance probability distribution of state s, R is the probability of compensation r Where O is the set of observations o and Ω is the probability distribution of the observations to the actual state. γ has a real value between 0 and 1 at a discount rate.

(S, a ', s') = P (s '| s, a) where s is the current state and s' '| S, o).

In the MDP, it is based on a specific state. However, since it becomes impossible in POMDP, it is necessary to define a belief space that allows the state to be determined stochastically using observation information in part. Due to this trusting space, POMDP resolution becomes possible with MDP resolution.

Let b _t (s) be the distribution probability of state s at time t, and b _t (s) = P (s _t | h) is determined by agent history. Here, h = ((a ₀ , o ₁ , r ₁ ), (a ₁ , o ₂ , r ₂ ), ..., (a _t-1 , o _t , r _t ).

The problem of obtaining the optimal policy in POMDP is the same as obtaining the four tuples [B, A, τ, R _B ] in the trust MDP.

The trust state b is updated to the next trust state b 'using the state evaluator τ, and b' = τ (b, a, o). The status update is determined by the Bayesian rule [5].

(16)

The compensation function is defined by the following equation (17).

(17)

Here, V ^* is the value function of the policy π. If V ^* is greater than or equal to the value function of all other policies that can be defined, then the policy is called the optimal policy and denoted by π *.

As a result, the process of obtaining the solution in the MDP is a process of obtaining the optimal policy (π *).

The optimal policy is defined by the Bellman equation of Equation 18 below.

(18)

Here, Q (b, a) is the behavior value function, and the optimal behavior value function is Q ^* (b, a).

The optimal behavior value function can be obtained by the following equation (19).

(19)

이다.On the other hand,

to be.

The state space in the CA-POMDP is defined by the system control space which depends on the context space. Here, the context may be some information that affects the system performance of the eye tracking module 100. The image context may be the direction, brightness, contrast and spectral composition of the illumination.

The set of actions A represents the adaptation of system control parameters in the state space. The set of observations O indicates the tracking performance and the state of image quality in which the object appears.

 The actor can not directly know the current state of the environment, but can only obtain observations. Therefore, the actor maintains a probability distribution of the current environmental condition through all the actions to date and corresponding observations. This is called the belief state, and this space is called the trust space B. Thus, the actor detects uncertainty based on the trust space.

However, in a typical POMDP, real-time resolution is not provided due to the surge in the trust space B in the state number | S |.

The probability distributions of state transitions, observations, and compensation should be combined with limited world context models to avoid such processing difficulties.

The world context models represent an understanding of the area of uncertainty of eye tracking, and an understanding of the area is used for time range searching.

Unlike normal POMDP, the state is observed in a single model, and the transition behavior can be focused on the local state dependent on the current world context model.

It is clear that the proposed CA-POMDP is less accurate due to additional simplification but has a computational gain that meets real-time limitations.

The CA-POMDP is defined as eight elements: S, A, O, T, Ω, R, γ, and Φ.

Here,? Represents the context space, and the context space may be the direction, brightness, viewing angle, and scale of illumination affecting the eye image.

The context space indicates that the image quality is measured in consideration of the environment in which the image is acquired. For example, the context space includes several variables, where Φ ₁ is the direction of the illumination, Φ ₂ is the brightness, Φ ₃ is the viewing angle, and Φ ₄ is the scale of the object.

This is measured elements, such as discrete values, the context state φ is expressed in terms of the sorted values for each variable, i.e., _{φ = {Φ 1 = φ 1} , ..., φ M = φ M}.

'에 의해 추정된다.The state space S is defined as a system control parameter in the eye tracking space, and is affected by changes in the environment. Since the correct context is impossible, the state space S is processed in consideration of the partial observability. Since the image quality state can not be substantially measured,

'.

The state space S is a set of system control spaces X that depend on the context space?. That is, S = (X | Φ) and, from here with _{X = {X 1, ...,} X L} a, {Φ = Φ _1, ..., Φ _M}, the system control that depends on the context space It contains a set of random variables of space.

The random variable X _i means a threshold or parameter in eye tracking. Alignment of the system control parameters of the individual random variable x = denotes a _{{X 1 = x 1, ...} , X L = x L}.

The state s in the state space S is expressed by the following equation (20).

(20)

Where s? S, s is the scalar representing the control part, and D _i is the discretized area of the system control parameter X _i .

The control unit 310 can change the system control parameter in the eye tracking apparatus by an action, but can not change the image quality in the context space.

Image quality can not actually be measured completely and is only partially and limitedly available. Also, state s is partially observable due to uncertainty in the context space.

The action set a is defined in action space A. The action a _i is defined for each random variable X _i in the system control space X that depends on the context space Φ, and this relationship is expressed by Equation (21).

(21)

Here, action a belongs to action space A, and each X _i The -d _i action of the control, which represents the drop of d _i units horizontally or represents saturation, rise, 0, + d _i It is a Scala that represents an action.

For the eye tracking algorithm, the following states are represented by the transition probability function T modeling the uncertainty.

는 다은 상태 s'의 확률 분포로 아래 수학식 22와 같이 정의된다.The function T is the transition probability from the current state to the next state s' given the current state s, the context state φ∈Φ, and the behavior a. Probability function of state transition

Is the probability distribution of the state s' and is defined as: < EMI ID = 22.0 >

(22)

이다. From here,

to be.

를 사용해서 관찰 공간에 걸쳐 아래 수학식 23로 정의된다.Observation space O is a distribution space that spans snow tracing performance and image quality, and is expressed as O = {τ, φ}. Where? Is the tracking accuracy and? Is the image quality measurement. Observation o is the observation probability function

Lt; RTI ID = 0.0 > (23) < / RTI >

(23)

Where o _τ and o _φ denote tracking accuracy and image quality measurements.

로 표현되며, 보상 r은 r=R(s,a)로 표현된다. 보상 함수는 상태 s에서 행위 a를 취했을 때 얻는 보상을 나타내며, 보상 함수는 추적 성능에 근거해서 측정되며, 문턱값보다 추적 신뢰가 크면 높은 점수가 보상되고, 이와 달라지면 적은 점수가 보상된다. γ는 할인율로 0과 1사이의 실수 값을 가진다.In addition, compensation is given immediately by action,

, And the compensation r is expressed as r = R (s, a). The compensation function represents the compensation obtained when the action a is taken in the state s. The compensation function is measured based on the tracking performance. If the tracking confidence is higher than the threshold value, the higher score is compensated. γ has a real value between 0 and 1 at a discount rate.

In order to overcome the enormous computational demands of the CA-POMDP, we simplify the flexible world context models and online learning of each world context model.

The world context models are pre-compiled by the world context modeling unit 320 over the combined space of transition, observation, and compensation distributions in on-line learning.

On-line learning is formulated by dynamic adaptation of the system control parameters of the eye tracking algorithm due to insufficient information and real-time limitations and solved by a real-time Q-learning approach [16].

Policy learning is divided into online policy learning and offline policy learning [10,11,12]. Offline learning requires enormous computation to determine the behavior selection policy. This is sometimes advantageous in real-time tasks, but is difficult to apply in a new environment that occurs very frequently at run time.

Online access is highly adaptable, requiring little computation in real time. There is an advantage in calculating the off-line learning for the world context model by the world context modeling unit 320 and for on-line learning for real-time local policy decision by the real-time Q-learning unit 330.

The CA-POMDP selects the best state for the probability distributions T and R during execution through the world context modeling unit 320 and constructs an offline world context model.

Since the world context model is precompiled without real-time performance degradation, on-line learning explores limited trust states to discover good policies.

As the maximum value is calculated in the current world context model, the on-line learning is performed and the policy is locally determined.

The world context model is maintained for several steps as long as sufficient action occurs for longer than an individual action. Keeping the same world context model for several time intervals has the advantage of matching the search strategy. Online policy output needs to consider a balance between system accuracy and real-time limits.

The CA-POMDP performs offline learning in the world context model (performed by the world context modeling unit 320), and performs on-line learning in a state dependent on the current world context model (real-time Q-learning unit 330) Lt; / RTI >

Since the cardinality of the S, A, O, and R sets is limited, CA-POMDP allows sufficient limits in metastasis, observation, and compensation.

The world context modeling unit 320 precompiles the world context model at the coupling space CM = Tx? XxR for off-line learning. A similar approach is found in Ref. [17], which does not take into account the context space that reflects image quality labels.

Referring to FIG. 4, a comparison between a general POMDP and a CA-POMDP of the present invention is shown.

While the general POMDP is controlled by a single world context model, the transition behavior in CA-POMDP is affected by the current state and the world context model.

CA-POMDP consists of multiple world context models and has a local trust space under the MDP. CA-POMDP is based on the Markovian nature of partial observations of environment and interaction.

The random variables A, O, and R are associated with the probability distributions T, < RTI ID = 0.0 > O ₁ , O ₂ , ..., O _n are random variables of the observation probability distribution Ω. _{A 0, A 1, ...,} A N-1 are random constants for the transition probability distribution _{T, R 1, R 2,} ..., R N is a random constant to compensate the probability distribution R.

Action _{A 0 = a 0, A 1} = a 1, ..., A N-1 = each observation with respect to _{a N -1 O 1 = o 1} , O 2 = o 2, ..., O N = O _We can obtain the compensation R ₁ = r ₁ , R ₂ = r ₂ , ..., R _N = r _N.

로 나타내어진다. 월드 컨텍스트 모델과 상태는 부분적으로 관찰가능하며, 결합 신뢰는 아래 수학식 24로 표현된다.The collected training data for the world context model c for the CM space is h = {(a ₀ , o ₁ , r ₁ ), (a ₁ , o ₂ , r ₂ ), ..., (a _n- o _n , r _n )}. Total training data for all world context models

Lt; / RTI > The world context model and state are partially observable, and the joint confidence is expressed by Equation 24 below.

(24)

Where b _c (s) = b (s | c, h) and is determined by the trust update. The behavior is selected by b _CM (c | φ) for the world context model that maximizes the expected discount compensation over the local execution time.

b _CM (c) represents trust over the world context model c, and is simplified to Equation 25 below [17].

(25)

는 화질 라벨 φ에서 월드 컨텍스트 모델의 델타 함수를 나타내며, w_i는 c_i와 관련된 가중치를 나타낸다. 컨텍스트 상태 φ는 화질 라벨 ψ의 간략화이며, 닫힌 형식이 아니다. 상기 월드 컨텍스트 모델링부(320)에 의해 구성된 월드 컨텍스트 모델의 신뢰는 영역 행위, 관찰 및 보상의 히스토리에 의해 결정된다. 전이, 관찰 및 보상 모델들이 이산 다중으로, 분포[19}를 학습하기 위해 디클레(Dirichlet) 정리와 베이시언(bayesian) 방법을 사용한다.From here,

Represents the delta function of the world context model in the image quality label ϕ, and w _i represents a weight value associated with c _i . The context state? Is a simplification of the picture quality label?, Not a closed form. The trust of the world context model configured by the world context modeling unit 320 is determined by the history of area behavior, observation and compensation. Transition, observation, and compensation models use the Dirichlet theorem and the Bayesian method to learn the distribution [19] in discrete multiplicities.

If the new random variable is (a, o, r), the probability is calculated by the current world context model, and the next context model is determined by the MAP clustering rule. The probability is determined by the world context model c by the following equation (26).

(26)

Here, p ((a, o, r) | c, h) represents the condition density of the current world context model and the theorem probability P (c) is the current world context model trust b _CM (c). And h and H denote the training set of the respective world context model and the entire world context model, respectively. The next world context model for the new random variable is determined using Equation 27 using the MAP rule.

(27)

Maintaining the same world context model for some time improves the consistency of the search strategy. The world context model is modified based on sufficient synchronization of behavior, observation, and compensation over a period of time steps that are slightly less than individual time steps.

Meanwhile, the world context modeling unit 320 models world context models through offline learning.

Then, the real-time Q-learning unit 330 calculates an optimal policy based on the world context model.

Thus, CA-POMDP determines the optimal policy based on the current world context model. This allows the controller 310 to take into account the local time range of the world context model and to focus on a small set of trust states based on on-line learning.

Although building the world context model off-line reduces the overhead of many computation times in CA-POMDP, but still time complexity exponentially increases, that is, O (| A∥O |) ^K (where k Is the depth of the recursive computation, and "O" is the big O notation). The reduction of the search space of CA-POMDP can not be solved by using a new online approach because the search depth is not satisfied. Real-time constraints on eye tracking require a simple and fast algorithm, and real-time Q-learning has dynamic programs asynchronously with incomplete information rather than solving in direct trust.

The approximate optimal value function V ^* of CA-POMDP is simplified to solve all uncertainties at the next time step, that is, all uncertainties beyond the current confidence after the next action.

The empirical approach of QMDP is that the behavior with compensation from all states with a very long time is weighted by the trust state selected at each time step and is expressed by Equation 28 below.

(28)

Here, under MDP, V ^* _MDP (S) represents the optimal value function of state s, and the trust state b _c is calculated using equation (16).

The simplified optimal values are linear under the MDP and can be effectively solved. The execution time under the MDP for the world context model c is defined by Equation 29 below.

(29)

,

이다.From here,

,

to be.

Under the MDP for each world context model, each local search space reduces the MDP resolution to a small time span. There is still incomplete information under the MDP.

The problem of removing eye tracking modules and tracking the behavior while achieving optimal performance is a sensitive issue. This kind of contradiction requires a lot of searching and an uncommon mechanism.

MDP solves this problem using Bayesian or non-Bayesian methods. Real-time Q-learning is not a direct Bayesian method.

For each world context model by the real-time Q-learning unit 330, the Q-learning maintains the Q-table, and instead of the evaluation function, the Q- Lt; / RTI >

이다.The Q-table of the world context model c is initialized by the collected training data. Q ^c _k (s _i , a _i ) represents the evaluation of Q ^{c *} (s _i , a _i ) at stage k,

to be.

로 표현되도록 하면, Q-값은 스테이지 k에서 업데이트된다. (s_i,a_i)∈

에 대하여, Q-값은 다음 수학식 30에 따라 스테이지 k+1에서 백업된다.A state-action pair that can be realized at each stage of k = 0, 1, ..., n

, The Q-value is updated at stage k. (s _i , a _i ) ∈

, The Q-value is backed up at stage k + 1 according to the following equation (30).

 (30)

Here, s _i 'is the next state, that is _{s i' = T (s i} , a i) and, α (s _i, a _i) is the current state-of-a learning rate parameter in haewi pair, r _i is a compensation function to be. Under MDP, optimal policy can be determined without constructing an explicit model.

In real-time applications, the controller 310, and observing the state s _t at each time step t, and calculates the optimum Q- value in time steps of any proceeding after the initialization state.

을 받는다.The controller 310 selects the state a _t? A _c (s _t ) and performs it, and when the state of the eye tracking module 200 transitions to the next state,

.

At this time, the Q-value of the world context model c is backed up according to the following equation (31) at time t + 1.

(31)

Where α (s _t , a _t ) is the learning rate parameter in the current state-behavior pair at time t and r _t is the compensation function.

는 (s_t, u_t)가 된다.The real-time Q-learning unit 330 is a proposed case of offline Q-learning, in which a Q-value is backed up at each step t,

(S _t , u _t ).

The real-time Q-learning unit 330 diverges in the state Q ^* in which the offline learning is required to be diverted.

In the optimal policy, Q-learning is provided by Watkins [47] where each feasible action is backed up in each state in which an infinite number of time steps are repeated and when the time step t decreases to the learning rate α in an incremental manner.

The real-time Q-learning unit 330 is advantageous because less backup is required in real-time applications than asynchronous DP backup.

If there is an n state and the largest number of operations allowed in any state is m, then the backup of the Q-learning is O (m) and less than O (mn) of the DP backup.

In the present invention, additional computation is required when the establishment of the local policy is not optimized, i.e., not meeting the successful eye tracking criteria. In that case, the next feasible world context model is tried, and the time slice ends.

The accuracy and real-time requirements of eye tracking can be balanced by adjusting the length of the time slice.

The operation of the eye tracking apparatus using CA-POMDP for robust human computer interaction will be described with reference to the flowchart of FIG.

The control unit 310 of the CA-POMDP module 300 initializes the state s for each of the world context models (S100).

At this time, the image quality label unit 210 of the image quality evaluation module 200 acquires the first image, calculates the image quality label, and provides the image quality label to the control unit 310 and the world context modeling unit 320 of the CA-POMDP module 300 (S200).

Meanwhile, the control unit 310 of the CA-POMDP module 300 queries the real-time Q-learning unit 330 for the world context model c (S300).

Thereafter, the control unit 310 initializes the time slice (S400).

Accordingly, the real-time Q-learning unit 330 selects the action a from the current state s using the glyph policy (S500).

Since the glyph policy does not currently allow learning low value behavior, it does not search for untried operation sequences. However, high values can be derived in the future. In this regard, ε-Greek polis can be used to balance behavior between exploration and development.

The real-time Q-learning unit 330 obtains an action, observes an immediate compensation r by calculating an internal compensation, and observes a new internal state s' (S600).

Next, the real-time Q-learning unit 330 updates the Q-table of the world context model c, and the equation (31) is used at this time (S700).

Then, the real-time Q-learning unit 330 changes the state to the next state (S800), and if the time slice is completed or the eye tracking satisfies the success condition (S900), the process proceeds to step S200, If not satisfied, the process proceeds to step S500.

According to the present invention, CA-POMDP can be used to optimize system control parameters to ensure performance, instead of using a costly image capture device or a very limited situation.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments of the present invention are not intended to limit the scope of the present invention but to limit the scope of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

100: eye tracking module 200: image quality evaluation module
210: image quality labeling unit 220: picture quality learning unit
300: CA-POMDP module 310:
320: World context modeling unit 330: Real-time Q-learning unit

Claims (6)

At the input of the facial image, the eye region is extracted, the eye image is binarized with the binarization threshold, the pupil position is tracked in the binarized eye image, the eye tracking is performed, and the compensation value An outputting eye tracking module;
An image quality evaluation module that receives a face image and evaluates image quality to provide an image quality label; And
The system control parameter is defined as a state, a world context model is configured by a combination of various objects according to image quality labels, and the world context model represented by attributes and attribute values of various objects according to the image quality label provided by the image quality evaluation module Determining a current state based on the current state, performing an executable system control parameter adjustment operation based on the determined current state, updating the behavior function value based on the compensation value of the system control parameter adjustment operation, And a CA-POMDP module for selecting an optimal system control parameter adjustment behavior based on the value of the CA-POMDP module. The method according to claim 1,
Wherein the system control parameter comprises a binarization threshold, an angle parameter of the partial Huff transform, and a noise parameter of the Kalman filter. The method according to claim 1,
The image quality evaluation module
An image quality learning unit for learning an image quality index from the image quality index prototype using the collected training image; And
And an image quality labeling unit for extracting a region of interest and dividing the image into a rectangular patch, loading a prototype of the image quality learning unit, extracting pixels of the scanning window, and calculating an image quality label. The method according to claim 3,
Wherein the image quality label portion calculates a simplified image quality label in consideration of illumination change in calculating an image quality label. The method according to claim 1,
The CA-POMDP module
The system control parameter is defined as a state, the current state of each state variable is determined based on the information on the world context model represented by the attributes and property values of various objects, and an executable action based on the updated behavior function value is performed ;
A world context modeling unit for constructing a world context model based on a combination of various objects for each image quality label; And
And a real-time Q-learning unit for updating the behavior function value based on the compensation value of the behavior for the current state and storing the updated behavior function value. The method according to claim 1,
Wherein the real-time Q-learning unit updates the behavior function value of the world context model by acquiring an action, calculating an internal compensation, and observing a new internal state based on immediate compensation.

Images