WO2016115895A1 - On-line user type identification method and system based on visual behaviour - Google Patents

Abstract

Disclosed are an on-line user type identification method and system based on visual behaviour. Eye movement data of one or more different types of users is collected and processed, a watching information data set and a user type set are obtained, one or more pieces of eye movement feature data are obtained according to watching information in the watching information data set so as to form a sampling data set, the eye movement feature data is selected from the sampling data set to be input into a support vector machine, a user type classifier is obtained by training, a machine learning process is completed so as to obtain a classifier, collected eye movement data of a random on-line user is input into the trained user type classifier, and a user type of the random on-line user is identified according to the classifier. Three types of eye movement feature data when each user browses a web page are acquired and calculated by mainly utilizing eye movement tracking technology, and the types of on-line users are judged according to differences in the eye movement feature data. By means of user identification based on visual behaviour, the eye movement data of the on-line users can be actively recorded, the data can be extracted simply and reliably, and accuracy and reliability are high.

Description

Online user type recognition method and system based on visual behavior Technical field

The invention relates to the field of user type automatic identification technology, in particular to a method and system for online type recognition based on visual behavior.

Background technique

With the development of technology and the popularity of the network, the network has become an indispensable communication tool and information exchange platform for people's life, study, work, etc. At present, the network can only passively accept users through the keyboard, mouse, touch screen, etc. of computer hardware. The information request slowly receives the user's manual input, but the user can quickly obtain a large amount of information from the computer interface and audio, thereby causing a problem of unbalanced human-computer interaction bandwidth. With the widespread use of computer networks and the increasing demands of mass demand, research on computer network intelligence has attracted widespread attention.

Network intelligence not only needs to realize information processing intelligence, but also human-computer interaction intelligence, and webpage is an important human-machine interface for information interaction between people and networks. It is especially important to realize intelligent identification of online user types. Eye tracking technology provides a way for the realization of network intelligence. Eye tracking technology (referred to as eye movement technology) can record the user's eye movements, enabling users to directly operate the interface through the visual channel. The problem of overlapping bandwidth imbalance.

It is easier to know that different types of online users will have different visual modes when they operate on the interface through eye movement technology. For example, due to age, the elderly have decreased vision, decreased eyesight, narrowed vision, reduced cognitive function, reduced information processing ability, and visual behavior and youth. People are obviously different. When browsing the web, older people need more psychological effort than young people to get and process information from the web. Studies have shown that the elderly pay more attention to the central area of the webpage when viewing visually. The browsing strategy presents a central characteristic, while the young people use a free browsing strategy without obvious rules.

The existing online user type identification is mainly through questionnaires, online click-through rate and other methods, so it is difficult to obtain psychological activities in the process of online users online, the recognition accuracy is low, and the credibility is not high.

Therefore, it is necessary to provide a new visual behavior-based online user type identification method and system to solve the above technical problems.

Summary of the invention

The object of the present invention is to provide a method and system for identifying online users based on visual behavior, which can actively record eye movement data of online users, identify users according to different eye movement data, and extract data is simple and reliable, and the recognition accuracy is high and reliable. High degree.

According to an aspect of the present invention, a method for recognizing an online user type based on visual behavior is provided. In the first step, eye movement data of one or more different types of users is collected and processed to obtain a data set F including a gaze information and a user. Type set C;

In the second step, one or more eye movement feature data are obtained according to the gaze information in the gaze information data set F to form a sample data set;

In the third step, the eye movement feature data input support vector machine is selected from the sample data set, and the user type classifier is trained to complete the machine learning process to obtain the classifier;

In the fourth step, the collected eye movement data of any user on the network is input to the trained user type classifier, and the user type of any user on the network is identified according to the classifier.

In the above technical solution, f _m is a quaternary array (t _fk , n _fk , d _lk , d _rk ) in the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , ... f _m } , t _fk is the time of the browsing; n _fk is the number of gaze points of the browsing in t _fk time; d _lk is the diameter of the left pupil; d _rk is the diameter of the right pupil.

In the above technical solution, the plurality of eye movement characteristic data forming the sampling data set includes the following steps:

The first step, through the calculation formula

Calculate that all m S _Dk constitute the eye hop distance data set S={S _D1, S _D2 , S _D3 ,..., S _Dm }, where (x _k , y _k ) and (x _k+1 , y _k+1 ) is the coordinates of the kth and k+1 gaze points respectively, and i represents the number of gaze points of a certain user's browsing task;

In the second step, by calculating the gaze frequency fq _fk =n _fk /t _fk, all m fq _{fk are} calculated to constitute a gaze frequency data set f _f ={f _f1 , f _f2 , f _f3 ,...,f _fm };

The third step, through the calculation formula

Calculate that all m D _I sets constitute an array of pupil diameters A _d =[D ₁ , D ₂ , D ₃ ,..., D _m ], where d _ij is the jth gaze point of each task for the ith user Pupil diameter value;

The fourth step selects the three eye movement characteristics of the i-th gaze frequency fq _fi , the pupil diameter D _i and the saccade distance S _Di and the corresponding user type C _{q to} form a basic sampling unit M _i ={fq _fi ,S _Di , D _i , c _q }, all m basic sampling units constitute a sampling data set: M' _m = {M ₁ , M ₂ , . . . M _m }.

In the above technical solution, training to obtain the classifier includes the following steps:

In the first step, a basic sampling unit Mi={fq _fi , SD _i, D _i , c _q };

The second step is to extract the eye movement characteristic data, that is, the training sample characteristic parameters fq _fi , SD _i and D _{i to} form a feature parameter vector;

In the third step, the sampling symbol function is used as the judgment statement. If the statement belongs to the user type c _q corresponding to the feature parameter, the SVM outputs yi=1, otherwise yi=-1, so that the classifier is trained.

In the above technical solution, the user type identification is implemented by the following steps:

The first step is to input the eye movement data of any user on the network into the trained user type classifier;

In the second step, the user type of any user on the network is identified according to the classifier.

According to another aspect of the present invention, a visual behavior based online user type identification system is provided, comprising an acquisition processing unit, an acquisition unit, a training unit and an identification unit connected in sequence; wherein the collection processing unit is used for one or more The eye movement data of different types of users are collected and processed to obtain a gaze information data set and a user type set; the obtaining unit is configured to obtain one or more eye movement characteristic data according to the gaze information in the gaze information data set F, to form a sampling data set; the training unit is configured to select an eye movement feature data input support vector machine from the sample data set, train a user type classifier to complete the machine learning process to obtain a classifier; and the identification unit is configured to collect the eye of any user on the network. The dynamic data is input to the trained user type classifier, and the user type of any user on the network is identified according to the classifier.

In the above technical solution, the acquisition processing unit further includes: a gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , . . . f _m }, where f _m is a four-element array (t _fk , n _fk , d _lk , d _rk ), t _fk is the time of the browsing; n _fk is the number of gaze points of the browsing in t _fk time; d _lk is the diameter of the left pupil; d _rk is the diameter of the right pupil.

In the above technical solution, the obtaining unit further includes:

Through calculation formula

Calculate all m S _{Dk to} form the eye hop distance data set S={S _D1 , S _D2 , S _D3 ,..., S _Dm }, where (x _k , y _k ) and (x _k+1 , y _k+1 ) is the coordinates of the kth and k+1 gaze points respectively, and i represents the number of gaze points of a certain user's browsing task;

By calculating the gaze frequency f _qfk =n _fk /t _fk , all m f _{qfk are calculated to} constitute the gaze frequency data set ff={f _f1 , f _f2 , f _f3 , . . . , f _fm };

Through calculation formula

Calculate that all m D _i sets constitute an array of pupil diameters Ad=[D ₁ , D ₂ , D ₃ ,..., D _m ], where d _ij is the pupil of the jth fixation point for each task of the i-th user Diameter value

Selecting the three eye movement characteristics of the i-th gaze frequency f _qfi , the pupil diameter D _i and the saccade distance S _Di and the corresponding user type C _{q to} form a basic sampling unit Mi={fqfi, S _Di , D _i , c _q }, all m basic sampling units constitute a sample data set: M'm = {M ₁ , M ₂ , . . . M _m }.

In the above technical solution, the training unit further includes: selecting a basic sampling unit M _i ={fq _fi , S _Di , D _i , c _q },

Extracting the eye movement feature data, that is, the training sample feature parameters fq _fi , S _Di and D _{i to} form a feature parameter vector;

The sampling symbol function is used as the judgment statement. If the statement belongs to the user type c _q corresponding to the feature parameter, the SVM outputs yi=1, otherwise yi=-1, so that the classifier is trained.

In the above technical solution, the identifying unit further includes: inputting the collected eye movement data of any user on the network to the trained user type classifier;

The user type of any user on the network is identified according to the classifier.

The invention discloses a visual behavior-based online user type recognition method and system, which mainly utilizes an eye tracking technology to identify an online user type according to an online user visual mode and a plurality of eye movement features. It is used in the eye-moving human-computer interaction environment, and three kinds of eye movements are obtained by calculating the user browsing the webpage. The data is collected, and the type of online users is determined according to the difference of the eye movement characteristic data. User recognition based on visual behavior can actively record eye movement data of online users, and the data is simple and reliable, with high accuracy and high credibility.

DRAWINGS

1 is a flow chart of an embodiment of a visual behavior based online user type identification method according to the present invention;

2 is a schematic diagram of an embodiment of eye movement data;

FIG. 3 is a schematic structural diagram of an embodiment of a visual behavior based online user type identification system according to the present invention.

detailed description

The present invention will be further described in detail below with reference to the specific embodiments thereof and the accompanying drawings. It is to be understood that the description is not intended to limit the scope of the invention. In addition, descriptions of well-known structures and techniques are omitted in the following description in order to avoid unnecessarily obscuring the inventive concept.

1 is a flowchart of an embodiment of a visual behavior-based online user type identification method of the present invention, and an embodiment of the method of the present invention is described in conjunction with an embodiment of the eye movement data shown in FIG. .

In one embodiment, the visual behavior based online user type identification method may mainly include the following steps:

In step S1, eye movement data (m eye movement data) of one or more different types of users are collected and processed to obtain a gaze information data set F={f1, f2, f ₃ , f ₄ , ... f _m } and set of user type sets C={c ₁ , c ₂ , c ₃ ,...c _q }.

Visual behavior, the sensitivity of people to the information of graphic symbols and the way of thinking reflected by visual senses (the behavior of eyeballs based on visual senses), here refers to the characteristics of different types of online users when browsing the web, such as when the elderly browse the web More attention is paid to the central area of the webpage, and young people present an irregular free browsing strategy.

Eye movement data, here refers to data related to eye movements, including but not limited to data related to eye movements (or eye movement patterns) such as gaze, saccade, and follow-up. A method for collecting eye movement data, for example, can be realized by a combination of an optical system, a pupil center coordinate extraction system, a vision and pupil coordinate superposition system, and an image and data recording and analysis system. The camera's eye tracker, etc., can collect the eye movement data of the online user, and can also eliminate the abnormal data to obtain a correct gaze information data set. For example, the eye tracker can collect and record the eye movement data. Eye movement data and user types are used as learning sets to learn eye movement patterns (eye movement patterns) of different users. Among them, according to the eye movement data, the sensitivity of the user who browses the webpage to different graphic symbol information and/or the behavior of visual sensory reflection and the like can be known.

Looking at the information data, here refers to the data related to such eye movement information of the "observed" object being observed in the eye movement data.

User type, here refers to the type of network access user corresponding to the collected eye movement data. Among them, the types that need to be divided can be preset, such as types by age (elderly, young people), types by gender (men, women), and so on.

Collecting eye movement data of the user can be processed as needed. For example, it can be saved by collection, array, matrix, etc., and all records are divided into several basic data sets, including, for example, gaze information data set F. ={f ₁ ,f ₂ ,f ₃ ,f ₄ ,...f _m }, a set of user types C={c ₁ , c ₂ , c ₃ ,...c _q }, and so on.

In an example where the online user type is pre-set to the age type, it is possible to collect visual behaviors of web users of different ages (eg, seniors and young people) in the browser interface. As a specific way, 52 different types can be collected and recorded at a sampling frequency of 120 Hz by using a sensing device including an eye tracker device (eg, an infrared camera of a Tobii T120 non-invasive eye tracker manufactured in Sweden). Among the users (including 26 senior citizens and 26 young people), each user performs 10 times of eye movement data generated by the visual behavior of the task in the web interface. In the above eye movement data when the collected 52 users respectively perform 10 browsing tasks, the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , . . . f _m } may be F={f ₁ , f ₂ , f ₃ , f ₄ , ... f ₅₂₀ }, that is, the m eye movement data of this example is 52*10=520, that is, the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ ,...f ₅₂₀ } contains all the gaze information. An example of a data set C={c ₁ , c ₂ , c ₃ , ... c ₅₂ } of a user type of 52 (p=52) different types of users corresponding to the above eye movement data: the preset type flag is Young people are marked as 1, and older people are marked as 2, so, C = {1, 2, 2, 1 ... 2}.

For the data set F set {f ₁ , f ₂ , f ₃ , f ₄ , ... f _m } of the gaze information, if any of the elements is represented by f _k , then f _k is a four-element array, which may contain The four kinds of information (t _fk , n _fk , d _lk , d _rk ) can in turn represent the browsing time t _{fk of} the kth user, the number of gaze points browsed during the t _fk time, and the diameter of the left pupil at this time. The diameter of the right pupil at this time. The gaze point may refer to a point at which the eye does not move at the position of the web page when browsing the webpage. As in the above example, the gaze information data f ₁ at the time of the first browsing of the first user includes four kinds of information (t _f1 , n _f1 , d _l1 , d _r1 ), where t _f1 is the first browsing of the first user. Time; n _f1 is the number of gaze points viewed in the t _f1 time; d _l1 is the left pupil diameter (left eye pupil diameter); d _r1 is the right pupil diameter (right eye pupil diameter).

In step S2, one or more eye movement feature data (or at least one eye movement feature data) is obtained based on the gaze information in the gaze information data set F to form a sample data set.

A specific method is as follows: extracting the gaze information included in the gaze information data set F, and calculating, by calculating, the saccade distance S _Dk , the gaze frequency fq _fk , the pupil diameter d _{fk ,} and the like for each user browsing task Dynamic feature data (ie, feature data that characterizes eye movements).

The eye hop distance refers to the Euclidean distance of the two gaze points when each user performs a browsing task and the position of the gaze point changes. In the example of step S1, the calculation can be performed based on the information in the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , ... f ₅₂₀ } when the 52 users perform the browsing task 10 times.

In the present invention, a method for calculating the saccade distance S _Dk may be: when the first user browses the task for the first time, the coordinates of the ith gaze point are (x _i , y _i ), and the i+1th gaze The coordinates of the point are (x _i+1 , y _i+1 ), and the average value of the i-th eye-jump distance is taken as the feature of the current eye-jump distance (S _D1 ). The calculation formula is:

Where (x _k , y _k ) and (x _k+1 , y _k+1 ) are the coordinates of the kth and k+1 gaze points, respectively, and i represents the number of gaze points of a certain user's browsing task. Thus, S _D1 =0.7552 is calculated. Further, the information in the gaze information data set F = {f ₁ , f ₂ , f ₃ , f ₄ , ... f ₅₂₀ } is sequentially extracted, and the corresponding ones are calculated: S _D2 = 0.9119; ...; S _D520 = 1.0004. To obtain a total of 52 users to perform 10 browsing tasks (ie 520 times) squint distance data sets (collections):

S={0.7552,0.9119,...,1.0004}

The gaze frequency refers to the number of gaze points per unit time each time the user performs a browsing task. Similarly, in the example of step S1, the calculation can be performed based on the information in the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , ... f ₅₂₀ } when the 52 users perform the browsing task 10 times respectively. .

In the present invention, a way of calculating the gaze frequency may be: the gaze frequency fq _fn = n _fk / t _fk , as in the above example, assume that the first user who is collecting the first time browsing the task t _f1 = 24, the gaze point The number n _f1 =10511, the calculation of the number of gaze points per unit time (ie, the gaze frequency) is: fq _f1 =n _f1 /t _f1 =10511/24=437.9583, and then, the gaze information data set F= The information in {f ₁ ,f ₂ ,f ₃ ,f ₄ ,...f ₅₂₀ } is calculated as: fq _f2 =n _f2 /t _f2 =10365/45=230.3333;...;fq _f520 =n _f520 /t _f520 = ₁₀₅₁₇ /18=584.2778. Thus, the gaze frequency data set (set) of all 52 users performing 10 browsing tasks (ie, 520 times) is obtained:

FQ _f = {437.9683, 230.3333, ..., 584.2778};

The pupil diameter d _fk may refer to the diameter value of the pupil of a certain fixation point of each user at a certain browsing. For example, taking the gaze information data set collected in step S1 as an example, the left and right pupil diameter data d _lk and d _rk collected in the set are extracted, and the pupil diameter can be calculated. A calculation method, for example, can calculate the average of the left and right pupil diameters to represent the corresponding pupil diameter value of a certain user when browsing, that is, the pupil diameter value d _fk = (d _lk + d _rk )/2. Thereby, all the pupil diameters can be obtained, and a pupil diameter matrix is provided. For example, suppose that the qth user performs a browsing task, and n gaze points are selected in each task, which constitutes a pupil diameter matrix Sd of q×n:

Each of the rows represents the pupil diameter value of each gaze point of the same user under a certain browsing task, and there are a total of n gaze points, so each row has n pupil diameter values;

The element Di in the pupil diameter matrix is the average value of each row of the pupil matrix, which is:

All m D _i sets constitute an array of pupil diameters A _d =[D ₁ , D ₂ , D ₃ ,..., D _m ], where d _ij is the pupil diameter of the jth fixation point for each task performed by the ith user value;

An example of the 52 people browsing in step S1 10 times: according to the collected gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , ... f ₅₂₀ }, the information in the above calculation manner can be sequentially calculated D ₁ =1.2523; D ₂ =1.3799;...;D ₅₂₀ =-0.986, thus obtaining a pupil diameter data group composed of 52 users respectively performing 10 browsing tasks, ie 520 times:

A _d = {1.2523, 1.3799, ..., -1.2757}.

According to the above example, the three eye movement characteristic data of the gaze frequency fq _fn , the pupil diameter D _m and the saccade distance S _Di are selected, and the saccade distance S _Di and the gaze frequency fq _fi of each of the above-mentioned users each time browsing tasks are performed. The pupil diameter D _i and this time the user type c _i can form a basic sampling unit (ie, a sample data set, that is, a combination of eye movement characteristic data): M _i ={fq _fi ,S _Di ,D _i ,c _q } . Therefore, the sample data set of q users such as 52 users performing n times such as 10 browsing tasks is: M' _{q × n} = {M ₁ , M ₂ , ..., M _{q × n} }, such as M' ₅₂₀ = { M ₁ , M ₂ , . . . M ₅₂₀ }.

Further, it is also possible to perform normal normalization processing on the sampled data set M' to obtain M" to improve the value or to optimize subsequent processing and the like.

In step S3, the eye movement feature data input support vector machine is selected from the sampled data set, and the user type classifier is trained. Thereby completing the machine learning process to obtain the classifier.

In one embodiment, the eye movement feature data is selected from the sampled data set in step S2, that is, a set of numerical values of the gaze frequency array, the pupil diameter array, and the squint distance array are input to the support vector machine SVM for training, thereby training the user. Type classifier.

Taking the above-mentioned 52 users' browsing tasks for 10 times as an example: when using SVM training, the eye movement characteristic data sentences of the elderly and young people are selected as training samples from the eye movement characteristic data, and one of the user types is selected as the recognition target for training. Specifically, a basic sampling unit may be selected from the sampling data set M' ₅₂₀ = {M ₁ , M ₂ , . . . M ₅₂₀ } composed of 52 user browsing tasks, for example, selecting the first one. The user type is the first basic sampling unit M ₁ ={fq _f1 , S _D1 , D ₁ , 1} when the young person performs the first browsing task, and the specific value is M ₁ = {437.9583, 0.7552, 1.2523, ₁ } The eye movement characteristic data is extracted, that is, the training sample characteristic parameter fq _f1 =437.9583, S _D1 =0.7552 and D ₁ =1.2523 constitute a feature parameter vector, and the sampling symbol function is used as a judgment statement if the statement belongs to the corresponding feature parameter. User type 1, then let the SVM output yi=1, otherwise yi=-1, (where i=1, 2, 3...n); if the 52nd user type is selected as the last time for the 10th browsing task for the elderly A basic sampling unit M ₅₂₀ = {fq _f520 , S _D520 , D ₅₂₀ , 2}, the specific value is

M ₅₂₀ = {584.2778, 1.0004, -0.986, 2},

The characteristic parameters fq _f520 =584.2778, S _D520 =1.0004 and D ₅₂₀ =-0.986 are formed to form a feature parameter vector. The sampling symbol function is used as the judgment statement. If the statement belongs to the user type 2 corresponding to the feature parameter, the SVM is output. Yi=1, otherwise yi=-1, (where i=1, 2, 3...n). Thus, using the feature parameter vector of the training sample and the SVM output as the training set, the kernel function is selected as a Gaussian (radial basis) function, and the existing decomposition type algorithm can be used for the corresponding user type (eg, elderly or young people). The support vector machine SVM is trained to obtain the support vector xi (i=1, 2, 3...n), the support vector weight coefficient a and the offset coefficient of the training set; for example: training into elderly and young user type classification Device.

In step S4, the collected eye movement data of any user on the network is input to the trained user type classifier, and the user type of any user on the network is identified according to the classifier.

In one embodiment, the eye movement data is any collected eye movement data of the online user (such as captured or acquired by the eye tracker), and may include, for example, all that has been collected (eg, all collected in step S1). Eye tracking data), and/or real-time (or current) eye movement data that is further tracked when the user browses the Internet in real time, etc., that is, any eye movement data of the user who browses online, And enter this data into the trained user type classifier.

In the classifier, one way may be to determine the corresponding type of online user through the output decision function, thereby identifying the type of user of the online user corresponding to any eye movement data (for example: young people or elderly people, women or men, luxury Product users or general item users, etc.).

In accordance with another aspect of the present invention, a block diagram of an embodiment of a visual behavior based online user type identification system in accordance with the present invention is shown in FIG.

In this example, the visual behavior based online user type identification system 300 includes an acquisition processing unit 301, an acquisition unit 302, a training unit 303, and an identification unit 304.

The collection processing unit 301 is configured to collect and process eye movement data (m eye movement data) of one or more different types of users, and obtain a data set including gaze information F={f ₁ , f ₂ , f ₃ , f ₄ , ... f _m } and a set of user type sets C = {c ₁ , c ₂ , c ₃ , ... c _q }. The unit can use various eye movement data collection devices such as an eye tracker to collect eye movement data of the online user, and then can also cull the abnormal data to obtain a correct set of gaze information data sets, such as step S1. The example of distinguishing user types by age (elderly and young people) records the eye movement data when the user browses the webpage in the interface, and the eye movement data and the user type are used as learning sets to learn the eye movements of different users. Mode, after collecting the user's eye movement data, slightly process and divide all the records into two basic data sets as needed, respectively, the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ ,...f _m } and user type set C={c ₁ , c ₂ , c ₃ ,...c _q }. Wherein, the gaze information data set F={f ₁ , f ₂ , f ₃ , f ₄ , . . . f _m } contains all the gaze information, and f _k is a quaternary array containing four kinds of information (t _fk , n _fk , d _Lk , d _rk ), t _fk is the time of the browsing; n _fk is the number of gaze points for browsing in t _fk time; d _lk is the diameter of the left pupil; d _rk is the diameter of the right pupil. Among them, the user type set C={c ₁ , c ₂ , c ₃ , ... c _n } contains young people and the elderly, the user type is young, the flag is 1, and the user type is old, the flag is 2.

For the specific processing and function of the acquisition processing unit 301, refer to the description of step S1.

The obtaining unit 302 is configured to obtain one or more eye movement feature data (or obtain at least one eye movement feature data) according to the gaze information in the gaze information data set F to form a sample data set. For example, in the example of step S2, it is possible to extract and calculate a plurality of eye movement characteristic data based on the gaze information data set from the acquisition processing unit 301 to constitute a sample data set. The eye movement characteristic data includes the eye movement distance S _Dk , the fixation frequency fq _fk , the pupil diameter d _{fk ,} and the like. Each eye movement characteristic data has a corresponding data group: eye hop distance data set S={S _D1, S _D2 , S _D3 , . . . , S _Dm }, gaze frequency data set FQ={f _f1 , f _f2 , f _f3 , ..., f _fm }, pupil diameter data set Ad = [D1, D2, D3, ..., Dm], and so on. And the gaze frequency fq _{fk, the} saccade distance S _{Di , the} pupil diameter D _i and the user type C _q form a basic sampling unit, M _i ={fq _fi , S _Di , D _i , c _q }, thereby obtaining a sampled data set. It is: M' _{q × n} = {M ₁ , M ₂ , ..., M _{q × n} }, such as M' ₅₂₀ = {M ₁ , M ₂ , . . . M ₅₂₀ }. Furthermore, the sampled eye movement data set can be normalized to obtain an optimized new sample data set M".

For the specific processing and function of the obtaining unit 302, refer to the description of step S2.

The training unit 303 is configured to select an eye movement feature data input support vector machine from the sample data set, and obtain a user type classifier. Thereby completing the machine learning process to obtain the classifier.

For example, the eye movement feature data of the acquisition data set of the acquisition unit 2, that is, the gaze frequency array, the pupil diameter array, and the set of values in the eye hop distance array are selected, and the support vector machine SVM is input, and the user type classifier is trained. Specifically, the SVM training can select the elderly and young people's eye movement feature data sentences as training samples from the eye movement feature array; select one of the user types as the recognition target, and extract the characteristic parameters for the i-th eye movement data statement. A feature parameter vector is formed, and the sample symbol function is used as a judgment statement. If the statement belongs to the user type, the SVM outputs yi=1, otherwise yi=-1. In this way, the feature parameter vector and SVM output of the training sample are used as the training set, and the kernel function is a Gaussian (radial basis) function. The existing decomposition algorithm is used to train the user type support vector machine to obtain support of the training set. The vector xi (i = 1, 2, 3...n), the support vector weight coefficient a and the offset coefficient are trained by the elderly and the young people respectively.

The specific processing and function of the training unit 303 is described in the description of step S3.

The identification unit 304 is configured to input the collected eye movement data of any user on the network to the trained user type classifier, and identify the user type of any user on the network according to the classifier.

For example, the eye movement data may be eye movement data (current, past, real-time, etc.) of any online user captured or collected by the eye tracker, including: all collected (eg, collected in step S1) All eye movement data, and/or real-time (or current) eye movement data, etc., which are further tracked when the user browses the Internet in real time. That is, any eye movement data of the user who browses on the Internet is obtained, and the data is input to the trained user type classifier.

In the classifier, one way may be that the classifier determines the corresponding online user type through the output decision function, thereby identifying the type of user of the online user corresponding to any eye movement data (for example: young or elderly, woman or Men, luxury users or general goods users, etc.).

The specific processing and function of the identification unit 304 is described in the description of step S4.

The processing and functions implemented by the system in this embodiment are basically corresponding to the foregoing method embodiments shown in FIG. 1 to FIG. 2, and therefore, in the description of the present embodiment, reference may be made to the related description in the foregoing embodiments. I will not repeat them here.

The following is an application example of the identification method and system of the present invention:

According to the above-mentioned 52 cases of 52 people, 52 users were recorded at a sampling frequency of 120 Hz by using the Tobii T120 non-invasive eye tracker produced in Sweden, including 26 senior citizens and 26 young people, respectively, 10 times. The eye movement data of the task is used to learn the eye movement mode when different types of users browse the webpage. The collected eye movement data of the 52 users and the corresponding user type data divide all the records into two basic data sets: the gaze information data set of the eye movement data of the user including all the gaze information.

F={f ₁ ,f ₂ ,f ₃ ,f ₄ ,...f ₅₂₀ }, and,

Corresponding user type data set

C={c ₁ ,c ₂ ,c ₃ ,...c ₅₂ }={1,1,...,2}.

From the gaze information, the user's saccade distance is calculated: S _D1 =0.7552, S _D2 =0.9119,...,S _D520 =1.0004, and the saccade distance data group is obtained:

S = {0.7552, 0.9119, ..., 1.0004}.

From the gaze information, calculate the user's gaze frequency: fq _f1 =n _f1 /t _f1 =10511/24=437.9583,fq _f2 =n _f2 /t _f2 =10365/45=230.3333,...,fq _f520 =n _f520 /t _f520 = 10517/18=584.2778, get the gaze frequency data set:

FQ _F = {437.9683, 230.3333, ..., 584.2778}.

From the gaze information, the pupil diameter of the user is calculated: D ₁ =1.2523, D ₂ =1.3799,..., D ₅₂₀ =-0.986, and the pupil diameter data set is obtained:

Ad={1.2523, 1.3799,...,-1.2757}.

Thus, the basic sampling unit is:

M ₁ = {437.9583, 1.2523, 0.7552, ₁ };

M ₂ = {230.3333, 1.3799, 0.9119, 1};

...

M ₅₂₀ = {584.2778, -0.986, 1.0004, 2};

The resulting sample data set is:

By normalizing the sampled eye movement data set, a new sample data set can be obtained:

According to the above embodiment of the method and system of the present invention, the sampled data set to be identified is input (extracting the sample training and obtaining the classifier) and judged by the output decision function, that is, selecting the gaze frequency, the pupil diameter, and the saccade distance. Combining the features, the classification function selects a linear function, inputs the eye movement data of the user to be identified into the trained classifier, and outputs the identified user type.

For example, the line jump distance, the fixation frequency, the pupil diameter, and the feature combination are respectively classified by the Liner function, the Polynomial function, the Rbf kernel function, and the Sigmoid function. Table 1 shows the classification results as follows:

Table 1:

	Liner	Polynomial	Rbf	Sigmoid
					Gaze frequency	0.5537	0.4942	0.5471	0.5537
Pupil diameter	0.8946	0.7910	0.8997	0.8963
					Eye jump distance	0.5652	0.5652	0.5652	0.5652
Feature combination	0.9148	0.6426	0.7426	0.5185
					Normalized combination	0.9346	0.8962	0.9346	0.9346

The present invention is directed to a visual behavior based online user type identification method and system for eye movement In the interactive environment, by obtaining three kinds of eye movement characteristic data when the user browses the webpage, according to the difference of the eye movement characteristic data, the identification of the visual behavior of the online user type is determined, and the eye movement data of the online user can be actively recorded, and the data is simple and reliable. , high accuracy and high credibility.

The above-described embodiments of the present invention are intended to be illustrative only and not to limit the invention. Therefore, any modifications, equivalent substitutions, improvements, etc., which are made without departing from the spirit and scope of the invention, are intended to be included within the scope of the invention. Rather, the scope of the appended claims is intended to cover all such modifications and modifications

Claims (10)

1. An online user type identification method based on visual behavior, comprising: steps:

   S1, collecting and processing eye movement data of one or more different types of users, obtaining a gaze information data set F and a user type set C;

   S2. Obtain one or more eye movement feature data according to the gaze information in the gaze information data set F to form a sample data set.

   S3. Selecting an eye movement characteristic data input support vector machine from the sampled data set, and training to obtain a user type classifier, thereby completing a machine learning process to obtain a classifier;

   S4. Input the eye movement data of any user on the network into the trained user type classifier, and identify the user type of any user on the network according to the classifier.
2. The visual behavior-based online user type identification method according to claim 1, wherein the step S1 further comprises:

   Looking at the information data set F = {f ₁ , f ₂ , f ₃ , f ₄ , ... f _m }, where f _m is a quaternion array (t _fk , n _fk , d _lk , d _rk ), t _{fk is} The time of the second browsing; n _fk is the number of gaze points for browsing in t _fk time; d _lk is the diameter of the left pupil; d _rk is the diameter of the right pupil.
3. The visual behavior-based online user type identification method according to claim 2, wherein the step S2 further comprises:

   S21, through the calculation formula

   

   Calculate all m S _{Dk to} form the saccade distance data set S={S _D1 , S _D2 , S _D3 ,..., S _Dm }, where (x _k , y _k ) and (x _k+1 , y _{k+ 1} ) is the coordinates of the kth and k+1 gaze points respectively, and i indicates the number of gaze points of a certain user's browsing task;

   S22. Calculate the frequency fq _fk =n _fk /t _fk by calculating a formula, and calculate that all m fq _fk constitute a gaze frequency data set f _f ={f _f1 , f _f2 , f _f3 , . . . , f _fm };

   S23, through the calculation formula

   

   Calculate that all m D _I sets constitute an array of pupil diameters A _d =[D ₁ , D ₂ , D ₃ ,..., D _m ], where d _ij is the jth gaze point of each task for the ith user Pupil diameter value;

   S24. Selecting the three eye movement characteristics of the i-th gaze frequency fq _fi , the pupil diameter D _i and the eye-jump distance S _Di and the corresponding user type C _{q to} form a basic sampling unit M _i ={fq _fi ,S _Di ,D _i , c _q }, all m basic sampling units constitute a sampling data set: M' _m = {M ₁ , M ₂ , . . . M _m }.
4. The visual behavior-based online user type identification method according to any one of claims 1 to 3, wherein the step S3 further comprises:

   S31. Select a basic sampling unit M _i ={fq _fi , S _Di , D _i , c _q },

   S32, extracting the eye movement characteristic data, that is, the training sample feature parameters fq _fi , S _Di and D _{i to} form a feature parameter vector;

   S33, using the sampling symbol function as the judgment statement. If the statement belongs to the user type c _q corresponding to the feature parameter, let the SVM output yi=1, otherwise yi=-1, so that the classifier is trained.
5. The visual behavior-based online user type identification method according to any one of claims 1 to 3, wherein: step S4 further comprises:

   S41. Input the eye movement data of any user on the network into the trained user type classifier;

   S42. Identify, according to the classifier, a user type of any user on the network.
6. An online user type identification system based on visual behavior, comprising: a data acquisition processing unit, an acquisition unit, a training unit unit and a recognition unit connected in sequence; wherein

   An acquisition processing unit, configured to collect and process eye movement data of one or more different types of users, to obtain a gaze information data set F and a user type set C;

   An acquiring unit, configured to obtain one or more eye movement feature data according to the gaze information in the gaze information data set to form a sample data set;

   a training unit, configured to select an eye movement feature data input support vector machine from the sample data set, and obtain a user type classifier, thereby completing a machine learning process to obtain a classifier;

   The identification unit is configured to input the collected eye movement data of any user on the network to the trained user type classifier, and identify the user type of any user on the network according to the classifier.
7. The system of claim 6 wherein the acquisition processing unit further comprises:

   Looking at the information data set F = {f ₁ , f ₂ , f ₃ , f ₄ , ... f _m }, where f _m is a quaternion array (t _fk , n _fk , d _lk , d _rk ), t _{fk is} The time of the second browsing; n _fk is the number of gaze points for browsing in t _fk time; d _lk is the diameter of the left pupil; d _rk is the diameter of the right pupil.
8. The system of claim 7, wherein the obtaining unit further comprises:

   Through calculation formula

   

   Calculate that all m S _Dk constitute the eye hop distance data set S={S _D1, S _D2 , S _D3 ,..., S _Dm }, where (x _k , y _k ) and (x _k+1 , y _k+1 ) is the coordinates of the kth and k+1 gaze points respectively, and i represents the number of gaze points of a certain user's browsing task;

   By calculating the gaze frequency fq _fk =n _fk /t _fk , all m fq _{fk are} calculated to constitute the gaze frequency data set f _f ={f _f1 , f _f2 , f _f3 ,...,f _fm };

   Through calculation formula

   

   Calculate that all m D _I sets constitute an array of pupil diameters A _d =[D ₁ , D ₂ , D ₃ ,..., D _m ], where d _ij is the jth gaze point of each task for the ith user Pupil diameter value;

   Selecting the three eye movement characteristics of the i-th gaze frequency fq _fi , the pupil diameter D _i and the saccade distance S _Di and the corresponding user type C _q constitute a basic sampling unit M _i ={fq _fi ,S _Di ,D _i , c _q }, all m basic sampling units constitute a sample data set: M' _m = {M ₁ , M ₂ , . . . M _m }.
9. The system of any one of claims 6-8, wherein the training unit further comprises:

   Select a basic sampling unit M _i ={fq _fi ,S _Di ,D _i ,c _q },

   Extracting the eye movement feature data, that is, the training sample feature parameters fq _fi , S _Di and D _{i to} form a feature parameter vector;

   The sampling symbol function is used as the judgment statement. If the statement belongs to the user type c _q corresponding to the feature parameter, the SVM outputs yi=1, otherwise yi=-1, so that the classifier is trained.
10. The system of any one of claims 6-8, wherein the identifying unit further comprises:

    Inputting the eye movement data of any user on the network to the trained user type classifier;

    The user type of any user on the network is identified according to the classifier.

Images