WO2016112690A1 - 一种基于眼动数据的网上用户状态识别方法和装置 - Google Patents

一种基于眼动数据的网上用户状态识别方法和装置 Download PDF

Info

Publication number
WO2016112690A1
WO2016112690A1 PCT/CN2015/087641 CN2015087641W WO2016112690A1 WO 2016112690 A1 WO2016112690 A1 WO 2016112690A1 CN 2015087641 W CN2015087641 W CN 2015087641W WO 2016112690 A1 WO2016112690 A1 WO 2016112690A1
Authority
WO
WIPO (PCT)
Prior art keywords
eye movement
state
online user
movement data
pupil diameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/087641
Other languages
English (en)
French (fr)
Inventor
吕胜富
栗觅
张孟杰
钟宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Publication of WO2016112690A1 publication Critical patent/WO2016112690A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/197—Matching; Classification

Definitions

  • the invention relates to the field of pattern recognition human-computer interaction technology, in particular to an online user state recognition method and device based on eye movement data.
  • Web intelligence is a method and application for realizing intelligence in the Web environment. It is a combination of artificial intelligence and computer network. Web intelligence includes two aspects: one is web-based human-computer interaction intelligence, and the other is information service intelligence.
  • the web page serves as an important human-machine interface for human interaction with the network, but currently it can only passively accept user information requests through keyboards, mice, and touch screens, thereby responding and providing information services. This passive sensing technology has become less and less able to meet people's demand for machine intelligence.
  • An object of the present invention is to provide an online user state recognition method based on eye movement data, which is capable of identifying two working states of online user search and browsing, and adopting active sensing technology to extract two eye movement features, and adopting such as
  • Based on the SVM classification method the technical problem of efficient and accurate machine recognition of user state is solved.
  • the learning speed is fast, the classification accuracy is high, and the recognition efficiency is high.
  • the present invention provides an online user state recognition method based on eye movement data, comprising the following steps:
  • Step S1 collecting eye movement data during the process of viewing the webpage by the online user, and preprocessing the eye movement data to obtain pre-processed eye movement data;
  • Step S2 performing feature extraction on the pre-processed eye movement data to obtain a plurality of eye movement characteristic parameters
  • step S3 the plurality of eye movement feature parameters of the online user are input into a preset state classifier for comparison to identify the current working state of the online user as a search state or a browsing state.
  • an infrared camera is used to collect eye movement data of the online user during viewing of the web page.
  • the eye movement data includes: a gaze point position, a gaze time, and a pupil diameter of the online user.
  • the plurality of eye movement characteristic parameters include a median change rate of the pupil diameter and a center distance of the fixation point
  • the step S2 further includes:
  • Step S21 forming a pupil diameter data set matrix u according to the pupil diameter in each eye movement data, wherein
  • n n sampling points selected among m gaze points
  • u ij is the pupil diameter value of the jth sampling of the ith gaze point, wherein , 1 ⁇ i ⁇ m, 1 ⁇ j ⁇ n;
  • D 0 is the baseline value of the pupil diameter
  • Step S22 calculating a gaze point center distance CD fk according to each of the gaze point positions, where
  • the coordinates of the fixation point are (X fk , Y fk ), and the coordinates of the center of the web page are (X 0 , Y 0 );
  • the number of webpages viewed by the online user is p, and q gaze points are selected for each webpage, which constitutes a gaze center distance matrix, and the median of each row of data in the gaze center distance matrix is calculated to constitute the following gaze point.
  • the median matrix of the center distance where
  • step S23 the median rate of change of the pupil diameter and the center distance of the fixation point are combined.
  • step S3 further includes:
  • Step S31 performing eye movement data and feature extraction in the process of searching state or browsing state of the online user collected before step S1, according to the first column as the tag value of the training or prediction sample, and other columns as the characteristics of the corresponding feature.
  • the values constitute the following combined feature data set:
  • Step S32 the combined feature data set in step S31 is input into a support vector machine, and trained to obtain a state classifier;
  • step S33 the plurality of eye movement feature parameters obtained in step S2 are input to the state classifier obtained in step S32, and the current working state of the online user is identified as a search state or a browsing state.
  • the online user state recognition method based on eye movement data solves the problem that the working state of the online user cannot be recognized in the field of human-machine interface based on visual behavior.
  • the invention recognizes two working states of online user search and browsing, adopts active sensing technology to extract two kinds of eye movement features, and adopts a classification method based on support vector machine, which has fast learning speed, high classification correct rate and high Identify efficiency.
  • the invention can recognize whether the working state of the user is a purposeful search state or an unintended browsing state according to the eye movement information when the online user views the webpage.
  • the method and system for identifying the working state of an online user based on eye movement data realizes a human-computer interaction in the form of eye movement, and identifies the working state of the user through the eye movement data of the user.
  • Another object of the present invention is to provide an online user state recognition device based on eye movement data, which is capable of identifying two working states of online user search and browsing, and adopting active sensing technology to extract two eye movement features, and adopting at the same time
  • the classification method based on support vector machine SVM solves the technical problem of efficient and accurate machine recognition of user state, which has fast learning speed, high classification correct rate and high recognition efficiency.
  • an online user state recognition device based on eye movement data, comprising: a data collection and preprocessing unit, configured to collect eye movement data during a process of viewing a webpage by an online user, and The eye movement data is preprocessed to obtain pre-processed eye movement data; the feature extraction unit is configured to perform feature extraction on the pre-processed eye movement data to obtain a plurality of eye movement characteristic parameters; and a state determination unit, configured to: The plurality of eye movement characteristic parameters of the online user are input into a preset state classifier for comparison to identify the current working state of the online user as a search state or a browsing state.
  • the data acquisition and pre-processing unit comprises an infrared camera for collecting eye movement data of the online user during viewing of the webpage.
  • the eye movement data includes: a gaze point position, a gaze time, and a pupil diameter of the online user.
  • the plurality of eye movement characteristic parameters include a pupil diameter change rate and a median of the fixation point center distance
  • the feature extraction unit further includes: a pupil diameter change rate calculation subunit and a fixation point center distance Median calculation subunit, where
  • the pupil diameter change rate calculation subunit is configured to form a pupil diameter data set matrix u according to a pupil diameter in each eye movement data, wherein
  • n n sampling points selected among m gaze points
  • u ij is the pupil diameter value of the jth sampling of the ith gaze point, wherein , 1 ⁇ i ⁇ m, 1 ⁇ j ⁇ n;
  • the pupil diameter change rate calculation subunit obtains an average value by the row in the pupil diameter data set matrix u, subtracts the baseline value D 0 of the pupil diameter, and divides the baseline value D 0 of the pupil diameter to obtain the following pupil diameter Rate of change matrix U,
  • D 0 is the baseline value of the pupil diameter
  • a median calculation sub-unit of the gaze point center distance is used to calculate a gaze point center distance CD fk according to each of the gaze point positions, wherein
  • the coordinates of the fixation point are (X fk , Y fk ), and the coordinates of the center of the web page are (X 0 , Y 0 ));
  • the median calculation sub-unit of the gaze point center distance sets the number of web pages viewed by the online user to p, and q gaze points are selected for each web page, thereby forming a gaze point center distance matrix, and calculating the gaze point center distance matrix
  • the median of each line of data constitutes the median matrix W of the center distance of the following fixation points, where
  • Median () is a function of the median.
  • the state determining unit further includes: eye movement data and feature extraction during a search state or a browsing state of the previously collected online user, and the first column is a training or prediction sample.
  • the tag value, other feature values listed as corresponding features constitute the following combined feature data set:
  • label is the target value of the training data set;
  • index is an integer starting from 1 and indicating the sequence number of the feature;
  • value is the data used for training or prediction, that is, the feature value, wherein the search status flag is 1, and the browsing status flag is 2 ;
  • the state judging unit inputs the combined feature data set collected by the data acquisition and preprocessing unit into the support vector machine, trains the state classifier, and then obtains multiple eye movements obtained by the data acquisition and preprocessing unit.
  • the feature parameter is input into the state classifier, and the current working state of the online user is identified as a search state or a browsing state.
  • the invention provides an online user state recognition device based on eye movement data, which solves the problem of human-computer based on visual behavior In the field of mouths, there is still the problem of not being able to identify the working status of online users.
  • the invention recognizes two working states of online user search and browsing, adopts active sensing technology to extract two kinds of eye movement features, and adopts a classification method based on support vector machine, which has fast learning speed, high classification correct rate and high Identify efficiency.
  • the invention can recognize whether the working state of the user is a purposeful search state or an unintended browsing state according to the eye movement information when the online user views the webpage.
  • the method and system for identifying the working state of an online user based on eye movement data realizes a human-computer interaction in the form of eye movement, and identifies the working state of the user through the eye movement data of the user.
  • FIG. 1 is a flow chart of an embodiment of an eye movement data based online user state recognition method according to the present invention
  • FIG. 2 is a flow chart of an embodiment of feature extraction of eye movement data in accordance with the present invention.
  • FIG. 3 is a flow chart of an embodiment of identifying a current working state of the online user in the method according to the present invention
  • FIG. 4 is a schematic structural diagram of an embodiment of an online user state recognition device based on eye movement data in the method according to the present invention
  • Figure 5 is a block diagram showing an embodiment of a feature extraction unit in a device in accordance with the present invention.
  • a human-machine interface of a webpage if the behavior of the user's online and the psychological state of the user can be obtained, the user can be provided with a better service.
  • visual search and browsing are the two main visual behaviors of users surfing the Internet.
  • Visual search is a user with a clear search target, purposefully looking for information on the page.
  • Visual browsing is the user's lack of clear goals, free to browse and view information on the web page.
  • the information search process on a web page is influenced by various factors such as the form of information presentation, information layout, and floating advertisements.
  • As more and more webpage designs adopt the design concept of floating advertisements, floating advertisements and other content in the process of online users searching for webpage content will have a negative impact on users, distracting users' attention and causing people to have aversion. Reduce user search efficiency. If you can distinguish the working status of online users, increasing or decreasing floating ads in a timely manner will help users pay attention to floating advertising content and improve search efficiency.
  • the eye movement information in the process of viewing the webpage by the online user will respectively display different characteristics in different states, can identify the characteristics of the eye movement information, judge the thinking state of the user during the process of viewing the webpage, and interact.
  • the eye movement data-based online user state recognition method and apparatus can generate human-computer interaction information according to the eye user data feature definition of the online user and the corresponding feature data combination, and use the support vector machine SVM.
  • Status identification determines whether the online user is searching for a web page or browsing a web page.
  • the present invention provides a specific embodiment of an online user state recognition method based on eye movement data, including the following steps:
  • step S1 the eye movement data of the online user in the two states of the specific target and the random browsing webpage in the search webpage is collected, and the eye movement data is preprocessed to obtain the pre-processed eye movement data.
  • the eye movement tracking device is used to acquire eye movement data of the online user in both the search and browse webpage states.
  • the eye tracking device can be a Tobii T120 eye tracker, using eye movement
  • the infrared camera in the instrument collects eye movement data of the online user in two states at a sampling frequency of 120 Hz.
  • eye movement data refers to data related to eye movement, including but not limited to data related to eye movements (or eye movement patterns) such as gaze, eye sac and follow.
  • the eye movement data includes: a gaze point of the online user (ie, a point at the web page when the user does not move the eye when viewing the webpage), a gaze time, and a pupil diameter.
  • the gaze information data set F of each gaze point contains all of the gaze information f k , that is, data related to such eye movement information of "watching" the observed object.
  • the f k is a five-element array (t fk , d lk , d rk , x fk , y fk ), t fk is the fixation time of the fixation point, d lk is the pupil diameter of the left eye, and d rk is the diameter of the pupil of the right eye.
  • x fk is the abscissa of the fixation point, and y fk is the ordinate of the fixation point.
  • the eye tracking technology can record the eye movement of the user on the webpage in real time.
  • the eye movement trajectory the trajectory of the eyeball
  • the diameter of the pupil the shape of the eyeball
  • the behavioral index of the eyeball according to the visual sensory response can more objectively reflect the user's behavior, thereby revealing the psychological state of the person on the webpage.
  • Pre-processing the above-mentioned eye movement data includes removing extreme data in the collected eye movement data and abnormal data other than three standard deviations, and obtaining pre-processed eye movement data.
  • Step S2 performing feature extraction on the pre-processed eye movement data to obtain a plurality of eye movement characteristic parameters.
  • each gaze point may be selected separately, and the gaze information corresponding to the gaze point includes gaze time information.
  • the pupil diameter change rate and the fixation point center distance of each fixation point during the fixation time are calculated.
  • the plurality of eye movement characteristic parameters include a median change rate of the pupil diameter and a center distance of the fixation point.
  • the gaze point center distance of the user may be calculated according to the gaze point location data of the user, for example, the gaze point coordinate (x fk , y fk ) and the Euclidean distance between the web page center coordinates are the gaze point center distance;
  • the user's pupil diameter d fk is used to calculate the pupil diameter of the user during viewing of the web page as a function of time; further, the median of all gaze points in the web page and the pupil diameter change rate of each gaze point are calculated as characteristic parameters. .
  • step S2 may further include the following steps:
  • step S21 the pupil diameter change rate is calculated to obtain a pupil diameter change rate characteristic data set.
  • the number of times the gaze point is in the gaze time is the gaze time divided by the sampling period, wherein the preset sampling period may be, for example, 8 milliseconds.
  • n is the n sampling points selected in the mth fixation point
  • u ij is the pupil diameter value of the jth sampling of the i-th fixation point, wherein , 1 ⁇ i ⁇ m, 1 ⁇ j ⁇ n;
  • Baseline pupil diameter value D u averaging data set matrix rows, the pupil diameter by subtracting 0, divided by the baseline pupil diameter D 0, an m-th row obtain the following rate of change in the pupil diameter of the matrix U,
  • D 0 is the baseline value of the pupil diameter
  • step S22 the median of the center distance of the gaze point is calculated, and the gaze point center distance feature data set is obtained. Calculating the gaze point center distance CD fk according to each gaze point position, wherein
  • the coordinates of the fixation point are (X fk , Y fk ), and the coordinates of the center of the web page are (X 0 , Y 0 ).
  • the gaze point information data set can be simplified to a triple (t fk , d fk , cd fk ).
  • step S23 the median rate of change in pupil diameter and the center distance of the fixation point are combined.
  • step S3 the plurality of eye movement characteristic parameters of the online user are input into a preset state classifier for comparison to identify the current working state of the online user as the search state or the browsing state.
  • the working state of the online user in the process of viewing the webpage is divided into two state types, one being a search state and one being a browsing state.
  • Obtaining the eye movement data of the user during the process of viewing the webpage will show different characteristics in the two states respectively.
  • the online user is in the search state
  • the pupil diameter change will be displayed on the eye movement data compared to the browsing state.
  • the rate is reduced, and the center distance of the fixation point exhibits characteristics of peripheral characteristics. Therefore, different characteristics of the eye movement data of the two states are obtained, that is, the state in which the user is in the process of viewing the web page can be determined.
  • step S3 may further include the following steps:
  • Step S31 performing eye movement data and feature extraction during the search state or browsing state of the online user collected before step S1.
  • One method is: extracting the eye movement data of the search task, obtaining the eye movement characteristic parameter in the search state; performing feature extraction on the eye movement data of the browsing task, and obtaining the eye movement characteristic parameter in the browsing state.
  • the other feature values listed as corresponding features constitute the following combined feature data set:
  • label is the target value of the training data set; index is an integer starting from 1 and indicating the sequence number of the feature; value is the data used for training or prediction, that is, the feature value, wherein the search status flag is 1, and the browsing status flag is 2 .
  • Step S32 performing machine learning on the combined feature data set in step S31 to train the state classifier.
  • One way is to input the combined feature data set into the support vector machine SVM and train to obtain the state classifier.
  • the eye movement feature parameters of the search state and the browsing state may be input into the support vector machine SVM for training, and the trained state classifier is obtained, thereby realizing the recognition of the visual behavior of the network user searching or browsing.
  • the support vector machine is used to train the obtained eye movement characteristic parameters, and the stateifier is trained.
  • a specific embodiment may be: using the feature parameter data set with the category label as the training sample set, and selecting the radial basis function as the radial basis function.
  • the kernel function is trained using five-fold cross-validation, that is, the number of training samples is 4/5 of the total sample, and the test sample is 1/5 of the total sample.
  • the best parameters C and gamma are selected by the grid search method.
  • the state classifier can take various forms, and other machine learning algorithms such as external, neural network, genetic algorithm, etc. can be implemented in addition to the support vector machine (SVM classifier) used in the above example to perform classifier training.
  • the state classifier can also be a libsvm support vector machine toolkit or the like.
  • step S33 the plurality of eye movement feature parameters (including real-time or already acquired) obtained in step S2 are input to the state classifier obtained in step S32, and the current working state of the online user is identified as a search state or a browsing state.
  • the eye movement feature data extracted in step S1 is sorted according to the data set format in step S2, and input into the trained state classifier, thereby determining whether the working state of the online user is search or browse, that is, identifying The type of work status of online users. For example, if the output of the support vector machine is 1, it corresponds to the search state; when the output is 2, it corresponds to the browse state.
  • the embodiment adopts an online user state recognition method based on eye movement data, and solves the problem that the working state of the online user cannot be recognized in the field of human-machine interface based on visual behavior.
  • the invention recognizes two working states of online user search and browsing, adopts active sensing technology to extract two kinds of eye movement features, and adopts a classification method based on support vector machine, which has fast learning speed, high classification correct rate and high Identify efficiency.
  • the invention can recognize whether the working state of the user is a purposeful search state or an unintended browsing state according to the eye movement information when the online user views the webpage.
  • the method and system for identifying the working state of an online user based on eye movement data realizes a human-computer interaction in the form of eye movement, and identifies the working state of the user through the eye movement data of the user.
  • the present invention also provides an online user state recognition device based on eye movement data.
  • the data acquisition and preprocessing unit 100, the feature extraction unit 200, and the state determination unit 300 are provided. .
  • the data collection and pre-processing unit 100 is configured to collect eye movement data during the process of viewing the webpage by the online user, and preprocess the eye movement data to obtain pre-processed eye movement data.
  • the data acquisition and pre-processing unit 100 includes an infrared camera for collecting eye movement data during the process of viewing the webpage by the online user.
  • the eye movement data includes: the location of the gaze point of the online user, the gaze time, and the pupil diameter.
  • the eye tracking data is used to obtain eye movement data when the online user views the webpage.
  • the eye tracking device may be a Tobii T120 eye tracker, and the eye movement data of the online user during the viewing of the webpage is collected by using the infrared camera in the eye tracker at a sampling frequency of 120 Hz.
  • eye movement data refers to data related to eye movement, including but not limited to data related to eye movements (or eye movement patterns) such as gaze, eye sac and follow.
  • the eye movement data includes: the gaze point of the online user (ie, the eye when the user views the webpage) The position of the web page at the point where the ball is not moving, the gaze time, and the pupil diameter.
  • the gaze information data set F of each gaze point contains all of the gaze information f k , that is, data related to such eye movement information of "watching" the observed object.
  • the f k is a five-element array (t fk , d lk , d rk , x fk , y fk ), t fk is the fixation time of the fixation point, d lk is the pupil diameter of the left eye, and d rk is the diameter of the pupil of the right eye.
  • x fk is the abscissa of the fixation point, and y fk is the ordinate of the fixation point.
  • the eye tracking technology can record the eye movement of the user on the webpage in real time.
  • the eye movement trajectory the trajectory of the eyeball
  • the diameter of the pupil the shape of the eyeball
  • the behavioral index of the eyeball according to the visual sensory response can more objectively reflect the user's behavior, thereby revealing the psychological state of the person on the webpage.
  • the data acquisition and pre-processing unit 100 pre-processes the above-mentioned eye movement data, including culling extreme data in the collected eye movement data and abnormal data other than three standard deviations, to obtain pre-processed eye movement data.
  • step S1 The specific implementation and processing of the data collection and pre-processing unit 100 is as described in step S1.
  • the feature extraction unit 200 is connected to the data acquisition and pre-processing unit 100 for performing feature extraction on the pre-processed eye movement data to acquire a plurality of eye movement characteristic parameters.
  • each gaze point may be selected separately, and the gaze information corresponding to the gaze point includes gaze time information.
  • the pupil diameter change rate, the fixation frequency, and the fixation point center distance of each fixation point during the fixation time are calculated.
  • the plurality of eye movement characteristic parameters include a median change rate of the pupil diameter and a center distance of the fixation point.
  • the feature extraction unit 200 includes a pupil diameter change rate calculation subunit 210 and a median calculation subunit 220 of the fixation point center distance.
  • the pupil diameter change rate calculation subunit 210 calculates the pupil diameter change rate to obtain a pupil diameter change rate characteristic data set.
  • the number of times the gaze point is in the gaze time is the gaze time divided by the sampling period, wherein the preset sampling period may be, for example, 8 milliseconds.
  • the pupil diameter change rate calculation subunit 210 is used according to each
  • the pupil diameter in the eye movement data constitutes a pupil diameter data set matrix u, wherein
  • n is the n sampling points selected in the mth fixation point
  • u ij is the pupil diameter value of the jth sampling of the i-th fixation point, wherein , 1 ⁇ i ⁇ m, 1 ⁇ j ⁇ n;
  • the pupil diameter change rate calculation subunit 210 obtains the mean value by the row in the pupil diameter data set matrix u, subtracts the baseline value D 0 of the pupil diameter, and divides the baseline value D 0 of the pupil diameter to obtain the following pupil diameter change rate matrix U,
  • D 0 is the baseline value of the pupil diameter
  • the median calculation sub-unit 220 of the gaze point center distance is used to calculate the gaze point center distance CD fk according to each gaze point position, wherein
  • the coordinates of the fixation point are (X fk , Y fk ), and the coordinates of the center of the web page are (X 0 , Y 0 ), and the data set of the fixation point information can be simplified into a triplet (t fk , d fk , cd fk ).
  • the median calculation sub-unit 220 of the gaze point center distance sets the number of web pages viewed by the online user to p, and selects q gaze points for each web page to form a gaze center distance matrix, and calculates the data of each line in the gaze center distance matrix.
  • the number of bits constitutes the median matrix W of the center distance of the following fixation points, wherein
  • Median () is a function of the median.
  • the feature extraction unit 200 further combines the above-described pupil diameter change rate and the median of the fixation point center distance.
  • step S2 The specific implementation and processing of the feature extraction unit 200 is as described in step S2.
  • the state judging unit 300 is connected to the feature extracting unit 200 for inputting a plurality of eye movement characteristic parameters of the online user into a preset state classifier for comparison, so as to identify the current working state of the online user as a search state or a browsing state. .
  • the state judging unit 300 performs eye movement data and feature extraction during the search state or the browsing state of the previously collected online user.
  • One method is: extracting the eye movement data of the search task, obtaining the eye movement characteristic parameter in the search state; performing feature extraction on the eye movement data of the browsing task, and obtaining the eye movement characteristic parameter in the browsing state.
  • the other feature values listed as corresponding features constitute the following combined feature data set:
  • label is the target value of the training data set; index is an integer starting from 1 and indicating the sequence number of the feature; value is the data used for training or prediction, that is, the feature value, wherein the search status flag is 1, and the browsing status flag is 2 .
  • the state judging unit 300 performs machine learning on the combined feature data set collected by the data collection and preprocessing unit to train the state classifier.
  • One way is: input the combined feature data set into the support vector machine SVM, train the state classifier, and then input the plurality of eye movement characteristic parameters obtained by the data acquisition and preprocessing unit into the state classifier to identify the online user.
  • the current working status is search status or browsing status.
  • the state determining unit 300 inputs the eye movement feature parameters of the search state and the browsing state into the support vector machine SVM for training, and obtains the trained state classifier, thereby implementing the subsequent visual behavior of searching or browsing the network user. Identification.
  • the obtained eye movement characteristic parameters are trained by using the support vector machine, and the training is obtained.
  • a specific embodiment may be: using a feature parameter data set with a category tag as a training sample set, selecting a radial basis function as a kernel function, and using a five-fold cross-validation during training, that is, the number of training samples is a total sample. For 4/5, the test sample is 1/5 of the total sample, and the best parameters C and gamma are selected by grid search.
  • state classifier can take various forms, and other machine learning algorithms such as external, neural network, genetic algorithm, etc. can be implemented in addition to the support vector machine (SVM classifier) used in the above example to perform classifier training.
  • SVM classifier support vector machine
  • the state classifier can also be a libsvm support vector machine toolkit or the like.
  • the state judging unit 300 inputs a plurality of eye movement feature parameters (including real-time or already acquired) into the state classifier, and identifies the current working state of the online user as a search state or a browsing state.
  • the extracted eye movement feature data is formatted in the manner of the feature extraction unit 200, and input into the trained state classifier to determine whether the working state of the online user is search or browse, that is, Identify the type of work status of online users. For example, if the output of the support vector machine is 1, it corresponds to the search state; when the output is 2, it corresponds to the browse state.
  • step S3 The specific implementation and processing of the state judging unit 300 is as described in step S3.
  • an online user state recognition device based on eye movement data is used, which solves the problem that the working state of the online user cannot be recognized in the field of human-machine interface based on visual behavior.
  • the invention recognizes two working states of online user search and browsing, adopts active sensing technology to extract two kinds of eye movement features, and adopts a classification method based on support vector machine, which has fast learning speed, high classification correct rate and high Identify efficiency.
  • the invention can recognize whether the working state of the user is a purposeful search state or an unintended browsing state according to the eye movement information when the online user views the webpage.
  • the method and system for identifying the working state of an online user based on eye movement data realizes a human-computer interaction in the form of eye movement, and identifies the working state of the user through the eye movement data of the user.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Eye Examination Apparatus (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种基于眼动数据的网上用户状态识别方法和装置，所述方法包括如下步骤：采集网上用户在观看网页的过程中的眼动数据，并对眼动数据进行预处理，得到预处理后的眼动数据（S1）；对预处理后的眼动数据进行特征提取，获取多个眼动特征参数（S2）；将网上用户的所述多个眼动特征参数输入到预设的状态分类器中进行比对，以识别所述网上用户的当前工作状态为搜索状态或浏览状态（S3）。针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了基于支持向量机的分类方法，学习速度快，分类正确率高，具有较高的识别效率。

Description

一种基于眼动数据的网上用户状态识别方法和装置 技术领域
本发明涉及模式识别人机交互技术领域，特别涉及一种基于眼动数据的网上用户状态识别方法和装置。
背景技术
当前，计算机网络已经成为人们工作、学习、生活等各个方面不可或缺的平台，计算机网络智能的研究成为了焦点。作为计算机网络智能的Web智能是研究Web环境中实现智能的方法和应用，是人工智能与计算机网络两个领域的结合。Web智能包括两个方面：一个是基于网页的人机交互智能，一个是信息服务智能。网页作为人与网络交互的重要人机界面，但是目前只能通过键盘、鼠标和触摸屏等被动的接受用户的信息请求，从而做出响应和提供信息服务。这种被动的感知技术已经越来越不能满足人们对于机器智能化程度的需求。
发明内容
本发明的一个目的是提供一种基于眼动数据的网上用户状态识别方法，该方法针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了诸如基于支持向量机SVM的分类方法，解决了高效准确的机器识别用户状态的技术问题,其学习速度快，分类正确率高，具有较高的识别效率。
为实现上述目的，本发明提供一种基于眼动数据的网上用户状态识别方法，包括如下步骤：
步骤S1，采集网上用户在观看网页的过程中的眼动数据，并对所述眼动数据进行预处理，得到预处理后的眼动数据；
步骤S2，对所述预处理后的眼动数据进行特征提取，获取多个眼动特征参数；
步骤S3，将所述网上用户的所述多个眼动特征参数输入到预设的状态分类器中进行比对，以识别所述网上用户的当前工作状态为搜索状态或浏览状态。
根据本发明的一个方面，采用红外线摄像机采集所述网上用户在观看网页的过程中的眼动数据。
根据本发明的一个方面，所述眼动数据包括：所述网上用户的注视点位置、注视时间和瞳孔直径。
根据本发明的一个方面，所述多个眼动特征参数包括瞳孔直径变化率和注视点中心距的中位数，所述步骤S2还包括：
步骤S21，根据每个眼动数据中的瞳孔直径构成瞳孔直径数据集矩阵u，其中，
Figure PCTCN2015087641-appb-000001
其中，m为所述眼动数据中的注视点的个数，n为在m个注视点中选取的n个采样点，uij为第i个注视点第j次采样的瞳孔直径值，其中，1≤i≤m，1≤j≤n；
在所述瞳孔直径数据集矩阵u按行求均值，减去瞳孔直径的基线值D0，除以所述瞳孔直径的基线值D0，得到下述瞳孔直径变化率矩阵U，
Figure PCTCN2015087641-appb-000002
其中，D0为瞳孔直径的基线值，
步骤S22，根据每个所述注视点位置计算注视点中心距CDfk，其中，
Figure PCTCN2015087641-appb-000003
其中，注视点的坐标为(Xfk,Yfk)，网页中心坐标为(X0,Y0)；
设所述网上用户观看网页数量为p，每个网页选取q个注视点，则构成注视点中心距矩阵，计算所述注视点中心距矩阵中每行数据的中位数，构成下述注视点中心距的中位数矩阵W，其中，
Figure PCTCN2015087641-appb-000004
其中，Median()为求中位数的函数；
步骤S23，组合所述瞳孔直径变化率和注视点中心距的中位数。
根据本发明的又一个方面，所述步骤S3还包括：
步骤S31，对所述步骤S1之前采集到的网上用户进行搜索状态或浏览状态过程中的眼动数据和特征提取，按第一列为训练或者预测样本的标签值，其他列为对应特征的特征值构成如下组合特征数据集：
Figure PCTCN2015087641-appb-000005
其中，label为训练数据集的目标值；index为从1开始的整数，表示特征的序号； value为用于训练或预测的数据，即特征值，其中搜索状态标记为1，浏览状态标记为2；
步骤S32，将所述步骤S31中的组合特征数据集输入到支持向量机中，训练得到状态分类器；
步骤S33，将所述步骤S2中得到的多个眼动特征参数输入到所述步骤S32中得到的所述状态分类器中，识别所述网上用户的当前工作状态为搜索状态或浏览状态。
本发明提供的基于眼动数据的网上用户状态识别方法，解决了在基于视觉行为人机接口领域，仍然存在无法识别网上用户工作状态的问题。本发明针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了基于支持向量机的分类方法，学习速度快，分类正确率高，具有较高的识别效率。本发明可以根据网上用户观看网页时的眼动信息，识别出用户的工作状态是有目的的搜索状态还是无目的的浏览状态。本发明基于眼动数据的网上用户工作状态识别的方法和系统实现了一种以眼动为方式的人机交互，通过用户的眼动数据来识别用户的工作状态。
本发明的另一个目的是提供一种基于眼动数据的网上用户状态识别装置，该装置针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了诸如基于支持向量机SVM的分类方法，解决了高效准确的机器识别用户状态的技术问题,其学习速度快，分类正确率高，具有较高的识别效率。
为实现上述目的，本发明提供一种基于眼动数据的网上用户状态识别装置，包括：数据采集和预处理单元，用于采集网上用户在观看网页的过程中的眼动数据，并对所述眼动数据进行预处理，得到预处理后的眼动数据；特征提取单元，用于对所述预处理后的眼动数据进行特征提取，获取多个眼动特征参数；状态判断单元，用于将所述网上用户的所述多个眼动特征参数输入到预设的状态分类器中进行比对，以识别所述网上用户的当前工作状态为搜索状态或浏览状态。
根据本发明的一个方面，所述数据采集和预处理单元包括红外线摄像机，所述红外线摄像机用于采集所述网上用户在观看网页的过程中的眼动数据。
根据本发明的一个方面，所述眼动数据包括：所述网上用户的注视点位置、注视时间和瞳孔直径。
根据本发明的一个方面，所述多个眼动特征参数包括瞳孔直径变化率和注视点中心距的中位数，所述特征提取单元还包括：瞳孔直径变化率计算子单元和注视点中心距的中位数计算子单元，其中，
所述瞳孔直径变化率计算子单元用于根据每个眼动数据中的瞳孔直径构成瞳孔直径数据集矩阵u，其中，
Figure PCTCN2015087641-appb-000006
其中，m为所述眼动数据中的注视点的个数，n为在m个注视点中选取的n个采样点，uij为第i个注视点第j次采样的瞳孔直径值，其中，1≤i≤m，1≤j≤n；
所述瞳孔直径变化率计算子单元在所述瞳孔直径数据集矩阵u按行求均值，减去瞳孔直径的基线值D0，除以所述瞳孔直径的基线值D0，得到下述瞳孔直径变化率矩阵U，
Figure PCTCN2015087641-appb-000007
其中，D0为瞳孔直径的基线值；
所述注视点中心距的中位数计算子单元用于根据每个所述注视点位置计算注视点中心距CDfk，其中，
Figure PCTCN2015087641-appb-000008
其中，注视点的坐标为(Xfk,Yfk)，网页中心坐标为(X0,Y0))；
所述注视点中心距的中位数计算子单元设所述网上用户观看网页数量为p，每个网页选取q个注视点，则构成注视点中心距矩阵，计算所述注视点中心距矩阵中每行数据的中位数，构成下述注视点中心距的中位数矩阵W，其中，
Figure PCTCN2015087641-appb-000009
其中，Median()为求中位数的函数。
根据本发明的又一个方面，所述状态判断单元还包括：对前一次采集到的网上用户进行搜索状态或浏览状态过程中的眼动数据和特征提取，按第一列为训练或者预测样本的标签值，其他列为对应特征的特征值构成如下组合特征数据集：
Figure PCTCN2015087641-appb-000010
其中，label为训练数据集的目标值；index为从1开始的整数，表示特征的序号；value为用于训练或预测的数据，即特征值，其中搜索状态标记为1，浏览状态标记为2；
所述状态判断单元将所述数据采集和预处理单元采集到的组合特征数据集输入到支持向量机中，训练得到状态分类器，然后将所述数据采集和预处理单元得到的多个眼动特征参数输入到所述状态分类器中，识别所述网上用户的当前工作状态为搜索状态或浏览状态。
本发明提供的基于眼动数据的网上用户状态识别装置，解决了在基于视觉行为人机接 口领域，仍然存在无法识别网上用户工作状态的问题。本发明针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了基于支持向量机的分类方法，学习速度快，分类正确率高，具有较高的识别效率。本发明可以根据网上用户观看网页时的眼动信息，识别出用户的工作状态是有目的的搜索状态还是无目的的浏览状态。本发明基于眼动数据的网上用户工作状态识别的方法和系统实现了一种以眼动为方式的人机交互，通过用户的眼动数据来识别用户的工作状态。
附图说明
图1是根据本发明的基于眼动数据的网上用户状态识别方法的一实施方式的流程图；
图2是根据本发明的对眼动数据进行特征提取的一实施方式的流程图；
图3是根据本发明的方法中识别所述网上用户的当前工作状态的一实施方式的流程图；
图4是根据本发明的方法中基于眼动数据的网上用户状态识别装置的一实施方式的结构示意图；
图5是根据本发明的装置中特征提取单元的一实施方式的结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚明了，下面结合具体实施方式并参照附图，对本发明进一步详细说明。应该理解，这些描述只是示例性的，而并非要限制本发明的范围。此外，在以下说明中，省略了对公知结构和技术的描述，以避免不必要地混淆本发明的概念。
下面首先对本发明实施例的基于眼动数据的网上用户状态识别方法和装置的技术背景内容进行说明。
鉴于被动感知技术不能满足人们对于机器智能化的需要，本采用自动感知技术实现人机交互智能的需要，其特点是机器能够主动感知用户需求并为用户提供信息服务。
具体地，作为网页的人机界面，如果能够获取用户网上的行为和用户的心理状态，则能给用户提供更好的服务。其中，视觉搜索和浏览是用户上网的两种主要的视觉行为方式。
视觉搜索是用户带有明确的搜索目标，有目的地在网页上查找需求信息。视觉浏览是用户没有明确目标，自由浏览和查看网页上的信息。网页上的信息搜索过程会受信息呈现形式、信息布局、以及浮动广告等多种因素的影响。由于，越来越多的网页设计采用了浮动广告的设计理念，而在网上用户搜索网页内容的过程中浮动广告等内容会对用户产生负面影响，分散用户的注意力，使人产生厌恶情绪从而降低用户的搜索效率。如果能够区分网上用户的工作状态，适时的增加或者减少浮动广告会有利于用户关注浮动广告内容并且提高搜索效率。
网上用户观看网页过程中的眼动信息在不同状态下会分别表现出不同的特征，可以识别眼动信息的特征，判断用户在观看网页过程中的思维状态，进行交互。
由此，本发明实施例的基于眼动数据的网上用户状态识别方法和装置，可以根据网上用户的眼动数据特征定义及其相应的特征数据组合生成人机交互信息并利用支持向量机SVM进行状态识别，判断该网上用户是在搜索网页还是浏览网页。
如图1所示，本发明提供一种基于眼动数据的网上用户状态识别方法的一具体实施例，包括如下步骤：
步骤S1，采集网上用户在搜索网页中特定目标和随机浏览网页两种状态下的眼动数据，并对眼动数据进行预处理，得到预处理后的眼动数据。
在一个实施方式中,具体地，利用眼动追踪装置，获取网上用户在搜索和浏览网页两种状态下的眼动数据。其中，眼动追踪装置可以为型号为Tobii T120眼动仪，利用眼动 仪中的红外线摄像机以120Hz的采样频率采集网上用户在两种状态下的眼动数据。其中，眼动数据,是指与眼球运动相关的数据，包括但不限于与注视、眼跳和追随等眼球运动(或说眼球运动模式)等相关的数据。这里，眼动数据包括：网上用户的注视点(即用户查看网页时眼球不动时位于网页的点)位置、注视时间和瞳孔直径。
进一步，每个注视点的注视信息数据集F包含所有的注视信息fk，即与“注视”被观察对象的这类眼球运动信息相关的数据。该fk是一个五元数组(tfk，dlk，drk，xfk,yfk),tfk为注视点的注视时间，dlk为左眼瞳孔直径，drk为右眼瞳孔直径，xfk为注视点的横坐标，yfk为注视点的纵坐标。一种方式，可以按照处理瞳孔直径的通用方法，采用左右眼瞳孔直径的平均值代表该注视点的瞳孔直径值，即算得dfk＝(dlk+drk)/2，将fk简化为四元组(tfk，dfk,xfk,yfk)。
进一步，眼动跟踪技术能够实时记录用户在网页上的眼球运动情况。通过分析眼动轨迹(眼球的运动轨迹)、瞳孔直径等根据视觉感官反应的眼球的行为指标，能更加客观的反映用户的行为，从而揭示人们在网页上的心理状态。
对上述眼动数据进行预处理，包括剔除采集到的眼动数据中的极端数据和三倍标准差以外的异常数据，得到预处理后的眼动数据。
步骤S2，对预处理后的眼动数据进行特征提取，获取多个眼动特征参数。
一个实施方式中，可以分别选定每一个注视点，该注视点对应的注视信息中包含注视时间信息。针对眼动数据，计算每个注视点在注视时间内的瞳孔直径变化率和注视点中心距。其中，多个眼动特征参数包括瞳孔直径变化率和注视点中心距的中位数。
在一个实施例中，可以根据用户的注视点位置数据计算用户的注视点中心距，比如，注视点坐标(xfk，yfk)和网页中心坐标间的欧式距离为注视点中心距；可以根据用户的瞳孔直径dfk大小计算用户在观看网页过程中的瞳孔直径大小随时间的变化曲线；进而，计算出网页中所有注视点的中位数和每个注视点的瞳孔直径变化率作为特征参数。
如图2所示，所述步骤S2还可以包括如下步骤：
步骤S21，计算瞳孔直径变化率，得到瞳孔直径变化率特征数据集。
在一个实施方式中，注视点在注视时间内的采样次数为注视时间除以采样周期，其中可以预设采样周期例如为：8毫秒。
根据每个眼动数据中的瞳孔直径构成瞳孔直径数据集矩阵u，其中，
Figure PCTCN2015087641-appb-000011
其中，m为眼动数据中的注视点的个数，n为在第m个注视点中选取的n个采样点，而uij为第i个注视点第j次采样的瞳孔直径值，其中，1≤i≤m，1≤j≤n；
在瞳孔直径数据集矩阵u按行求均值，减去瞳孔直径的基线值D0，除以瞳孔直径的基线值D0，得到下述m行一列瞳孔直径变化率矩阵U，
Figure PCTCN2015087641-appb-000012
其中，D0为瞳孔直径的基线值。
步骤S22，计算注视点中心距的中位数，得到注视点中心距特征数据集。根据每个注视点位置计算注视点中心距CDfk，其中，
Figure PCTCN2015087641-appb-000013
其中，注视点的坐标为(Xfk,Yfk)，网页中心坐标为(X0,Y0)。注视点信息数据集可以简化为三元组(tfk，dfk,cdfk)。
设网上用户观看网页数量为p，每个网页选取q个注视点，则构成注视点中心距矩阵，计算注视点中心距矩阵中每行数据的中位数，构成下述p行一列注视点中心距的中位数矩阵W，其中，
Figure PCTCN2015087641-appb-000014
其中，Median()为求中位数的函数；
步骤S23，组合瞳孔直径变化率和注视点中心距的中位数。
步骤S3，将网上用户的多个眼动特征参数输入到预设的状态分类器中进行比对，以识别网上用户的当前工作状态为搜索状态或浏览状态。
在一个实施方式中，将网上用户在观看网页的过程中的工作状态分为两个状态类型，一个为搜索状态，一个为浏览状态。获取用户在观看网页过程中的眼动数据会在这两个状态中分别表现出不同的特征，当网上用户处于搜索状态时，相较于浏览状态，在眼动数据上会表现出瞳孔直径变化率减小，注视点中心距呈现周边特性的特征。因此获取到这两个状态的眼动数据的不同特征，即可以判断用户在观看网页过程中所处的状态。
如图3所示，所述步骤S3还可以包括如下步骤：
步骤S31，对步骤S1之前采集到的网上用户进行搜索状态或浏览状态过程中的眼动数据和特征提取。一种方式为：对搜索任务的眼动数据进行特征提取，得到搜索状态下的眼动特征参数；对浏览任务的眼动数据进行特征提取，得到浏览状态下的眼动特征参数。进而，按第一列为训练或者预测样本的标签值，其他列为对应特征的特征值构成如下组合特征数据集：
Figure PCTCN2015087641-appb-000015
其中，label为训练数据集的目标值；index为从1开始的整数，表示特征的序号；value为用于训练或预测的数据，即特征值，其中搜索状态标记为1，浏览状态标记为2。
步骤S32，将步骤S31中的组合特征数据集进行机器学习，以训练得到状态分类器。一种方式为：将组合特征数据集输入到支持向量机SVM中，训练得到状态分类器。例如，可以对搜索状态和浏览状态的眼动特征参数输入到支持向量机SVM中进行训练，得到训练过的状态分类器，从而实现后续对网络用户进行搜索或浏览的视觉行为的识别。其中，利用支持向量机对所得的眼动特征参数进行训练，训练得到状态分类器，一个具体实施例可以是：利用带有类别标签的特征参数数据集作为训练样本集，选择径向基函数作为核函数，训练时使用五折交叉验证，即训练样本数为总样本的4/5，测试样本为总样本的1/5，用网格搜索法选择最佳的参数C和gamma。进一步，状态分类器可以采用多种形式，除了上述例子中采用的支持向量机(SVM分类器)进行训练得到外、神经网络、遗传算法等其他机器学习算法也可以实现分类器训练。优选地，状态分类器还可以为libsvm支持向量机工具包等。
步骤S33，将步骤S2中得到的多个眼动特征参数(包括实时的或已经采集的)输入到步骤S32中得到的状态分类器中，识别网上用户的当前工作状态为搜索状态或浏览状态。
在一个实施方式中，将步骤S1中提取的眼动特征数据按步骤S2中数据集格式整理，输入到训练好的状态分类器中，从而判断网上用户的工作状态为搜索还是浏览，即识别出网上用户的工作状态类型。例如：支持向量机的输出为1时，则对应为搜索状态；输出为2时，则对应为浏览状态。
综上，本实施例采用基于眼动数据的网上用户状态识别方法，解决了在基于视觉行为人机接口领域，仍然存在无法识别网上用户工作状态的问题。本发明针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了基于支持向量机的分类方法，学习速度快，分类正确率高，具有较高的识别效率。本发明可以根据网上用户观看网页时的眼动信息，识别出用户的工作状态是有目的的搜索状态还是无目的的浏览状态。本发明基于眼动数据的网上用户工作状态识别的方法和系统实现了一种以眼动为方式的人机交互，通过用户的眼动数据来识别用户的工作状态。
本发明还提出一种基于眼动数据的网上用户状态识别装置，参见图4所示的装置的一个实施例结构示意图，包括：数据采集和预处理单元100、特征提取单元200和状态判断单元300。
具体地，数据采集和预处理单元100用于采集网上用户在观看网页的过程中的眼动数据，并对眼动数据进行预处理，得到预处理后的眼动数据。
在本发明的实施例中，数据采集和预处理单元100包括红外线摄像机，红外线摄像机用于采集网上用户在观看网页的过程中的眼动数据。其中，眼动数据包括：网上用户的注视点位置、注视时间和瞳孔直径。
在一个实施方式中,具体地，利用眼动追踪装置，获取网上用户观看网页时的眼动数据。其中，眼动追踪装置可以为型号为Tobii T120眼动仪，利用眼动仪中的红外线摄像机以120Hz的采样频率采集网上用户在观看网页的过程中的眼动数据。其中，眼动数据,是指与眼球运动相关的数据，包括但不限于与注视、眼跳和追随等眼球运动(或说眼球运动模式)等相关的数据。这里，眼动数据包括：网上用户的注视点(即用户查看网页时眼 球不动时位于网页的点)位置、注视时间和瞳孔直径。
进一步，每个注视点的注视信息数据集F包含所有的注视信息fk，即与“注视”被观察的对象的这类眼球运动信息相关的数据。该fk是一个五元数组(tfk，dlk，drk，xfk，yfk),tfk为注视点的注视时间，dlk为左眼瞳孔直径，drk为右眼瞳孔直径，xfk为注视点的横坐标，yfk为注视点的纵坐标。一种方式，可以按照处理瞳孔直径的通用方法，采用左右眼瞳孔直径的平均值代表该注视点的瞳孔直径值，即算得dfk＝(dlk+drk)/2，将fk简化为四元组(tfk，dfk,xfk,yfk)。
进一步，眼动跟踪技术能够实时记录用户在网页上的眼球运动情况。通过分析眼动轨迹(眼球的运动轨迹)、瞳孔直径等根据视觉感官反应的眼球的行为指标，能更加客观的反映用户的行为，从而揭示人们在网页上的心理状态。
数据采集和预处理单元100对上述眼动数据进行预处理，包括剔除采集到的眼动数据中的极端数据和三倍标准差以外的异常数据，得到预处理后的眼动数据。
数据采集和预处理单元100的具体实施和处理如步骤S1的描述。
特征提取单元200连接至数据采集和预处理单元100，用于对预处理后的眼动数据进行特征提取，获取多个眼动特征参数。一个实施方式中，可以分别选定每一个注视点，该注视点对应的注视信息中包含注视时间信息。针对眼动数据，计算每个注视点在注视时间内的瞳孔直径变化率、注视频率和注视点中心距。其中，多个眼动特征参数包括瞳孔直径变化率和注视点中心距的中位数。
如图5所示，特征提取单元200包括：瞳孔直径变化率计算子单元210和注视点中心距的中位数计算子单元220。
瞳孔直径变化率计算子单元210计算瞳孔直径变化率，得到瞳孔直径变化率特征数据集。在一个实施方式中，注视点在注视时间内的采样次数为注视时间除以采样周期，其中可以预设采样周期例如为：8毫秒。具体地，瞳孔直径变化率计算子单元210用于根据每
个眼动数据中的瞳孔直径构成瞳孔直径数据集矩阵u，其中，
Figure PCTCN2015087641-appb-000016
其中，m为眼动数据中的注视点的个数，n为在第m个注视点中选取的n个采样点，而uij为第i个注视点第j次采样的瞳孔直径值，其中，1≤i≤m，1≤j≤n；
瞳孔直径变化率计算子单元210在瞳孔直径数据集矩阵u按行求均值，减去瞳孔直径的基线值D0，除以瞳孔直径的基线值D0，得到下述瞳孔直径变化率矩阵U，
Figure PCTCN2015087641-appb-000017
其中，D0为瞳孔直径的基线值。
注视点中心距的中位数计算子单元220用于根据每个注视点位置计算注视点中心距CDfk，其中，
Figure PCTCN2015087641-appb-000018
其中，注视点的坐标为(Xfk,Yfk)，网页中心坐标为(X0,Y0)，注视点信息数据集可以简化为三元组(tfk，dfk,cdfk)。
注视点中心距的中位数计算子单元220设网上用户观看网页数量为p，每个网页选取q个注视点，则构成注视点中心距矩阵，计算注视点中心距矩阵中每行数据的中位数，构成下述注视点中心距的中位数矩阵W，其中，
Figure PCTCN2015087641-appb-000019
其中，Median()为求中位数的函数。
特征提取单元200进一步组合上述瞳孔直径变化率和注视点中心距的中位数。
特征提取单元200的具体实施和处理如步骤S2的描述。
状态判断单元300连接至特征提取单元200，用于将网上用户的多个眼动特征参数输入到预设的状态分类器中进行比对，以识别网上用户的当前工作状态为搜索状态或浏览状态。
状态判断单元300对前一次采集到的网上用户进行搜索状态或浏览状态过程中的眼动数据和特征提取。一种方式为：对搜索任务的眼动数据进行特征提取，得到搜索状态下的眼动特征参数；对浏览任务的眼动数据进行特征提取，得到浏览状态下的眼动特征参数。进而，按第一列为训练或者预测样本的标签值，其他列为对应特征的特征值构成如下组合特征数据集：
Figure PCTCN2015087641-appb-000020
其中，label为训练数据集的目标值；index为从1开始的整数，表示特征的序号；value为用于训练或预测的数据，即特征值，其中搜索状态标记为1，浏览状态标记为2。
状态判断单元300将数据采集和预处理单元采集到的组合特征数据集进行机器学习，以训练得到状态分类器。一种方式为：将组合特征数据集输入到支持向量机SVM中，训练得到状态分类器，然后将数据采集和预处理单元得到的多个眼动特征参数输入到状态分类器中，识别网上用户的当前工作状态为搜索状态或浏览状态。
具体地，状态判断单元300对搜索状态和浏览状态的眼动特征参数输入到支持向量机SVM中进行训练，得到训练过的状态分类器，从而实现后续对网络用户进行搜索或浏览的视觉行为的识别。其中，利用支持向量机对所得的眼动特征参数进行训练，训练得到状态 分类器，一个具体实施例可以是：利利用带有类别标签的特征参数数据集作为训练样本集，选择径向基函数作为核函数，训练时使用五折交叉验证，即训练样本数为总样本的4/5，测试样本为总样本的1/5，用网格搜索法选择最佳的参数C和gamma。
进一步，状态分类器可以采用多种形式，除了上述例子中采用的支持向量机(SVM分类器)进行训练得到外、神经网络、遗传算法等其他机器学习算法也可以实现分类器训练。优选地，状态分类器还可以为libsvm支持向量机工具包等。
状态判断单元300将多个眼动特征参数(包括实时的或已经采集的)输入到状态分类器中，识别网上用户的当前工作状态为搜索状态或浏览状态。在一个实施方式中，将提取的眼动特征数据按特征提取单元200中的方式进行数据集格式整理，输入到训练好的状态分类器中，从而判断网上用户的工作状态为搜索还是浏览，即识别出网上用户的工作状态类型。例如：支持向量机的输出为1时，则对应为搜索状态；输出为2时，则对应为浏览状态。
状态判断单元300的具体实施和处理如步骤S3的描述。
本实施例采用基于眼动数据的网上用户状态识别装置，解决了在基于视觉行为人机接口领域，仍然存在无法识别网上用户工作状态的问题。本发明针对网上用户搜索和浏览两种工作状态进行识别，采用主动感知技术提取两种眼动特征，同时采用了基于支持向量机的分类方法，学习速度快，分类正确率高，具有较高的识别效率。本发明可以根据网上用户观看网页时的眼动信息，识别出用户的工作状态是有目的的搜索状态还是无目的的浏览状态。本发明基于眼动数据的网上用户工作状态识别的方法和系统实现了一种以眼动为方式的人机交互，通过用户的眼动数据来识别用户的工作状态。
由于本实施例的系统所实现的处理及功能基本相应于前述图1～图3所示的方法实施例，故本实施例的描述中未详尽之处，可以参见前述实施例中的相关说明，在此不做赘述。
应当理解的是，本发明的上述具体实施方式仅仅用于示例性说明或解释本发明的原理，而不构成对本发明的限制。因此，在不偏离本发明的精神和范围的情况下所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。此外，本发明所附权利要求旨在涵盖落入所附权利要求范围和边界、或者这种范围和边界的等同形式内的全部变化和修改例。

Claims (10)

  1. 一种基于眼动数据的网上用户状态识别方法，包括如下步骤：
    步骤S1，采集网上用户在观看网页的过程中的眼动数据，并对所述眼动数据进行预处理，得到预处理后的眼动数据；
    步骤S2，对所述预处理后的眼动数据进行特征提取，获取多个眼动特征参数；
    步骤S3，将所述网上用户的所述多个眼动特征参数输入到预设的状态分类器中进行比对，以识别所述网上用户的当前工作状态为搜索状态或浏览状态。
  2. 根据权利要求1所述的基于眼动数据的网上用户状态识别方法，其中，采用红外线摄像机采集所述网上用户在观看网页的过程中的眼动数据。
  3. 根据权利要求1或2所述的基于眼动数据的网上用户状态识别方法，其中，所述眼动数据包括：所述网上用户的注视点位置、注视时间和瞳孔直径。
  4. 根据权利要求3所述的基于眼动数据的网上用户状态识别方法，其中，所述多个眼动特征参数包括瞳孔直径变化率和注视点中心距的中位数，所述步骤S2还包括：
    步骤S21，根据每个眼动数据中的瞳孔直径构成瞳孔直径数据集矩阵u，其中，
    Figure PCTCN2015087641-appb-100001
    其中，m为所述眼动数据中的注视点的个数，n为在m个注视点中选取的n个采样点，uij为第i个注视点第j次采样的瞳孔直径值，其中，1≤i≤m，1≤j≤n；
    在所述瞳孔直径数据集矩阵u按行求均值，减去瞳孔直径的基线值D0，除以所述瞳孔直径的基线值D0，得到下述瞳孔直径变化率矩阵U，
    Figure PCTCN2015087641-appb-100002
    其中，D0为瞳孔直径的基线值，
    步骤S22，根据每个所述注视点位置计算注视点中心距CDfk，其中，
    Figure PCTCN2015087641-appb-100003
    其中，注视点的坐标为(Xfk,Yfk)，网页中心坐标为(X0,Y0)；
    设所述网上用户观看网页数量为p，每个网页选取q个注视点，则构成注视点中 心距矩阵，计算所述注视点中心距矩阵中每行数据的中位数，构成下述注视点中心距的中位数矩阵W，其中，
    Figure PCTCN2015087641-appb-100004
    其中，Median()为求中位数的函数；
    步骤S23，组合所述瞳孔直径变化率和注视点中心距的中位数。
  5. 根据权利要求4所述的基于眼动数据的网上用户状态识别方法，所述步骤S3还包括：
    步骤S31，对所述步骤S1之前采集到的网上用户进行搜索状态或浏览状态过程中的眼动数据和特征提取，按第一列为训练或者预测样本的标签值，其他列为对应特征的特征值构成如下组合特征数据集：
    Figure PCTCN2015087641-appb-100005
    其中，label为训练数据集的目标值；index为从1开始的整数，表示特征的序号；value为用于训练或预测的数据，即特征值，其中搜索状态标记为1，浏览状态标记为2；
    步骤S32，将所述步骤S31中的组合特征数据集输入到支持向量机中，训练得到状态分类器；
    步骤S33，将所述步骤S2中得到的多个眼动特征参数输入到所述步骤S32中得到的所述状态分类器中，识别所述网上用户的当前工作状态为搜索状态或浏览状态。
  6. 一种基于眼动数据的网上用户状态识别装置，包括：
    数据采集和预处理单元，用于采集网上用户在观看网页的过程中的眼动数据，并对所述眼动数据进行预处理，得到预处理后的眼动数据；
    特征提取单元，用于对所述预处理后的眼动数据进行特征提取，获取多个眼动特征参数；
    状态判断单元，用于将所述网上用户的所述多个眼动特征参数输入到预设的状态分类器中进行比对，以识别所述网上用户的当前工作状态为搜索状态或浏览状态。
  7. 根据权利要求6所述的基于眼动数据的网上用户状态识别装置，其中，所述数据采集和预处理单元包括红外线摄像机，所述红外线摄像机用于采集所述网上用户在观看网页的过程中的眼动数据。
  8. 根据权利要求6或7所述的基于眼动数据的网上用户状态识别装置，其中，所述眼动数据包括：所述网上用户的注视点位置、注视时间和瞳孔直径。
  9. 根据权利要求8所述的基于眼动数据的网上用户状态识别装置，其中，所述多个眼动特征参数包括瞳孔直径变化率和注视点中心距的中位数，所述特征提取单元还包括：瞳孔直径变化率计算子单元和注视点中心距的中位数计算子单元，其中，
    所述瞳孔直径变化率计算子单元用于根据每个眼动数据中的瞳孔直径构成瞳孔直径数据集矩阵u，其中，
    Figure PCTCN2015087641-appb-100006
    其中，m为所述眼动数据中的注视点的个数，n为在m个注视点中选取的n个采样点，uij为第i个注视点第j次采样的瞳孔直径值，其中，1≤i≤m，1≤j≤n；
    所述瞳孔直径变化率计算子单元在所述瞳孔直径数据集矩阵u按行求均值，减去瞳孔直径的基线值D0，除以所述瞳孔直径的基线值D0，得到下述瞳孔直径变化率矩阵U，
    Figure PCTCN2015087641-appb-100007
    其中，D0为瞳孔直径的基线值；
    所述注视点中心距的中位数计算子单元用于根据每个所述注视点位置计算注视点中心距CDfk，其中，
    Figure PCTCN2015087641-appb-100008
    其中，注视点的坐标为(Xfk,Yfk)，网页中心坐标为(X0,Y0)；
    所述注视点中心距的中位数计算子单元设所述网上用户观看网页数量为p，每个网页选取q个注视点，则构成注视点中心距矩阵，计算所述注视点中心距矩阵中每行数据的中位数，构成下述注视点中心距的中位数矩阵W，其中，
    Figure PCTCN2015087641-appb-100009
    其中，Median()为求中位数的函数。
  10. 根据权利要求9所述的基于眼动数据的网上用户状态识别装置，其中，所述状态判断单元还包括:
    对前一次采集到的网上用户进行搜索状态或浏览状态过程中的眼动数据和特征提取，按第一列为训练或者预测样本的标签值，其他列为对应特征的特征值构成如下组合特征数据集：
    Figure PCTCN2015087641-appb-100010
    其中，label为训练数据集的目标值；index为从1开始的整数，表示特征的序号；value为用于训练或预测的数据，即特征值，其中搜索状态标记为1，浏览状态标记为2；
    所述状态判断单元将所述数据采集和预处理单元采集到的组合特征数据集输入到支持向量机中，训练得到状态分类器，然后将所述数据采集和预处理单元得到的多个眼动特征参数输入到所述状态分类器中，识别所述网上用户的当前工作状态为搜索状态或浏览状态。
PCT/CN2015/087641 2015-01-14 2015-08-20 一种基于眼动数据的网上用户状态识别方法和装置 Ceased WO2016112690A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2015100195184 2015-01-14
CN201510019518.4A CN104504390B (zh) 2015-01-14 2015-01-14 一种基于眼动数据的网上用户状态识别方法和装置

Publications (1)

Publication Number Publication Date
WO2016112690A1 true WO2016112690A1 (zh) 2016-07-21

Family

ID=52945786

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/087641 Ceased WO2016112690A1 (zh) 2015-01-14 2015-08-20 一种基于眼动数据的网上用户状态识别方法和装置

Country Status (2)

Country Link
CN (1) CN104504390B (zh)
WO (1) WO2016112690A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109144262A (zh) * 2018-08-28 2019-01-04 广东工业大学 一种基于眼动的人机交互方法、装置、设备及存储介质
CN111568367A (zh) * 2020-05-14 2020-08-25 中国民航大学 一种识别和量化眼跳入侵的方法
CN112869744A (zh) * 2021-01-20 2021-06-01 上海市精神卫生中心（上海市心理咨询培训中心） 一种精神分裂症辅助诊断方法、系统和存储介质
CN113349776A (zh) * 2020-03-03 2021-09-07 朴光宇 一种提高眼动注视坐标精度技术
CN113970965A (zh) * 2020-07-21 2022-01-25 华为技术有限公司 消息显示方法和电子设备
CN114333042A (zh) * 2022-01-04 2022-04-12 中国民航大学 管制员工作模式的感知方法和装置、电子设备、存储介质
CN115424065A (zh) * 2022-08-31 2022-12-02 北京科技大学 一种充填场景下的深锥浓密机状态分类方法
CN115607159A (zh) * 2022-12-14 2023-01-17 北京科技大学 基于眼动序列时空特征分析的抑郁状态鉴别方法及装置

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104504390B (zh) * 2015-01-14 2017-09-19 北京工业大学 一种基于眼动数据的网上用户状态识别方法和装置
CN106073805B (zh) * 2016-05-30 2018-10-19 南京大学 一种基于眼动数据的疲劳检测方法和装置
CN107544660B (zh) * 2016-06-24 2020-12-18 联想(北京)有限公司 一种信息处理方法及电子设备
CN106200905B (zh) 2016-06-27 2019-03-29 联想(北京)有限公司 信息处理方法及电子设备
CN108338798B (zh) * 2017-01-25 2020-11-06 北京数字新思科技有限公司 一种注意力测量方法和系统
CN106920129B (zh) * 2017-03-09 2020-05-05 山东师范大学 一种基于眼动追踪的网络广告效果测评系统及其方法
CN107341468B (zh) * 2017-06-30 2021-05-04 北京七鑫易维信息技术有限公司 驾驶员状态识别方法、装置、存储介质及处理器
CN108985164A (zh) * 2018-06-11 2018-12-11 厦门大学 基于物体运动状态的眼部关注预测方法
CN108921199A (zh) * 2018-06-11 2018-11-30 厦门大学 基于物体表征状态的眼部关注偏好预测方法
CN109117711B (zh) * 2018-06-26 2021-02-19 西安交通大学 基于眼动数据的分层特征融合的专注度检测装置及方法
CN109189222B (zh) * 2018-08-28 2022-01-11 广东工业大学 一种基于检测瞳孔直径变化的人机交互方法及装置
CN109259775A (zh) * 2018-08-30 2019-01-25 清华大学 一种人脸刺激范式生成方法及自闭症分析系统
CN109199412B (zh) * 2018-09-28 2021-11-09 南京工程学院 基于眼动数据分析的异常情绪识别方法
CN109960892B (zh) * 2019-04-04 2020-09-01 北京理工大学 一种基于眼动信号的cad指令生成方法及系统
CN111475391B (zh) * 2020-04-03 2024-04-16 中国工商银行股份有限公司 眼动数据处理方法、装置及系统
CN111708434B (zh) * 2020-06-03 2023-09-29 东南大学 一种结合眼动识别的网上答题反馈系统
CN111966223B (zh) * 2020-08-17 2022-06-28 陈涛 非感知的mr眼镜人机识别方法、系统、设备及存储介质
CN113180667B (zh) * 2021-04-29 2022-09-23 南昌虚拟现实研究院股份有限公司 认知负荷评估方法、装置、存储介质及计算机设备
CN117460126B (zh) * 2023-10-27 2024-07-05 石家庄铁道大学 基于乘客舒适度的地铁站台光环境设计方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103336576A (zh) * 2013-06-28 2013-10-02 优视科技有限公司 一种基于眼动追踪进行浏览器操作的方法及装置
US20130325463A1 (en) * 2012-05-31 2013-12-05 Ca, Inc. System, apparatus, and method for identifying related content based on eye movements
CN104504390A (zh) * 2015-01-14 2015-04-08 北京工业大学 一种基于眼动数据的网上用户状态识别方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AUPR872301A0 (en) * 2001-11-08 2001-11-29 Sleep Diagnostics Pty Ltd Alertness monitor
JP5302193B2 (ja) * 2007-06-27 2013-10-02 パナソニック株式会社 ヒト状態推定装置およびその方法
CN103324287B (zh) * 2013-06-09 2016-01-20 浙江大学 基于眼动和笔触数据的计算机辅助草图绘制的方法和系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130325463A1 (en) * 2012-05-31 2013-12-05 Ca, Inc. System, apparatus, and method for identifying related content based on eye movements
CN103336576A (zh) * 2013-06-28 2013-10-02 优视科技有限公司 一种基于眼动追踪进行浏览器操作的方法及装置
CN104504390A (zh) * 2015-01-14 2015-04-08 北京工业大学 一种基于眼动数据的网上用户状态识别方法和装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LI, MI ET AL.: "Exploring Visual Search and Browsing Strategies on Web Pages Using the Eye-Tracking", JOURNAL OF BEIJING UNIVERSITY OF TECHNOLOGY, vol. 37, no. 5, 31 May 2011 (2011-05-31) *
YAN, HUIXIA: "Study of Decoding Mental State Based on Eye Tracks Using SVM", ELECTRONIC TECHNOLOGY & INFORMATION SCIENCE , CHINA MASTER'S THESES FULL-TEXT DATABASE, 15 October 2010 (2010-10-15) *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109144262A (zh) * 2018-08-28 2019-01-04 广东工业大学 一种基于眼动的人机交互方法、装置、设备及存储介质
CN113349776A (zh) * 2020-03-03 2021-09-07 朴光宇 一种提高眼动注视坐标精度技术
CN111568367A (zh) * 2020-05-14 2020-08-25 中国民航大学 一种识别和量化眼跳入侵的方法
CN111568367B (zh) * 2020-05-14 2023-07-21 中国民航大学 一种识别和量化眼跳入侵的方法
CN113970965A (zh) * 2020-07-21 2022-01-25 华为技术有限公司 消息显示方法和电子设备
CN112869744A (zh) * 2021-01-20 2021-06-01 上海市精神卫生中心（上海市心理咨询培训中心） 一种精神分裂症辅助诊断方法、系统和存储介质
CN112869744B (zh) * 2021-01-20 2024-05-31 上海市精神卫生中心（上海市心理咨询培训中心） 一种精神分裂症辅助诊断方法、系统和存储介质
CN114333042A (zh) * 2022-01-04 2022-04-12 中国民航大学 管制员工作模式的感知方法和装置、电子设备、存储介质
CN114333042B (zh) * 2022-01-04 2025-04-04 中国民航大学 管制员工作模式的感知方法和装置、电子设备、存储介质
CN115424065A (zh) * 2022-08-31 2022-12-02 北京科技大学 一种充填场景下的深锥浓密机状态分类方法
CN115607159A (zh) * 2022-12-14 2023-01-17 北京科技大学 基于眼动序列时空特征分析的抑郁状态鉴别方法及装置
CN115607159B (zh) * 2022-12-14 2023-04-07 北京科技大学 基于眼动序列时空特征分析的抑郁状态鉴别方法及装置

Also Published As

Publication number Publication date
CN104504390B (zh) 2017-09-19
CN104504390A (zh) 2015-04-08

Similar Documents

Publication Publication Date Title
WO2016112690A1 (zh) 一种基于眼动数据的网上用户状态识别方法和装置
CN104504404B (zh) 一种基于视觉行为的网上用户类型识别方法及系统
US12053308B2 (en) Brain-computer interface with adaptations for high-speed, accurate, and intuitive user interactions
US20230107040A1 (en) Human-computer interface using high-speed and accurate tracking of user interactions
US9727645B2 (en) Method and system of scoring documents based on attributes obtained from a digital document by eye-tracking data analysis
CN103324287B (zh) 基于眼动和笔触数据的计算机辅助草图绘制的方法和系统
JP2022525829A (ja) 神経筋データに基づく制御スキームのためのシステムおよび方法
Hu et al. Deep neural network-based speaker-aware information logging for augmentative and alternative communication
US20140099623A1 (en) Social graphs based on user bioresponse data
Huang et al. Real‐Time Precise Human‐Computer Interaction System Based on Gaze Estimation and Tracking
CN108647657A (zh) 一种基于多元行为数据的云端教学过程评价方法
CN109726713B (zh) 基于消费级视线追踪仪的用户感兴趣区域检测系统和方法
Revanth et al. Comprehensive SVM based Indian sign language recognition
EP3018558B1 (en) Method and system for detecting objects of interest
CN117036877A (zh) 一种人脸表情及姿态融合的情绪识别方法及系统
CN107346207A (zh) 一种基于隐马尔科夫模型的动态手势切分识别方法
CN120732420A (zh) 结合眼动与视听觉算法的评估与反馈方法及系统
Ahmed et al. A novel approach for recognizing real-time american sign language (ASL) using the hand landmark distance and machine learning algorithms
Adiani et al. Evaluation of webcam-based eye tracking for a job interview training platform: Preliminary results
Yerramreddy et al. Gaze estimation using VGG16 architecture through XGBoost classifier
CN115444422A (zh) 一种基于眼动数据的真实环境心理负荷评估方法和系统
Oyekoya Eye tracking: A perceptual interface for content based image retrieval
CN115857678B (zh) 眼动测试方法、装置、设备及存储介质
CN114863093B (zh) 基于眼动技术的神经网络训练方法及建筑设计方法和系统
Elmatary Intelligent Sign Multi-Language Real-Time Prediction System with Effective Data Preprocessing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15877603

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15877603

Country of ref document: EP

Kind code of ref document: A1