WO2018192312A1 - 一种眼球追踪操作的方法和装置 - Google Patents

一种眼球追踪操作的方法和装置 Download PDF

Info

Publication number
WO2018192312A1
WO2018192312A1 PCT/CN2018/078091 CN2018078091W WO2018192312A1 WO 2018192312 A1 WO2018192312 A1 WO 2018192312A1 CN 2018078091 W CN2018078091 W CN 2018078091W WO 2018192312 A1 WO2018192312 A1 WO 2018192312A1
Authority
WO
WIPO (PCT)
Prior art keywords
eyeball
information
user
unit
operation instruction
Prior art date
Application number
PCT/CN2018/078091
Other languages
English (en)
French (fr)
Inventor
黄建东
Original Assignee
上海耕岩智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海耕岩智能科技有限公司 filed Critical 上海耕岩智能科技有限公司
Publication of WO2018192312A1 publication Critical patent/WO2018192312A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the present invention relates to the field of electronic device applications, and in particular, to a method and apparatus for eyeball tracking operation.
  • touch display panels have been widely used in devices that require human-computer interaction interfaces, such as operating screens of industrial computers, tablet computers, touch screens of smart phones, and the like.
  • human-computer interaction interfaces such as operating screens of industrial computers, tablet computers, touch screens of smart phones, and the like.
  • VR/AR virtual reality or augmented reality
  • the user's eye movements are usually combined with operational instructions, so eye tracking is one of the important human-machine opportunities.
  • Interactive interface technology is one of the important human-machine opportunities.
  • Eye tracking refers to tracking changes in the characteristics of the user's eyeball and the periphery of the eyeball to predict the user's state and needs, and respond to achieve the purpose of the user actively or passively controlling the device with the eye.
  • the VR/AR device or the camera outside the display screen of the mobile device is generally used to capture the change feature of the eyeball feature, because the camera outside the display screen of such device device is often independently disposed at the edge of the device (such as at the top of the mobile phone). ), deviating from the optical axis of the eyeball imaging, and the user's eyeball feature changes very quickly, using the existing off-screen camera can not accurately and quickly capture the user's eyeball feature changes.
  • the current off-screen camera structure is used for eye tracking.
  • the user's eyes focus on the screen during the use of the equipment. If the user's eyeball notices the details of the different positions of the screen, the user begins to perform.
  • Activity because the accuracy of the eyeball feature information collected by the off-screen camera that deviates from the optical axis is not high enough, or the response time is not fast enough, during this period, the picture cannot be accurately and real-time switched according to the angle of focus of the user's eyeball, huge The picture delay brings a bad sensory experience to the user, and even causes symptoms such as dizziness and vomiting.
  • the technical problem to be solved by the present invention is to provide a technical solution for eyeball tracking operation, which is to solve the problem that the eyeball tracking angle is not accurate because the position of the camera outside the display screen deviates from the optical axis, and the characteristic information of the user's eyeball cannot be accurately and timely captured.
  • Human Factors Engineering such as delays, long-term delays, inability to respond quickly, and poor sensory experience.
  • the technical solution adopted by the present invention is: a method for eyeball tracking operation, the method is applied to an apparatus for eyeball tracking operation, the device includes a display unit and a sensing unit; and the display unit is configured
  • the eyeball activity recognition area is located below the eyeball activity recognition area for sensing the light signal reflected by the user's eyeball and capturing the user's eyeball activity information, and the method includes the following steps:
  • Presetting operational configuration information where the operational configuration information includes a correspondence between eyeball activity information and an operation instruction
  • step of “presetting the operation configuration information” includes:
  • the operation instruction identifier list includes identifiers corresponding to one or more operation instructions, and each operation instruction identifier corresponds to an operation instruction;
  • the operation instruction includes one or more of a text operation instruction, an image operation instruction, a video operation instruction, and an application operation instruction.
  • the device further includes an infrared light source for emitting infrared light
  • the sensing unit includes an infrared sensing layer
  • the infrared sensing layer is configured to receive an infrared light signal reflected by a user's eye ball to capture a user's Eye activity information.
  • the sensing unit comprises a TFT image sensing array film
  • the infrared sensing layer is an array formed by an infrared photodiode or an infrared photosensitive transistor.
  • step of “determining whether there is eyeball activity information matching the captured eyeball activity information in the operation configuration information” specifically includes:
  • the feature value is calculated according to the captured eye activity information of the user, and compared with the feature value of the eyeball activity information preset in the configuration information; when the error is less than the preset value, it is determined to be matched, otherwise it is determined as a mismatch.
  • the method further includes the steps of:
  • the prompt information is issued.
  • the prompt information includes one or more of voice prompt information, image prompt information, light prompt information, and video prompt information.
  • the display unit comprises an AMOLED display or an LCD liquid crystal display.
  • a backlight unit is further disposed under the sensing unit, and the sensing unit is disposed between the backlight unit and the LCD liquid crystal display.
  • the eyeball activity recognition area includes a plurality of eyeball activity recognition sub-regions, and a sensing unit is disposed corresponding to each of the eyeball activity recognition sub-regions.
  • the device further includes a sensing unit control circuit, the method further comprising:
  • the sensing unit control circuit turns on the sensing unit below the eyeball activity recognition sub-area, and receives a closing instruction of the user's eyeball activity recognition sub-area, the sensing unit control circuit The sensing unit below the eye activity recognition sub-area is closed.
  • the inventors also provide an apparatus for eyeball tracking operation, the apparatus comprising a display unit and a sensing unit; the display unit is provided with an eyeball activity recognition area, and the sensing unit is located below the eyeball activity recognition area
  • the device further includes an operation information setting unit, a determining unit, and a processing unit;
  • the operation information setting unit is configured to preset operation configuration information, where the operation configuration information includes a correspondence relationship between the eyeball activity information and the operation instruction;
  • the sensing unit is configured to capture eyeball activity information of the user on the eyeball activity recognition area, and the determining unit is configured to determine whether the eyeball activity information matching the captured eyeball activity information exists in the operation configuration information, and if yes, process
  • the unit is configured to execute an operation instruction corresponding to the eyeball activity information in the operation configuration information, otherwise the processing unit does not execute the operation instruction.
  • the device includes an operation instruction receiving unit, and the “operation information setting unit is configured to preset operation configuration information” includes:
  • the operation information setting unit is configured to receive a user setting command, and the display unit is configured to display an eyeball activity recognition area;
  • the operation instruction receiving unit is configured to receive an eyeball operation item selected by a user, and the sensing unit is configured to capture and save an eyeball activity information of the user on the eyeball recognition area;
  • the display unit is configured to display an operation instruction identifier list, where the operation instruction identifier list includes an identifier corresponding to one or more operation instructions, and each operation instruction identifier corresponds to an operation instruction;
  • the operation instruction receiving unit is further configured to receive a selection instruction of the operation instruction identifier by the user, establish a correspondence between the operation instruction corresponding to the selected operation instruction identifier and the captured eye movement information of the user, and save the corresponding relationship.
  • the operation instruction includes one or more of a text operation instruction, an image operation instruction, a video operation instruction, and an application operation instruction.
  • the device further includes an infrared light source for emitting infrared light
  • the sensing unit includes an infrared sensing layer
  • the infrared sensing layer is configured to receive an infrared light signal reflected by a user's eye ball to capture a user's Eye activity information.
  • the operation instruction is a screen switching instruction, and the “determining unit is configured to determine whether the eye movement activity information matching the captured eyeball activity exists in the operation configuration information, and if yes, the processing unit is configured to execute the operation configuration information.
  • the operation instruction corresponding to the eyeball activity information, otherwise the processing unit does not execute the operation instruction includes:
  • the determining unit is configured to determine whether the captured eyeball activity information of the captured user matches the eyeball activity information corresponding to the screen switching instruction, and if so, the processing unit is configured to switch the screen, otherwise the processing unit does not switch the screen.
  • the sensing unit comprises a TFT image sensing array film
  • the infrared sensing layer is an array formed by an infrared photodiode or an infrared photosensitive transistor.
  • the “determining unit is configured to determine whether there is eyeball activity information matching the captured eyeball activity information in the operation configuration information” specifically includes:
  • the determining unit is configured to calculate the feature value according to the captured eye activity information of the user, and compare with the feature value of the eyeball activity information preset in the configuration information; when the error is less than the preset value, the determination is matched, otherwise the determination is For no match.
  • processing unit is further configured to issue the prompt information when the determining unit determines that the eyeball activity information matching the captured eyeball activity information does not exist in the operation configuration information.
  • the prompt information includes one or more of voice prompt information, image prompt information, light prompt information, and video prompt information.
  • the display unit comprises an AMOLED display or an LCD liquid crystal display.
  • a backlight unit is further disposed under the sensing unit, and the sensing unit is disposed between the backlight unit and the LCD liquid crystal display.
  • the eyeball activity recognition area includes a plurality of eyeball activity recognition sub-regions, and a sensing unit is disposed corresponding to each of the eyeball activity recognition sub-regions.
  • the device further includes a sensing unit control circuit and an operation instruction receiving unit, wherein the operation instruction receiving unit is configured to receive a start instruction of the eyeball activity recognition sub-region, and the sensing unit control circuit is configured to open the a sensing unit below the eyeball activity recognition sub-region, and the operation command receiving unit is configured to receive a user's closing instruction for the eyeball activity recognition sub-region, the sensing unit control circuit is configured to close the eyeball activity identifier The sensing unit below the area.
  • the operation instruction receiving unit is configured to receive a start instruction of the eyeball activity recognition sub-region
  • the sensing unit control circuit is configured to open the a sensing unit below the eyeball activity recognition sub-region
  • the operation command receiving unit is configured to receive a user's closing instruction for the eyeball activity recognition sub-region
  • the sensing unit control circuit is configured to close the eyeball activity identifier The sensing unit below the area.
  • the beneficial effect after adopting the above technical solution is that: by providing a sensing unit under the eyeball recognition area of the display unit, a projection of the user's eyeball imaging through the optical device is located on the eyeball recognition area, and the sensing unit is disposed on the eyeball imaging optical axis.
  • the position or the paraxial position is compared with the structure in which the camera is disposed at the edge position independently of the display screen, and the present invention can timely capture the user's eyeball activity information, and then compare with the preset eyeball activity information, and execute the eyeball activity information.
  • the corresponding operation instruction enables the user to quickly control the device through the eyeball activity, thereby effectively improving the user experience.
  • the sensing unit is disposed under the display unit, and the overall thickness of the mobile device can be effectively reduced compared with the structure in which the camera is independently protruded from the display screen area, so that the wearable device or the mobile device is thinner and more suitable.
  • the wearable device or the mobile device is thinner and more suitable.
  • FIG. 1 is a flow chart of a method of eyeball tracking operation according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for eyeball tracking operation according to another embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an apparatus for eyeball tracking operation according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an apparatus for eyeball tracking operation according to another embodiment of the present invention.
  • FIG. 5 is a schematic diagram of an application scenario of an eyeball tracking operation according to an embodiment of the present invention.
  • Figure 6 is a schematic view of a conventional sensing unit
  • FIG. 7 is a schematic diagram of a sensing unit according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a sensing unit according to another embodiment of the present invention.
  • An operation information setting unit
  • An operation instruction receiving unit
  • FIG. 1 is a flowchart of a method for eyeball tracking operation according to an embodiment of the present invention.
  • the method is applied to an apparatus for eyeball tracking operation, and the device is an electronic device having a display screen or a touch display screen, such as a smart mobile device such as a mobile phone, a tablet computer, a personal digital assistant, or a personal computer or a computer for industrial equipment. And other electronic equipment.
  • the device can also be combined with an optical imaging device disposed between the display unit and the user's eyes.
  • the user's eyeball projection is first imaged in the optical imaging device, and the imaged projection is displayed.
  • the area of the eyeball activity recognition area on the unit is captured by the sensing unit under the eye movement recognition, and the effect of the analog VR/AR device can be achieved by the cooperation between the optical imaging device and the display unit.
  • the device includes a display unit and a sensing unit.
  • An eyeball activity recognition area is disposed on the display unit, and the sensing unit is located below the eyeball activity recognition area for sensing an optical signal reflected by the user's eyeball and capturing the user's eyeball activity information.
  • the display unit includes an AMOLED display or an LCD liquid crystal display; in other embodiments, the display unit may also be other electronic components having a display function.
  • the method includes the following steps:
  • the process proceeds to step S101 to preset operation configuration information, where the operation configuration information includes a correspondence relationship between the eyeball activity information and the operation instruction.
  • the operation instruction includes one or more of a text operation instruction, an image operation instruction, a video operation instruction, and an application operation instruction.
  • the text operation instruction includes a selected text instruction, a delete text instruction, a copy text instruction, and the like;
  • the image operation instruction includes a selected image instruction, a copy image instruction, a cut image instruction, an image deletion instruction, a switching image screen, and the like;
  • the video operation The instructions include intercepting, pausing, saving, deleting, fast forwarding, rewinding, zooming, and volume adjustment of the video;
  • the application operation instructions include starting, deleting, selecting, moving, etc.
  • the eyeball activity information may be information of movement in any direction of the eyeball, including linear motion of the user's eyes or monocular eyeball from top to bottom or from bottom to top, linear motion from left to right or from right to left; eyeball clockwise Or a rotation of one or more turns counterclockwise; the combination of linear and linear motion of the above-mentioned eyeball, the combination of circular motion, the combination of linear motion and circular motion, or the eye movement that any other device can discern.
  • the eyeball activity information in the operation configuration information is that the user inputs the stored eyeball activity information in advance, and each eyeball activity information corresponds to an operation instruction, and each operation instruction may correspond to a plurality of eyeball activity information.
  • the operation configuration information may be stored in a storage unit of the device, such as a memory of the mobile phone or a hard disk of the computer, or may be stored in a storage unit of the server.
  • the device When the operation configuration information needs to be acquired, the device only needs to establish a communication connection with the server, and then The operational configuration information stored in advance is obtained from the server, and the communication connection includes a wired communication connection or a wireless communication connection.
  • the eyeball activity information of the user on the eyeball activity recognition area is captured.
  • the coverage of the sensing unit is adapted to the size of the display unit.
  • the shape of the sensing unit is a rectangle, and the size of the rectangle is located at the center of the display unit, so as to ensure that the image of the eyeball is not offset.
  • Optical axis This ensures that as long as the user's eyes are aligned with the display unit, the sensing unit can collect the user's eye movement information in time, no matter how the user's eyeball moves.
  • step S103 determines whether there is eyeball activity information matching the captured eyeball activity information in the operation configuration information. If yes, the process proceeds to step S104 to execute an operation instruction corresponding to the eyeball activity information in the operation configuration information, otherwise the process proceeds to step S105.
  • the comparison of the eyeball activity information can be implemented by an eyeball feature recognition algorithm, and the eyeball feature recognition algorithm can be stored in the storage unit of the device. After the sensor unit acquires the eyeball activity information on the eyeball activity recognition zone, the processor of the device The eyeball feature recognition algorithm in the storage unit is called, and the acquired eyeball activity information is compared with the eyeball activity information in the preset operation configuration information to determine whether the two match.
  • the eyeball feature algorithm includes steps such as preprocessing of eyeball feature information, data feature extraction, feature matching, eyeball trajectory recognition, etc., which can be implemented by various algorithms, which are mature existing technologies and have been applied to each In the field, the details are not repeated here.
  • FIG. 2 is a flowchart of a method for eyeball tracking operation according to another embodiment of the present invention.
  • the step of “presetting the operation configuration information” includes:
  • step S201 the process proceeds to step S201 to receive a user setting command to display an eyeball activity recognition area.
  • the setting instruction can be triggered by the user clicking a button in the setting column on the screen, and after receiving the setting instruction, the device will display the eyeball activity recognition area, so that the user can input the eyeball activity information.
  • displaying the eye movement recognition area may include: increasing the brightness of the eye movement recognition area or displaying a prompt input box on the eye movement recognition area.
  • the method before receiving the user setting instruction, the method further includes receiving user account information, where the account information includes a user ID and a password.
  • the user needs to input the correct user ID and password by means of voice control, eye control, or key password control, and the setting command can be triggered only after the user account is logged in, so that the security of the operation configuration information setting can be improved on the one hand.
  • voice control eye control
  • key password control key password control
  • the setting command can be triggered only after the user account is logged in, so that the security of the operation configuration information setting can be improved on the one hand.
  • the eyeball operation item selected by the user is received, and the eyeball activity information of the user on the eyeball recognition area is captured and saved.
  • the eyeball operation item can be the default eye movement operation item of the system, or can be determined by user customization.
  • the eye movement project can be presented to the user in text, image or video, allowing the user to click on the selection.
  • the eyeball operation item can be drawn into a picture or a video made into a tutorial can be presented to the user so that the user can accurately select the eye movement operation item.
  • the current eyeball activity information of the user can be collected, and the collected eyeball activity information is preset eyeball activity information, which can be stored in the storage unit.
  • the step of “capturing the eyeball activity information of the user on the eyeball recognition area and saving” includes: determining whether the eyeball activity information in the user setting process has been stored in the storage unit, and prompting the user when the determination is yes. The eyeball activity information has been entered; when the determination is no, the eyeball activity information is saved to the storage unit. This can effectively avoid repeated entry of eye movement information.
  • step S203 to display an operation instruction identification list, and receives a selection instruction of the operation instruction identifier by the user, and establishes a correspondence between the operation instruction corresponding to the selected operation instruction identifier and the captured eye movement information of the user and saves.
  • the operation instruction identification list includes an identifier corresponding to one or more operation instructions, and each operation instruction identifier corresponds to an operation instruction, and the operation instruction identifier may be displayed in the form of a text or a picture.
  • the selection instruction can be triggered by the user clicking a check, double clicking, or the like.
  • the device can be combined with a VR/AR helmet to provide a relatively closed dark operating space, giving the user an immersive feeling, since in the dark conditions, the visible light does not enter the person.
  • the sensing unit cannot capture the light signal reflected by the user, thereby capturing the user's eye movement information. Therefore, in the present embodiment, the apparatus further includes an infrared light source for emitting infrared light.
  • the infrared light source is an electronic component that emits infrared light, such as an LED infrared device.
  • the infrared light source can be disposed on the same side of the mobile terminal and the display panel when designing, so as to ensure that the infrared light signal reflected by the user's eyeball can be under the display screen. Captured by the sensing unit.
  • the sensing unit includes a TFT image sensing array film.
  • LCD liquid crystal display
  • OLED organic light emitting diode
  • the main structure for forming the TFT switching function is a metal oxide semiconductor field effect transistor (MOSFET), and the well-known semiconductor layer material mainly includes amorphous silicon, polycrystalline silicon, indium gallium zinc oxide (IGZO), or organic mixed with carbon nano materials. Compounds and so on.
  • MOSFET metal oxide semiconductor field effect transistor
  • IGZO indium gallium zinc oxide
  • the TFT photodetecting diode Since the structure of the light sensing diode can also be prepared by using such a semiconductor material, and the production equipment is also compatible with the production equipment of the TFT array, in recent years, the TFT photodetecting diode has been produced by the TFT array preparation method.
  • the TFT image sensing array film described in this embodiment is the above-mentioned TFT photodetecting diode (such as the photo sensing diode region in FIG. 6 ).
  • the production process of the TFT image sensing array film is different from that of the display panel TFT in that the pixel opening area of the display panel is changed to the light sensing area in the production process.
  • the TFT can be prepared by using a thin glass substrate or a high temperature resistant plastic material as described in US Pat. No. 6,943,070 B2.
  • the sensing unit shown in FIG. 6 is susceptible to reflection or refraction of visible light emitted by ambient light or display pixels, causing optical interference, which seriously affects the signal of the TFT image sensing array film embedded under the display panel.
  • Noise ratio (SNR) in order to improve the signal-to-noise ratio, the sensing unit of the present invention is further improved on the basis of the sensing unit shown in FIG. 6, so that the TFT image sensing array film can detect and recognize the user's eyeball reflection.
  • the sensing unit includes an infrared sensing layer, and the infrared sensing layer is configured to receive an infrared light signal reflected by a user's eye ball to capture user's eyeball activity information.
  • the infrared sensing layer is an array formed by an infrared photosensitive transistor.
  • the TFT image sensing array film of FIG. 6 is improved, including: replacing the original by using a TFT photovoltaic field effect transistor (PVFET)
  • PVFET TFT photovoltaic field effect transistor
  • the infrared sensing layer is also designed as a field effect transistor (Field Effect Transistor) structure, in order to prepare the transistor structure, and the prepared transistor structure is completely compatible with the TFT image sensing array film shown in FIG.
  • the method is as follows: preparing an amorphous silicon layer by chemical vapor deposition, preparing a metal layer and a transparent electrode layer by physical sputtering, and defining a lithographic shape layer required for each layer device by photomask lithography during preparation. .
  • the potential difference between the drain and the source of the prepared infrared photosensitive transistor is an operating parameter for converting infrared light into an electrical signal.
  • the gate of the infrared photosensitive transistor is made of a photovoltaic material sensitive to infrared light.
  • the photovoltaic material can be a quantum dot, carbon nano material, metal modified by Bandgap Engineering.
  • An oxide film material or the like is obtained by mixing these materials into an organic or inorganic compound and stirring them into a gel or a liquid, and then preparing them as a gate by coating, printing, or the like.
  • the electron hole pair (Electron-Hole Pair) excited by the photovoltaic material induces the drain and the source of the infrared photosensitive transistor.
  • the electronic channel is formed, the potential difference between the drain and the source is thus reduced, and the drain output current is increased by the TFT for scanning.
  • the switching characteristic can realize the reading of the infrared light image electrical signal, thereby realizing the user. Capture of eye movement information.
  • the infrared sensing layer is an array formed by infrared photosensitive diodes.
  • the TFT image sensing array film of FIG. 6 is improved, specifically, the TFT image of FIG. 6 is replaced by an infrared photosensitive diode.
  • the photodiode layer of the array film is sensed, and the infrared photodiode comprises a microcrystalline silicon photodiode or an amorphous silicon photodiode.
  • Embodiment 1 The amorphous silicon p-type/i-type/n-type photodiode structure (photodiode layer in FIG. 6) is changed from a microcrystalline silicon p-type/i-type/n-type photodiode structure.
  • the degree of microcrystallization of the photodiode is mainly in a chemical meteorological deposition process, and a gaseous hydrogen silane (SiH4) is mixed with a suitable hydrogen concentration to control the dangling bond of the hydrogen atom-bonded amorphous silicon.
  • SiH4 gaseous hydrogen silane
  • Coating of microcrystalline silicon p-type / i-type / n-type photodiode structure By adjusting the hydrogen concentration of chemical weather deposition, the operating wavelength range of the microcrystalline photodiode can be extended to the wavelength range of 600 nm to 1000 nm.
  • the microcrystalline silicon photodiode may also be formed by stacking a p-type/i-type/n-type structure having a double junction or more.
  • the first junction p-type/i-type/n-type material of the photodiode is still an amorphous structure, and the p-type/i-type/n-type material above the second junction layer may be a microcrystalline structure or a polycrystalline structure.
  • Embodiment 2 Changing the amorphous silicon p-type/i-type/n-type photodiode structure (the photodiode layer in FIG. 6) to the p-type/i-type/n of the amorphous silicon compound doped with the expandable photosensitive wavelength range
  • a preferred photodiode structure, an example of a preferred compound is amorphous silicon germanium.
  • the intrinsic layer (i-type) of the photodiode is mixed with silane (SiH4) by gas decane (GeH4) during chemical vapor deposition to achieve amorphous silicon germanium p-type/i type/
  • the photosensitive range of the n-type photodiode reaches a range of light wavelengths of 600 nm to 1000 nm.
  • an amorphous silicon photodiode in order to increase the quantum efficiency of photoelectric conversion, may also be formed by stacking a p-type/i-type/n-type structure above a double junction.
  • the first junction p-type/i-type/n-type material of the photodiode is still an amorphous silicon structure, and the p-type/i-type/n-type material above the second junction layer may be a microcrystalline structure, a polycrystalline structure or a doped Compound materials that extend the range of photosensitive wavelengths.
  • the infrared sensing layer is an array formed by an infrared photosensitive diode
  • a TFT can be used for scanning driving plus a bias voltage between the p-type/i-type/n-type photodiodes, which will make the infrared
  • the photodiode is in the state of detecting the infrared light signal, and the infrared light signal reflected by the user's eyeball is converted into an infrared light image electrical signal and output, thereby realizing the capture of the user's eyeball activity information.
  • the operation instruction is a screen switching instruction
  • the corresponding operation instruction, otherwise the operation instruction is not executed includes: determining whether the captured eye movement information of the captured user matches the eye movement information corresponding to the screen switching instruction, and if so, switching the screen; otherwise, the screen is not switched. Since the video stream data is composed of one frame of image picture, the method of the embodiment is also applicable to the judgment of the video stream data.
  • the method for eyeball tracking operation is characterized in that the step “determining whether there is eyeball activity information matching the captured eyeball activity information in the operation configuration information” specifically includes: according to the captured user's eyeball The activity information calculates its feature value and compares it with the feature value of the eyeball activity information preset in the configuration information; when the error is less than the preset value, it is determined to be matched, otherwise it is determined as a mismatch.
  • the method further comprises the step of: issuing a prompt message when it is determined that there is no eyeball activity information matching the captured eyeball activity information in the operational configuration information.
  • the prompt information includes one or more of voice prompt information, image prompt information, light prompt information, and video prompt information.
  • the voice prompt information includes voice prompt information prompting the user to operate the eyeball activity item again
  • the image prompt information includes popup prompt information prompting the user to operate the eyeball activity item again
  • the video prompt information includes prompting the user to operate the eyeball activity item again.
  • the animation prompt information, the light prompt information includes changing the brightness of the screen or letting the display screen emit light of different colors.
  • a backlight unit is further disposed under the sensing unit, and the sensing unit is disposed on the backlight unit and the LCD liquid crystal. Between the displays. Since the LCD liquid crystal display is not a self-illuminating element, it is necessary to add a backlight unit below the sensing unit during installation.
  • the backlight unit may be an LCD backlight module or other electronic components having a self-luminous function.
  • the display unit is an AMOLED display screen, since the OLED display screen is a self-luminous element, there is no need to provide a backlight unit.
  • the eyeball activity recognition area includes a plurality of eyeball activity recognition sub-regions, and a sensing unit is disposed corresponding to each of the eyeball activity recognition sub-regions.
  • the device further includes a sensing unit control circuit, the method further comprising: receiving a start command of the user for the eyeball activity recognition sub-region, the sensing unit control circuit turning on the sensing unit below the eyeball activity recognition sub-region, and Receiving a closing command of the user to the eyeball activity recognition sub-region, the sensing unit control circuit turns off the sensing unit below the eyeball activity recognition sub-region.
  • the two eyeball activity recognition sub-regions may be evenly distributed on the screen one above or one left and one right, or may be distributed in the screen in other arrangements.
  • the range of the two eyeball recognition sub-regions covers the entire display screen, so that when both eyeball activity recognition sub-regions are set to the on state, the imaging projection of the user's eyeball is always within the range of the sensing unit, effectively improving Capture the user's eye features to enhance the user experience.
  • the range of the two eyeball recognition sub-regions may also occupy 2/3, 3/4, etc. of the entire display screen area, and only needs to satisfy the optical axis of the eyeball motion recognition sub-region without deviating from the eyeball imaging.
  • the user can also set one eye recognition sub-area to be turned on according to his or her preference, and the other eyeball recognition sub-area is closed. It is also possible to set both identification sub-areas to the off state when no operation is required on the device.
  • the number of eyeball recognition sub-regions may also be other values, which may be set according to actual needs.
  • the sensing unit below each eye activity recognition sub-area is turned on or off, and can be set according to the user's own preferences.
  • FIG. 3 is a schematic diagram of an apparatus for eyeball tracking operation according to an embodiment of the present invention.
  • the device is an electronic device with a touch display screen, such as a smart mobile device such as a mobile phone, a tablet computer, a personal digital assistant, or an electronic device such as a personal computer or a computer for industrial equipment.
  • the device can also be combined with an optical imaging device disposed between the display unit and the user's eyes.
  • the user's eyeball projection is first imaged in the optical imaging device, and the imaged projection is displayed.
  • the area of the eyeball activity recognition area on the unit is captured by the sensing unit under the eye movement recognition, and the effect of the analog VR/AR device can be achieved by the cooperation between the optical imaging device and the display unit.
  • the device includes a display unit 101 and a sensing unit 102.
  • the display unit 101 is provided with an eyeball activity recognition area.
  • the sensing unit 102 is located below the eyeball activity recognition area for sensing light reflected by the user's eyeball. Signaling and capturing user's eyeball activity information; the device further includes an operation information setting unit 104, a determining unit 108, and a processing unit 106.
  • the display unit includes an AMOLED display or an LCD liquid crystal display; in other embodiments, the display unit may also be other electronic components having a display function.
  • the operation information setting unit 104 is configured to preset operation configuration information, where the operation configuration information includes a correspondence relationship between the eyeball activity information and the operation instruction.
  • the operation instruction includes one or more of a text operation instruction, an image operation instruction, a video operation instruction, and an application operation instruction.
  • the text operation instruction includes a selected text instruction, a delete text instruction, a copy text instruction, and the like;
  • the image operation instruction includes a selected image instruction, a copy image instruction, a cut image instruction, an image deletion instruction, a switching image screen, and the like;
  • the instructions include intercepting, pausing, saving, deleting, fast forwarding, rewinding, zooming, and volume adjustment of the video;
  • the application operation instructions include starting, deleting, selecting, moving, etc.
  • the eyeball activity information may be information of movement in any direction of the eyeball, including linear motion of the user's eyes or monocular eyeball from top to bottom or from bottom to top, linear motion from left to right or from right to left; eyeball clockwise Or a rotation of one or more turns counterclockwise; the combination of linear and linear motion of the above-mentioned eyeball, the combination of circular motion, the combination of linear motion and circular motion, or the eye movement that any other device can discern.
  • the eyeball activity information in the operation configuration information is that the user inputs the stored eyeball activity information in advance, and each eyeball activity information corresponds to an operation instruction, and each operation instruction may correspond to a plurality of eyeball activity information.
  • the operation configuration information may be stored in a storage unit of the device, such as a memory of the mobile phone or a hard disk of the computer, or may be stored in a storage unit of the server.
  • the device When the operation configuration information needs to be acquired, the device only needs to establish a communication connection with the server, and then The operational configuration information stored in advance is obtained from the server, and the communication connection includes a wired communication connection or a wireless communication connection.
  • the sensing unit 102 is configured to capture eyeball activity information of the user on the eyeball activity recognition area.
  • the coverage of the sensing unit is adapted to the size of the display unit.
  • the shape of the sensing unit is a rectangle, and the size of the rectangle is located at the center of the display unit, so as to ensure that the image of the eyeball is not offset.
  • Optical axis This ensures that as long as the user's eyes are aligned with the display unit, the sensing unit can collect the user's eye movement information in time, no matter how the user's eyeball moves.
  • the determining unit 108 is configured to determine whether the eyeball activity information matching the captured eyeball activity information exists in the operation configuration information, and if yes, the processing unit 106 is configured to execute an operation instruction corresponding to the eyeball activity information in the operation configuration information, otherwise Processing unit 106 does not execute the operational instructions.
  • the comparison of the eyeball activity information can be implemented by an eyeball feature recognition algorithm, and the eyeball feature recognition algorithm can be stored in the storage unit of the device. After the sensor unit acquires the eyeball activity information on the eyeball activity recognition zone, the processor of the device The eyeball feature recognition algorithm in the storage unit is called, and the acquired eyeball activity information is compared with the eyeball activity information in the preset operation configuration information to determine whether the two match.
  • the eyeball feature algorithm includes steps such as preprocessing of eyeball feature information, data feature extraction, feature matching, eyeball trajectory recognition, etc., which can be implemented by various algorithms, which are mature existing technologies and have been applied to each In the field, the details are not repeated here.
  • the apparatus includes an operation instruction receiving unit 105, and the “operation information setting unit is configured to preset operation configuration information” includes: the operation information setting unit is configured to receive a user setting command.
  • the display unit is configured to display an eyeball activity recognition area.
  • the setting instruction can be triggered by the user clicking a button in the setting column on the screen, and after receiving the setting instruction, the device will display the eyeball activity recognition area, so that the user can input the eyeball activity information.
  • displaying the eye movement recognition area may include: increasing the brightness of the eye movement recognition area or displaying a prompt input box on the eye movement recognition area.
  • the method before receiving the user setting instruction, the method further includes receiving user account information, where the account information includes a user ID and a password.
  • the user needs to input the correct user ID and password, and the setting command can be triggered only after the user account is logged in, so that the security of the operation configuration information setting can be improved on the one hand, and different users can be distinguished on one device on the other hand. Save the effect of different eye movement information and the corresponding operation instructions.
  • the operation instruction receiving unit is configured to receive an eyeball operation item selected by a user, and the sensing unit is configured to capture and save the eyeball activity information of the user on the eyeball recognition area.
  • the eyeball operation item can be the default eye movement operation item of the system, or can be determined by user customization.
  • the eye movement project can be presented to the user in text, image or video, allowing the user to click on the selection.
  • the eyeball operation item can be drawn into a picture or a video made into a tutorial can be presented to the user so that the user can accurately select the eye movement operation item.
  • the current eyeball activity information of the user can be collected, and the collected eyeball activity information is preset eyeball activity information, which can be stored in the storage unit.
  • the step of “capturing the eyeball activity information of the user on the eyeball recognition area and saving” includes: determining whether the eyeball activity information in the user setting process has been stored in the storage unit, and prompting the user when the determination is yes. The eyeball activity information has been entered; when the determination is no, the eyeball activity information is saved to the storage unit. This can effectively avoid repeated entry of eye movement information.
  • the display unit is configured to display an operation instruction identification list, where the operation instruction receiving unit is further configured to receive a selection instruction of the operation instruction identifier by the user, and establish an operation instruction corresponding to the selected operation instruction identifier and the captured user's eyeball Correspondence of activity information is saved.
  • the operation instruction identifier list includes identifiers corresponding to one or more operation instructions, and each operation instruction identifier corresponds to an operation instruction.
  • the operation instruction identifier can be displayed in the form of text or a picture.
  • the selection instruction can be triggered by the user clicking a check, double clicking, or the like.
  • the device can be combined with a VR/AR helmet to provide a relatively closed dark operating space, giving the user an immersive feeling, since in the dark conditions, the visible light does not enter the person.
  • the sensing unit cannot capture the light signal reflected by the user, thereby capturing the user's eye movement information.
  • the apparatus further includes an infrared source for emitting infrared light.
  • the infrared light source is an electronic component that emits infrared light, such as an LED infrared device.
  • the infrared light source can be disposed on the same side of the mobile terminal and the display panel when designing, so as to ensure that the infrared light signal reflected by the user's eyeball can be under the display screen. Captured by the sensing unit.
  • the sensing unit includes a TFT image sensing array film.
  • LCD liquid crystal display
  • OLED organic light emitting diode
  • the main structure for forming the TFT switching function is a metal oxide semiconductor field effect transistor (MOSFET), and the well-known semiconductor layer material mainly includes amorphous silicon, polycrystalline silicon, indium gallium zinc oxide (IGZO), or organic mixed with carbon nano materials. Compounds and so on.
  • MOSFET metal oxide semiconductor field effect transistor
  • IGZO indium gallium zinc oxide
  • the TFT photodetecting diode Since the structure of the light sensing diode can also be prepared by using such a semiconductor material, and the production equipment is also compatible with the production equipment of the TFT array, in recent years, the TFT photodetecting diode has been produced by the TFT array preparation method.
  • the TFT image sensing array film described in this embodiment is the above-mentioned TFT photodetecting diode (such as the photo sensing diode region in FIG. 6 ).
  • the production process of the TFT image sensing array film is different from that of the display panel TFT in that the pixel opening area of the display panel is changed to the light sensing area in the production process.
  • the TFT can be prepared by using a thin glass substrate or a high temperature resistant plastic material as described in US Pat. No. 6,943,070 B2.
  • the sensing unit shown in FIG. 6 is susceptible to reflection or refraction of visible light emitted by ambient light or display pixels, causing optical interference, which seriously affects the signal of the TFT image sensing array film embedded under the display panel.
  • Noise ratio (SNR) in order to improve the signal-to-noise ratio, the sensing unit of the present invention is further improved on the basis of the sensing unit shown in FIG. 6, so that the TFT image sensing array film can detect and recognize the user's eyeball reflection.
  • the sensing unit includes an infrared sensing layer, and the infrared sensing layer is configured to receive an infrared light signal reflected by a user's eye ball to capture user's eyeball activity information.
  • the infrared sensing layer is an array formed by an infrared photosensitive transistor.
  • the TFT image sensing array film of FIG. 6 is improved, including: replacing the original by using a TFT photovoltaic field effect transistor (PVFET)
  • PVFET TFT photovoltaic field effect transistor
  • the infrared sensing layer is also designed as a field effect transistor (Field Effect Transistor) structure, in order to prepare the transistor structure, and the prepared transistor structure is completely compatible with the TFT image sensing array film shown in FIG.
  • the method is as follows: preparing an amorphous silicon layer by chemical vapor deposition, preparing a metal layer and a transparent electrode layer by physical sputtering, and defining a lithographic shape layer required for each layer device by photomask lithography during preparation. .
  • the potential difference between the drain and the source of the prepared infrared photosensitive transistor is an operating parameter for converting infrared light into an electrical signal.
  • the gate of the infrared photosensitive transistor is made of a photovoltaic material sensitive to infrared light.
  • the photovoltaic material can be a quantum dot, carbon nano material, metal modified by Bandgap Engineering.
  • An oxide film material or the like is obtained by mixing these materials into an organic or inorganic compound and stirring them into a gel or a liquid, and then preparing them as a gate by coating, printing, or the like.
  • the electron hole pair (Electron-Hole Pair) excited by the photovoltaic material induces the drain and the source of the infrared photosensitive transistor.
  • the electronic channel is formed, the potential difference between the drain and the source is thus reduced, and the drain output current is increased by the TFT for scanning.
  • the switching characteristic can realize the reading of the infrared light image electrical signal, thereby realizing the user. Capture of eye movement information.
  • the infrared sensing layer is an array formed by infrared photosensitive diodes.
  • the TFT image sensing array film of FIG. 6 is improved, specifically, the TFT image of FIG. 6 is replaced by an infrared photosensitive diode.
  • the photodiode layer of the array film is sensed, and the infrared photodiode comprises a microcrystalline silicon photodiode or an amorphous silicon photodiode.
  • Embodiment 1 The amorphous silicon p-type/i-type/n-type photodiode structure (photodiode layer in FIG. 6) is changed from a microcrystalline silicon p-type/i-type/n-type photodiode structure.
  • the degree of microcrystallization of the photodiode is mainly in a chemical meteorological deposition process, and a gaseous hydrogen silane (SiH4) is mixed with a suitable hydrogen concentration to control the dangling bond of the hydrogen atom-bonded amorphous silicon.
  • SiH4 gaseous hydrogen silane
  • Coating of microcrystalline silicon p-type / i-type / n-type photodiode structure By adjusting the hydrogen concentration of chemical weather deposition, the operating wavelength range of the microcrystalline photodiode can be extended to the wavelength range of 600 nm to 1000 nm.
  • the microcrystalline silicon photodiode may also be formed by stacking a p-type/i-type/n-type structure having a double junction or more.
  • the first junction p-type/i-type/n-type material of the photodiode is still an amorphous structure, and the p-type/i-type/n-type material above the second junction layer may be a microcrystalline structure or a polycrystalline structure.
  • Embodiment 2 Changing the amorphous silicon p-type/i-type/n-type photodiode structure (the photodiode layer in FIG. 6) to the p-type/i-type/n of the amorphous silicon compound doped with the expandable photosensitive wavelength range
  • a preferred photodiode structure, an example of a preferred compound is amorphous silicon germanium.
  • the intrinsic layer (i-type) of the photodiode is mixed with silane (SiH4) by gas decane (GeH4) during chemical vapor deposition to achieve amorphous silicon germanium p-type/i type/
  • the photosensitive range of the n-type photodiode reaches a range of light wavelengths of 600 nm to 1000 nm.
  • an amorphous silicon photodiode in order to increase the quantum efficiency of photoelectric conversion, may also be formed by stacking a p-type/i-type/n-type structure above a double junction.
  • the first junction p-type/i-type/n-type material of the photodiode is still an amorphous silicon structure, and the p-type/i-type/n-type material above the second junction layer may be a microcrystalline structure, a polycrystalline structure or a doped Compound materials that extend the range of photosensitive wavelengths.
  • the infrared sensing layer is an array formed by an infrared photosensitive diode
  • a TFT can be used for scanning driving plus a bias voltage between the p-type/i-type/n-type photodiodes, which will make the infrared
  • the photodiode is in the state of detecting the infrared light signal, and the infrared light signal reflected by the user's eyeball is converted into an infrared light image electrical signal and output, thereby realizing the capture of the user's eyeball activity information.
  • the operation instruction is a screen switching instruction
  • the “determining unit is configured to determine whether there is eye movement information in the operation configuration information that matches the captured eye movement, and if yes, the processing unit is configured to execute The operating instruction corresponding to the eyeball activity information in the operation configuration information, otherwise the processing unit does not execute the operation instruction
  • the method includes: the determining unit is configured to determine whether the captured eyeball activity information of the captured user matches the eyeball activity information corresponding to the screen switching instruction, If yes, the processing unit is used to switch the screen, otherwise the processing unit does not switch the screen.
  • the step “the determining unit is configured to determine whether the eyeball activity information matching the captured eyeball activity information exists in the operation configuration information” specifically includes: determining, by the determining unit, the eyeball activity information according to the captured user The feature value is calculated and compared with the feature value of the eyeball activity information preset in the configuration information; when the error is less than the preset value, it is determined to be matched, otherwise it is determined as a mismatch.
  • the processing unit is further configured to issue the prompt information when the determining unit determines that the eyeball activity information matching the captured eyeball activity information does not exist in the operation configuration information.
  • the prompt information includes one or more of voice prompt information, image prompt information, light prompt information, and video prompt information.
  • the voice prompt information includes voice prompt information prompting the user to operate the eyeball activity item again
  • the image prompt information includes popup prompt information prompting the user to operate the eyeball activity item again
  • the video prompt information includes prompting the user to operate the eyeball activity item again.
  • the animation prompt information, the light prompt information includes changing the brightness of the screen or letting the display screen emit light of different colors.
  • a backlight unit is further disposed under the sensing unit, and the sensing unit is disposed on the backlight unit and the LCD liquid crystal. Between the displays. Since the LCD liquid crystal display is not a self-illuminating element, it is necessary to add the backlight unit 103 below the sensing unit during installation.
  • the backlight unit may be an LCD backlight module or other electronic components having a self-luminous function.
  • the display unit is an AMOLED display screen, since the OLED display screen is a self-luminous element, there is no need to provide a backlight unit.
  • the eyeball activity recognition area includes a plurality of eyeball activity recognition sub-regions, and a sensing unit is disposed corresponding to each of the eyeball activity recognition sub-regions.
  • the device further includes a sensing unit control circuit, the method further comprising: receiving a start command of the user for the eyeball activity recognition sub-region, the sensing unit control circuit turning on the sensing unit below the eyeball activity recognition sub-region, and Receiving a closing command of the user to the eyeball activity recognition sub-region, the sensing unit control circuit turns off the sensing unit below the eyeball activity recognition sub-region.
  • the two eyeball activity recognition sub-regions may be evenly distributed on the screen one above or one left and one right, or may be distributed in the screen in other arrangements.
  • the range of the two eyeball recognition sub-regions covers the entire display screen, so that when both eyeball activity recognition sub-regions are set to the on state, the imaging projection of the user's eyeball is always within the range of the sensing unit, effectively improving Capture the user's eye features to enhance the user experience.
  • the range of the two eyeball recognition sub-regions may also occupy 2/3, 3/4, etc. of the entire display screen area, and only needs to satisfy the optical axis of the eyeball motion recognition sub-region without deviating from the eyeball imaging.
  • the user can also set one eye recognition sub-area to be turned on according to his or her preference, and the other eyeball recognition sub-area is closed. It is also possible to set both identification sub-areas to the off state when no operation is required on the device.
  • the number of eyeball recognition sub-regions may also be other values, which may be set according to actual needs.
  • the sensing unit below each eye activity recognition sub-area is turned on or off, and can be set according to the user's own preferences.
  • the invention has the following advantages: by providing a sensing unit under the eyeball recognition area of the display unit, a projection of the user's eyeball imaging through the optical device is located on the eyeball recognition area, and the sensing unit is disposed at or near the eyeball imaging optical axis position.
  • the present invention can capture the user's eyeball activity information more accurately and in time, and then compare with the preset eyeball activity information, and execute the eyeball activity information correspondingly.
  • the operation instructions enable the user to quickly control the device through the eyeball activity, thereby effectively improving the user experience.
  • the sensing unit is disposed under the display unit, and the overall thickness of the moving wearable device or the moving device can be effectively reduced compared to the structure in which the camera is independently protruded from the display screen, so that the wearable device or the mobile device More lightweight and meet the needs of the market.
  • the computer device includes but is not limited to: a personal computer, a server, a general purpose computer, a special purpose computer, a network device, an embedded device, a programmable device, a smart mobile terminal, a smart home device, a wearable smart device, a vehicle smart device, and the like;
  • the storage medium includes, but is not limited to, a RAM, a ROM, a magnetic disk, a magnetic tape, an optical disk, a flash memory, a USB flash drive, a mobile hard disk, a memory card, a memory stick, a network server storage, a network cloud storage, and the like.
  • the computer program instructions can also be stored in a computer device readable memory that can direct the computer device to operate in a particular manner, such that instructions stored in the computer device readable memory produce an article of manufacture comprising the instruction device, the instruction device being implemented in the process Figure One or more processes and/or block diagrams of the functions specified in a block or blocks.
  • These computer program instructions can also be loaded onto a computer device such that a series of operational steps are performed on the computer device to produce computer-implemented processing, such that instructions executed on the computer device are provided for implementing one or more processes in the flowchart And/or block diagram of the steps of a function specified in a box or blocks.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)

Abstract

一种眼球追踪操作的方法和装置，通过在显示单元的眼球识别区下方设置传感单元，用户眼球活动的投影位于所述眼球识别区上，相较于摄像头独立于显示屏设置在边缘位置的结构，可以及时捕捉到用户眼球活动信息，进而与预设的眼球活动信息进行比对，执行该眼球活动信息对应的操作指令，使得用户通过眼球活动就可快速对设备进行操控，有效提升用户体验。此外，将传感单元设置于显示单元的下方，相较于摄像头独立突出设置于显示屏外的结构，可以有效缩小移动设备的整体厚度，使得移动设备更加轻薄、满足市场的需求。

Description

一种眼球追踪操作的方法和装置 技术领域
本发明涉及电子设备应用领域，特别涉及一种眼球追踪操作的方法和装置。
背景技术
随着科技的发展和技术的进步，触控显示面板已经广泛应用在需要进行人机交互界面的装置中，如工业计算机的操作屏幕、平板计算机、智能手机的触控屏幕等等。然而以穿戴式电子装置而言，人机交互界面技术仍有众多进步空间。以虚拟现实或增强现实(VR/AR)装置为例，为了提高用户使用这些装置时的感官体验，通常会将用户的眼球活动与操作指令相结合，因此眼球追踪就是其中一项重要的人机交互界面技术。
眼球追踪是指对用户眼球和眼球周边的特征变化进行追踪，以预测用户的状态和需求，并进行响应，达到用户以眼睛主动或被动控制设备的目的。目前，一般采用VR/AR装置或是移动设备上显示屏外的摄像头来捕捉眼球特征变化信息，由于此类装置设备的显示屏外摄像头往往独立设置于设备的边缘位置(如设置于手机的顶部)，偏离了眼球成像的光轴，而用户眼球特征变化又十分快速，采用现有的屏外摄像头无法精确且快速捕捉到用户的眼球特征变化。
以画面切换为例，目前乃是采用现有的屏外摄像头结构来进行眼球追踪，然而用户眼睛在使用装备的过程当中是聚焦于屏幕画面，若用户眼球察觉画面不同位置的细节而开始进行了活动，而由于偏离光轴的屏外摄像头采集的眼球特征信息精确度不够高、或是响应时间不够快，这段期间内画面无法随着用户眼球聚焦之视角进行精准且实时地切换，巨大的画面延迟给用户带来了不良的感官体验，甚至引发头晕、呕吐等症状。
发明内容
本发明所要解决的技术问题是提供一种眼球追踪操作的技术方案，用以解决由于显示屏外摄像头设置位置偏离光轴、无法精准且及时捕捉到用户眼球的特征信息，导致眼球追踪视角不够精准、延迟时间久、操作指令无法快速得到响应、用户感官体验差等人因工程(Human Factor Engineering)的问题。
为解决上述技术问题，本发明采取的技术方案为：一种眼球追踪操作的方法，所述方法应用于眼球追踪操作的装置，所述装置包括显示单元和传感单元；所述显示单元上设置有眼球活动识别区，所述传感单元位于所述眼球活动识别区的下方，用于感知用户眼球反射的光信号，并捕捉用户的眼球活动信息，所述方法包括以下步骤：
预先设置操作配置信息，所述操作配置信息包括眼球活动信息与操作指令的对应关系；
捕捉用户在眼球活动识别区上的眼球活动信息，判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息，若是则执行操作配置信息中该眼球活动信息对应的操作指令，否则不执行所述操作指令。
进一步地，所述步骤“预先设置操作配置信息”包括：
接收用户设置命令，显示眼球活动识别区；
接收用户选择的眼球操作项目，以及捕捉用户在所述眼球识别区上的眼球活动信息并保存；
显示一操作指令标识列表，所述操作指令标识列表中包含着一个或多个操作指令对应的标识，每一操作指令标识对应一操作指令；
接收用户对操作指令标识的选择指令，建立所选中的操作指令标识对应的操作指令与所捕捉的用户的眼球活动信息的对应关系并保存。
进一步地，所述操作指令包括文字操作指令、图像操作指令、视频操作指令、应用操作指令中的一种或多种。
进一步地，所述装置还包括红外光源，所述红外光源用于发出红外光，所述传感单元包括红外感应层，所述红外感应层用于接收用户眼球反射的红外光 信号以捕捉用户的眼球活动信息。
进一步地，所述操作指令为画面切换指令，则步骤“判断操作配置信息中是否存在与所捕捉的眼球活动相匹配的眼球活动信息，若是则执行操作配置信息中该眼球活动信息对应的操作指令，否则不执行所述操作指令”包括：
判断捕捉到的用户的眼球活动信息与画面切换指令对应的眼球活动信息是否匹配，若是则对画面进行切换，否则不对画面进行切换。
进一步地，所述传感单元包括TFT影像感测阵列薄膜，所述红外感应层为红外光敏二极管或红外光敏电晶管所形成的阵列。
进一步地，步骤“判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息”具体包括：
根据捕捉到的用户的眼球活动信息计算其特征值，并与配置信息中预设的眼球活动信息的特征值进行对比；当误差小于预设值时，判定为相匹配，否则判定为不匹配。
进一步地，所述方法还包括步骤：
当判定操作配置信息中不存在与所捕捉的眼球活动信息相匹配的眼球活动信息时，发出提示信息。
进一步地，所述提示信息包括声音提示信息、图像提示信息、光线提示信息、视频提示信息中的一种或多种。
进一步地，所述显示单元包括AMOLED显示屏或LCD液晶显示屏。
进一步地，当所述显示单元为LCD液晶显示屏时，所述传感单元的下方还设置有背光单元，所述传感单元设置于背光单元和LCD液晶显示屏之间。
进一步地，所述眼球活动识别区包括多个眼球活动识别子区域，每一眼球活动识别子区域的下方对应设置一传感单元。
进一步地，所述装置还包括传感单元控制电路，所述方法还包括：
接收用户对眼球活动识别子区域的启动指令，传感单元控制电路开启所述眼球活动识别子区域的下方的传感单元，以及接收用户对眼球活动识别子区域 的关闭指令，传感单元控制电路关闭所述眼球活动识别子区域的下方的传感单元。
发明人还提供了一种眼球追踪操作的装置，所述装置包括显示单元和传感单元；所述显示单元上设置有眼球活动识别区，所述传感单元位于所述眼球活动识别区的下方，用于感知用户眼球反射的光信号，并捕捉用户的眼球活动信息；所述装置还包括操作信息设置单元、判断单元和处理单元；
所述操作信息设置单元用于预先设置操作配置信息，所述操作配置信息包括眼球活动信息与操作指令的对应关系；
所述传感单元用于捕捉用户在眼球活动识别区上的眼球活动信息，所述判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息，若是则处理单元用于执行操作配置信息中该眼球活动信息对应的操作指令，否则处理单元不执行所述操作指令。
进一步地，所述装置包括操作指令接收单元，所述“操作信息设置单元用于预先设置操作配置信息”包括：
所述操作信息设置单元用于接收用户设置命令，所述显示单元用于显示眼球活动识别区；
所述操作指令接收单元用于接收用户选择的眼球操作项目，所述传感单元用于捕捉用户在所述眼球识别区上的眼球活动信息并保存；
所述显示单元用于显示一操作指令标识列表，所述操作指令标识列表中包含着一个或多个操作指令对应的标识，每一操作指令标识对应一操作指令；
所述操作指令接收单元还用于接收用户对操作指令标识的选择指令，建立所选中的操作指令标识对应的操作指令与所捕捉的用户的眼球活动信息的对应关系并保存。
进一步地，所述操作指令包括文字操作指令、图像操作指令、视频操作指令、应用操作指令中的一种或多种。
进一步地，所述装置还包括红外光源，所述红外光源用于发出红外光，所 述传感单元包括红外感应层，所述红外感应层用于接收用户眼球反射的红外光信号以捕捉用户的眼球活动信息。
进一步地，所述操作指令为画面切换指令，所述“判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动相匹配的眼球活动信息，若是则处理单元用于执行操作配置信息中该眼球活动信息对应的操作指令，否则处理单元不执行所述操作指令”包括：
判断单元用于判断捕捉到的用户的眼球活动信息与画面切换指令对应的眼球活动信息是否匹配，若是则处理单元用于对画面进行切换，否则处理单元不对画面进行切换。
进一步地，所述传感单元包括TFT影像感测阵列薄膜，所述红外感应层为红外光敏二极管或红外光敏电晶管所形成的阵列。
进一步地，“判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息”具体包括：
判断单元用于根据捕捉到的用户的眼球活动信息计算其特征值，并与配置信息中预设的眼球活动信息的特征值进行对比；当误差小于预设值时，判定为相匹配，否则判定为不匹配。
进一步地，所述处理单元还用于在判断单元判定操作配置信息中不存在与所捕捉的眼球活动信息相匹配的眼球活动信息时，发出提示信息。
进一步地，所述提示信息包括声音提示信息、图像提示信息、光线提示信息、视频提示信息中的一种或多种。
进一步地，所述显示单元包括AMOLED显示屏或LCD液晶显示屏。
进一步地，当所述显示单元为LCD液晶显示屏时，所述传感单元的下方还设置有背光单元，所述传感单元设置于背光单元和LCD液晶显示屏之间。
进一步地，所述眼球活动识别区包括多个眼球活动识别子区域，每一眼球活动识别子区域的下方对应设置一传感单元。
进一步地，所述装置还包括传感单元控制电路和操作指令接收单元，所述 操作指令接收单元用于接收用户对眼球活动识别子区域的启动指令，所述传感单元控制电路用于开启所述眼球活动识别子区域的下方的传感单元，以及所述操作指令接收单元用于接收用户对眼球活动识别子区域的关闭指令，所述传感单元控制电路用于关闭所述眼球活动识别子区域的下方的传感单元。
采用以上技术方案后的有益效果为：通过在显示单元的眼球识别区下方设置传感单元，用户眼球活动通过光学器件成像的投影位于所述眼球识别区上，传感单元设置于眼球成像光轴位置或是近轴位置，相较于摄像头独立于显示屏设置在边缘位置的结构，本发明可以及时捕捉到用户眼球活动信息，进而与预设的眼球活动信息进行比对，执行该眼球活动信息对应的操作指令，使得用户通过眼球活动就可快速对设备进行操控，有效提升用户体验。此外，将传感单元设置于显示单元的下方，相较于摄像头独立突出设置于显示屏区域外的结构，可以有效缩小移动设备的整体厚度，使得穿戴式设备或是移动设备更加轻薄、更适用于柔性穿戴式设备或是移动设备、满足市场的需求。
附图说明
图1为本发明一实施方式涉及的眼球追踪操作的方法的流程图；
图2为本发明另一实施方式涉及的眼球追踪操作的方法的流程图；
图3为本发明一实施方式涉及的眼球追踪操作的装置的示意图；
图4为本发明另一实施方式涉及的眼球追踪操作的装置的示意图；
图5为本发明一实施方式涉及的眼球追踪操作的装置应用场景的示意图；
图6为现有的传感单元的示意图；
图7为本发明一实施方式涉及的传感单元的示意图；
图8为本发明另一实施方式涉及的传感单元的示意图。
标号说明：
101、显示单元；
102、传感单元；
103、背光单元；
104、操作信息设置单元；
105、操作指令接收单元；
106、处理单元；
107、存储单元；
108、判断单元。
具体实施方式
为详细说明本发明的技术内容、构造特征、所实现目的及效果，以下结合实施方式并配合附图详予说明。
请参阅图1，为本发明一实施方式涉及的眼球追踪操作的方法的流程图。所述方法应用于眼球追踪操作的装置，所述装置为具有显示屏或触摸显示屏的电子设备，如是手机、平板电脑、个人数字助理等智能移动设备，还可以是个人计算机、工业装备用计算机等电子设备。当然所述装置还可以与光学成像器件相结合，光学成像器件设置于所述显示单元与用户眼睛之间，如图5所示，用户眼球投影先在光学成像器件中成像，成像的投影位于显示单元上眼球活动识别区范围内，进而被眼球活动识别下方的传感单元捕捉，通过光学成像器件与显示单元之间的配合，可以达到模拟VR/AR设备的效果。
所述装置包括显示单元和传感单元。所述显示单元上设置有眼球活动识别区，所述传感单元位于所述眼球活动识别区的下方，用于感知用户眼球反射的光信号，并捕捉用户的眼球活动信息。在本实施方式中，所述显示单元包括AMOLED显示屏或LCD液晶显示屏；在其他实施方式中，显示单元也可以为其他具有显示功能的电子元件。所述方法包括以下步骤：
首先进入步骤S101预先设置操作配置信息，所述操作配置信息包括眼球活动信息与操作指令的对应关系。在本实施方式中，所述操作指令包括文字操作 指令、图像操作指令、视频操作指令、应用操作指令中的一种或多种。所述文字操作指令包括选中文字指令、删除文字指令、复制文字指令等；所述图像操作指令包括选中图像指令、复制图像指令、截取图像指令、删除图像指令、切换图像画面等；所述视频操作指令包括对视频进行截取、暂停、保存、删除、快进、快退、缩放画面、音量调整等；所述应用操作指令包括对软件应用程序(如手机APP)进行启动、删除、选中、移动等。所述眼球活动信息可以是眼球任意方向上进行运动的信息，包括用户双眼或单眼眼球从上向下或从下向上的直线运动、从左向右或从右向左的直线运动；眼球顺时针或逆时针转一圈或数圈的运动；上述眼球直线运动与直线运动的组合、转圈运动的组合、直线运动与转圈运动的组合，或者任何其他设备可以辨别的眼球运动。
操作配置信息中的眼球活动信息即为用户事先录入存储的眼球活动信息，每一眼球活动信息对应一操作指令，每一个操作指令可以与多个眼球活动信息相对应。操作配置信息可以存储于装置的存储单元，如手机的内存、计算机的硬盘中，也可以存储于服务器的存储单元中，当需要获取操作配置信息时，只需让装置与服务器建立通讯连接，而后再从服务器获取到事先存储的操作配置信息，所述通讯连接包括有线通讯连接或无线通信连接。
而后进入步骤S102捕捉用户在眼球活动识别区上的眼球活动信息。在本实施方式中，传感单元的覆盖的范围与显示单元的大小相适配，优选的，传感单元的形状为矩形，矩形的大小位于显示单元的中心，保证不偏移眼球活动成像的光轴。这样可以保证只要用户眼睛对准显示单元，无论用户眼球如何活动，传感单元都能及时采集到用户的眼球活动信息。
而后进入步骤S103判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息，若是则进入步骤S104执行操作配置信息中该眼球活动信息对应的操作指令，否则进入步骤S105不执行所述操作指令。眼球活动信息的比对可以通过眼球特征识别算法来实现，眼球特征识别算法可以实现存储于装置的存储单元中，当传感单元获取到眼球活动识别区上的眼球活动信息后， 装置的处理器将调用存储单元中的眼球特征识别算法，将所获取的眼球活动信息与预设的操作配置信息中的眼球活动信息进行比对，判断两者是否匹配。眼球特征算法包括对眼球特征信息进行预处理、数据特征提取、特征匹配、眼球运动轨迹识别等步骤，可以用多种算法来实现，这些算法都是成熟的现有技术，现已被应用于各个领域中，此处不再展开赘述。
请参阅图2，为本发明另一实施方式涉及的眼球追踪操作的方法的流程图。所述步骤“预先设置操作配置信息”包括：
首先进入步骤S201接收用户设置命令，显示眼球活动识别区。设置指令可以通过用户点击屏幕上设置栏中的某一按钮触发，装置接收到设置指令后，将对眼球活动识别区进行显示，便于用户输入眼球活动信息。在本实施方式中，显示眼球活动识别区可以包括：提高眼球活动识别区的亮度或在眼球活动识别区上显示一提示输入框。在某些实施例中，在接收用户设置指令之前，还包括接收用户的账号信息，所述账号信息包括用户ID及密码。用户需要以语音控制、眼球控制、或是按键密码控制等方式输入正确的用户ID及密码，登录用户账号后，才可触发所述设置指令，这样一方面可以提高操作配置信息设置的安全性，另一方面也可以达到在一个装置上区分不同用户、保存不同的眼球活动信息以及与之相对应的操作指令的效果。
而后进入步骤S202接收用户选择的眼球操作项目，以及捕捉用户在所述眼球识别区上的眼球活动信息并保存。眼球操作项目可以是系统默认的眼球操作项目，也可以通过用户自定义决定。眼球操作项目可以以文字、图片或视频的方式展现给用户，让用户点击选择。例如可以显示“两个眼球同时向左运动”、“两个眼球同时向右运动”、“两个眼球同时向上运动”、“两个眼球顺时针转动一圈”等等文字供用户选择，也可以将眼球操作项目绘制成图片或制作成教程的视频展现给用户，以便用户可以准确地选择眼球操作项目。当眼球操作项目确认后，就可以对用户当前的眼球活动信息进行采集，所采集到的眼球活动信息即为预设眼球活动信息，可以将其存储于存储单元中。本实施方式中，所述 步骤“捕捉用户在所述眼球识别区上的眼球活动信息并保存”包括：判断用户设置过程中的眼球活动信息是否已存储于存储单元，当判定为是时提示用户该眼球活动信息已录入；当判定为否时将该眼球活动信息保存至存储单元。这样可以有效避免眼球活动信息的重复录入。
而后进入步骤S203显示一操作指令标识列表，接收用户对操作指令标识的选择指令，建立所选中的操作指令标识对应的操作指令与所捕捉的用户的眼球活动信息的对应关系并保存。所述操作指令标识列表中包含着一个或多个操作指令对应的标识，每一操作指令标识对应一操作指令，操作指令标识可以以文字或图片的形式予以显示。选择指令可以通过用户点击勾选、双击等方式触发。
为了给用户呈现更好的视频画面场景，所述装置可以与VR/AR头盔相结合，提供相对封闭黑暗的操作空间，给用户身临其境之感，由于在黑暗条件下，可见光没有进入人眼，传感单元也就无法捕捉到用户反射回来的光信号，进而捕捉用户的眼球活动信息。因此，在本实施方式中，所述装置还包括红外光源，所述红外光源用于发出红外光。红外光源为具有发出红外光的电子元件，如LED红外器件，在设计时可以将红外光源设置在移动终端与显示面板相同一侧，以保证用户眼球反射回的红外光信号可以被显示屏下方的传感单元所捕捉。
如图6所示，在本实施方式中，所述传感单元包括TFT影像感测阵列薄膜。现有的液晶显示(LCD)面板或有机发光二极管(OLED)显示面板，皆是以TFT结构驱动扫描单一像素，以实现面板上像素阵列的显示功能。形成TFT开关功能的主要结构为金属氧化物半导体场效晶体管(MOSFET)，其中熟知的半导体层材料主要有非晶硅、多晶硅、氧化铟镓锌(IGZO)、或是混有碳纳米材料之有机化合物等等。由于光感测二极管的结构亦可采用此类半导体材料制备，且生产设备也兼容于TFT阵列的生产设备，因此近年来TFT光侦测二极管开始以TFT阵列制备方式进行生产。本实施方式所述的TFT影像感测阵列薄膜即为上述提到的TFT光侦测二极管(如图6中的光感测二极管区域部分)，具体结构可以参考美国专利US6943070B2、中华人民共和国专利CN204808361U中对传感单元 结构的描述。TFT影像感测阵列薄膜的生产工艺与显示面板TFT结构不同的是：原本在显示面板的像素开口区域，在生产工艺上改为光感测区域。其TFT制备方式可以采用薄型玻璃为基材，亦可采用耐高温塑性材料为基材，如美国专利US6943070B2所述。
图6所示的传感单元易受周围环境光或者显示屏像素所发出的可见光的反射、折射等因素影响，造成光学干扰，严重影响内嵌于显示面板下方的TFT影像感测阵列薄膜的信号噪声比(SNR)，为了提高信号噪声比，本发明的传感单元在图6所示的传感单元的基础上做了进一步改进，使得TFT影像感测阵列薄膜可以侦测识别用户眼球反射回的红外信号，所述传感单元包括红外感应层，所述红外感应层用于接收用户眼球反射的红外光信号以捕捉用户的眼球活动信息。
如图7所示，在某些实施例中，所述红外感应层为红外光敏电晶管所形成的阵列。为了将TFT影像感测阵列薄膜能够识别的光信号波长从可见光范围扩展至红外光范围，对图6的TFT影像感测阵列薄膜进行改进，具体包括：采取TFT光伏场效晶体管(PVFET)取代原有的阵列薄膜中的光侦测二极管，具体可以参考文献“Photovoltage field-effect transistors，Nature 542,324–327(16February2017)”。在本实施例中，红外感应层亦设计为场效晶体管(Field Effect Transistor)结构，为了制备此晶体管结构，且满足制备的晶体管结构完全兼容于图6所示的TFT影像感测阵列薄膜，可以通过以下方式来实现：以化学汽像沉积制备非晶硅层、物理溅镀制备金属层与透明电极层，在制备过程中以光罩微影蚀刻定义出各层器件需要之微影形状图层。
制备得出的红外光敏电晶管的漏极与源极之间的电位差即为将红外光转为电信号之操作参数。为达到此目的，红外光敏电晶管的栅极采用对红外光较为敏感的光伏材料制成，例如光伏材料可以采用以能隙工程(Bandgap Engineering)改质过之量子点、碳纳米材料、金属氧化物薄膜材料等，将这些材料混入有机或无机化合物搅拌成胶质或液态，再采用涂布、印刷等方式制备得到作为栅极 的材料。
当用户眼球反射回的红外光射入上述红外光敏电晶管所形成的阵列时，光伏材料所激发的电子电洞对(Electron-Hole Pair)诱发红外光敏电晶管的漏极与源极之间的电子通道形成，漏极与源极之间的电位差会因此下降，以TFT作扫描驱动漏极输出电流升高，此开关特性可实现红外光影像电信号之读出，从而实现对用户眼球活动信息的捕捉。
如图8所示，在某些实施例中，所述红外感应层为红外光敏二级管所形成的阵列。为了将TFT影像感测阵列薄膜能够识别的光信号波长从可见光范围扩展至红外光范围，对图6的TFT影像感测阵列薄膜进行改进，具体是采用红外光敏二级管替换图6中TFT影像感测阵列薄膜的光二极管层，红外光敏二极管包括微晶硅光电二极管或非晶硅光电二极管。
实施例一：将非结晶硅p型/i型/n型光电二极管结构(图6中的光二极管层)改由微晶硅p型/i型/n型光电二极管结构。在此实施例中，光电二极管的微结晶程度主要是在化学气象沉积过程中，以适当氢气浓度混入气体硅烷(SiH4)去控制氢原子键结非晶硅之悬空键(dangling bond)，以实现微晶硅p型/i型/n型光电二极管结构之镀膜。藉由调整化学气象沉积的氢气浓度，微晶光电二极管的操作波长范围可以扩展到光波长600nm到1000nm的范围。
在采用微晶光电二极管之实施例中，为了进一步地提高光电转换之量子效率，微晶硅光电二极管也可采用双结以上p型/i型/n型结构堆叠形成。该光电二极管第一结层p型/i型/n型材料仍然为非结晶结构，第二结层以上p型/i型/n型材料可以为微晶结构、多晶结构。
实施例二：将非结晶硅p型/i型/n型光电二极管结构(图6中的光二极管层)改为掺有可扩展光敏波长范围之非结晶硅化合物之p型/i型/n型光电二极管结构，优选之化合物实施例为非晶硅化锗。在此实施例中，光电二极管的本质层(i型)在以化学气象沉积镀膜过程中，通以气体锗烷(GeH4)混入硅烷(SiH4)，以实现非结晶硅化锗p型/i型/n型光电二极管之光敏范围达到光波长600nm到 1000nm的范围。
在采用非结晶硅化合物光电二极管之实施例中，为了提高光电转换之量子效率，非晶硅光电二极管也可采用双结以上p型/i型/n型结构堆叠形成。该光电二极管第一结层p型/i型/n型材料仍然为非晶硅结构，第二结层以上p型/i型/n型材料可以为微晶结构、多晶结构或是掺有可扩展光敏波长范围之化合物材料。
当红外感应层为红外光敏二级管所形成的阵列时，在实际应用过程中，可藉由TFT作扫描驱动外加一偏压在p型/i型/n型光电二极管之间，将使得红外光敏二极管处于侦测红外光信号状态，用户眼球反射回的红外光信号转换为红外光影像电信号并输出，而实现对用户眼球活动信息的捕捉。
在某些实施例中，所述操作指令为画面切换指令，则步骤“判断操作配置信息中是否存在与所捕捉的眼球活动相匹配的眼球活动信息，若是则执行操作配置信息中该眼球活动信息对应的操作指令，否则不执行所述操作指令”包括：判断捕捉到的用户的眼球活动信息与画面切换指令对应的眼球活动信息是否匹配，若是则对画面进行切换，否则不对画面进行切换。由于视频流数据是有一帧帧图像画面构成的，因而本实施例的方法同样也适用于对视频流数据的判断。
在某些实施例中，眼球追踪操作的方法，其特征在于，步骤“判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息”具体包括：根据捕捉到的用户的眼球活动信息计算其特征值，并与配置信息中预设的眼球活动信息的特征值进行对比；当误差小于预设值时，判定为相匹配，否则判定为不匹配。
在某些实施例中，所述方法还包括步骤：当判定操作配置信息中不存在与所捕捉的眼球活动信息相匹配的眼球活动信息时，发出提示信息。所述提示信息包括声音提示信息、图像提示信息、光线提示信息、视频提示信息中的一种或多种。所述声音提示信息包括提示用户再次操作眼球活动项目的语音提示信息，所述图像提示信息包括提示用户再次操作眼球活动项目的弹窗提示信息， 所述视频提示信息包括提示用户再次操作眼球活动项目的动画提示信息，光线提示信息包括改变屏幕亮度或者让显示屏发出不同颜色的光线等。
如图4所示，在某些实施例中，当所述显示单元为LCD液晶显示屏时，所述传感单元的下方还设置有背光单元，所述传感单元设置于背光单元和LCD液晶显示屏之间。由于LCD液晶显示屏不属于自发光元件，因而在安装时需要在传感单元的下方增加背光单元。背光单元可以为LCD背光模组，也可以为其他具有自发光功能的电子元件。在另一些实施例中，当所述显示单元为AMOLED显示屏时，由于OLED显示屏属于自发光元件，因而无需设置背光单元。通过上述两种方案的设置，可以有效满足不同厂家的生产需求，提高装置的适用范围。
在本实施方式中，所述眼球活动识别区包括多个眼球活动识别子区域，每一眼球活动识别子区域的下方对应设置一传感单元。所述装置还包括传感单元控制电路，所述方法还包括：接收用户对眼球活动识别子区域的启动指令，传感单元控制电路开启所述眼球活动识别子区域的下方的传感单元，以及接收用户对眼球活动识别子区域的关闭指令，传感单元控制电路关闭所述眼球活动识别子区域的下方的传感单元。
以眼球活动识别子区域的数量为两个为例，两个眼球活动识别子区域可以一上一下或一左一右均匀分布于屏幕中，也可以以其他排列方式分布于屏幕中。下面对具有两个眼球活动识别子区域的装置的应用过程做具体说明：在使用过程中，用户通过启动指令，开启将两个眼球活动识别子区域都设置成开启状态，优选的实施例中，两个眼球活动识别子区域构成的范围覆盖了整个显示屏，这样可以保证当两个眼球活动识别子区域都设置成开启状态时，用户眼球的成像投影始终位于传感单元范围内，有效提高对用户眼球特征的捕捉，提升用户体验。在其他实施例中，两个眼球活动识别子区域构成的范围也可以占整个显示屏面积的2/3、3/4等，只需满足眼球活动识别子区域的中心不偏离眼球成像的光轴即可。当然，用户也可以根据自身喜好，设置某一个眼球识别子区域开启， 另一个眼球识别子区域关闭。在不需要对装置进行操作时，还可以将两个识别子区域均设置为关闭状态。
在其他实施例中，眼球活动识别子区域的数量还可以为其他数值，可以根据实际需要进行设置。各个眼球活动识别子区域下方的传感单元处于开启或关闭，可以根据用户自身喜好进行设置。
请参阅图3，为本发明一实施方式涉及的眼球追踪操作的装置的示意图。所述装置为具有触摸显示屏的电子设备，如是手机、平板电脑、个人数字助理等智能移动设备，还可以是个人计算机、工业装备用计算机等电子设备。当然所述装置还可以与光学成像器件相结合，光学成像器件设置于所述显示单元与用户眼睛之间，如图5所示，用户眼球投影先在光学成像器件中成像，成像的投影位于显示单元上眼球活动识别区范围内，进而被眼球活动识别下方的传感单元捕捉，通过光学成像器件与显示单元之间的配合，可以达到模拟VR/AR设备的效果。
所述装置包括显示单元101和传感单元102；所述显示单元101上设置有眼球活动识别区，所述传感单元102位于所述眼球活动识别区的下方，用于感知用户眼球反射的光信号，并捕捉用户的眼球活动信息；所述装置还包括操作信息设置单元104、判断单元108和处理单元106。在本实施方式中，所述显示单元包括AMOLED显示屏或LCD液晶显示屏；在其他实施方式中，显示单元也可以为其他具有显示功能的电子元件。
所述操作信息设置单元104用于预先设置操作配置信息，所述操作配置信息包括眼球活动信息与操作指令的对应关系。在本实施方式中，所述操作指令包括文字操作指令、图像操作指令、视频操作指令、应用操作指令中的一种或多种。所述文字操作指令包括选中文字指令、删除文字指令、复制文字指令等；所述图像操作指令包括选中图像指令、复制图像指令、截取图像指令、删除图像指令、切换图像画面等；所述视频操作指令包括对视频进行截取、暂停、保存、删除、快进、快退、缩放画面、音量调整等；所述应用操作指令包括对软 件应用程序(如手机APP)进行启动、删除、选中、移动等。所述眼球活动信息可以是眼球任意方向上进行运动的信息，包括用户双眼或单眼眼球从上向下或从下向上的直线运动、从左向右或从右向左的直线运动；眼球顺时针或逆时针转一圈或数圈的运动；上述眼球直线运动与直线运动的组合、转圈运动的组合、直线运动与转圈运动的组合，或者任何其他设备可以辨别的眼球运动。
操作配置信息中的眼球活动信息即为用户事先录入存储的眼球活动信息，每一眼球活动信息对应一操作指令，每一个操作指令可以与多个眼球活动信息相对应。操作配置信息可以存储于装置的存储单元，如手机的内存、计算机的硬盘中，也可以存储于服务器的存储单元中，当需要获取操作配置信息时，只需让装置与服务器建立通讯连接，而后再从服务器获取到事先存储的操作配置信息，所述通讯连接包括有线通讯连接或无线通信连接。
所述传感单元102用于捕捉用户在眼球活动识别区上的眼球活动信息。在本实施方式中，传感单元的覆盖的范围与显示单元的大小相适配，优选的，传感单元的形状为矩形，矩形的大小位于显示单元的中心，保证不偏移眼球活动成像的光轴。这样可以保证只要用户眼睛对准显示单元，无论用户眼球如何活动，传感单元都能及时采集到用户的眼球活动信息。
所述判断单元108用于判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息，若是则处理单元106用于执行操作配置信息中该眼球活动信息对应的操作指令，否则处理单元106不执行所述操作指令。眼球活动信息的比对可以通过眼球特征识别算法来实现，眼球特征识别算法可以实现存储于装置的存储单元中，当传感单元获取到眼球活动识别区上的眼球活动信息后，装置的处理器将调用存储单元中的眼球特征识别算法，将所获取的眼球活动信息与预设的操作配置信息中的眼球活动信息进行比对，判断两者是否匹配。眼球特征算法包括对眼球特征信息进行预处理、数据特征提取、特征匹配、眼球运动轨迹识别等步骤，可以用多种算法来实现，这些算法都是成熟的现有技术，现已被应用于各个领域中，此处不再展开赘述。
在某些实施例中，所述装置包括操作指令接收单元105，所述“操作信息设置单元用于预先设置操作配置信息”包括：所述操作信息设置单元用于接收用户设置命令。所述显示单元用于显示眼球活动识别区。设置指令可以通过用户点击屏幕上设置栏中的某一按钮触发，装置接收到设置指令后，将对眼球活动识别区进行显示，便于用户输入眼球活动信息。在本实施方式中，显示眼球活动识别区可以包括：提高眼球活动识别区的亮度或在眼球活动识别区上显示一提示输入框。在某些实施例中，在接收用户设置指令之前，还包括接收用户的账号信息，所述账号信息包括用户ID及密码。用户需要输入正确的用户ID及密码，登录用户账号后，才可触发所述设置指令，这样一方面可以提高操作配置信息设置的安全性，另一方面也可以达到在一个装置上区分不同用户、保存不同的眼球活动信息以及与之相对应的操作指令的效果。
所述操作指令接收单元用于接收用户选择的眼球操作项目，所述传感单元用于捕捉用户在所述眼球识别区上的眼球活动信息并保存。眼球操作项目可以是系统默认的眼球操作项目，也可以通过用户自定义决定。眼球操作项目可以以文字、图片或视频的方式展现给用户，让用户点击选择。例如可以显示“两个眼球同时向左运动”、“两个眼球同时向右运动”、“两个眼球同时向上运动”、“两个眼球顺时针转动一圈”等等文字供用户选择，也可以将眼球操作项目绘制成图片或制作成教程的视频展现给用户，以便用户可以准确地选择眼球操作项目。当眼球操作项目确认后，就可以对用户当前的眼球活动信息进行采集，所采集到的眼球活动信息即为预设眼球活动信息，可以将其存储于存储单元中。本实施方式中，所述步骤“捕捉用户在所述眼球识别区上的眼球活动信息并保存”包括：判断用户设置过程中的眼球活动信息是否已存储于存储单元，当判定为是时提示用户该眼球活动信息已录入；当判定为否时将该眼球活动信息保存至存储单元。这样可以有效避免眼球活动信息的重复录入。
所述显示单元用于显示一操作指令标识列表，所述操作指令接收单元还用于接收用户对操作指令标识的选择指令，建立所选中的操作指令标识对应的操 作指令与所捕捉的用户的眼球活动信息的对应关系并保存。所述操作指令标识列表中包含着一个或多个操作指令对应的标识，每一操作指令标识对应一操作指令。操作指令标识可以以文字或图片的形式予以显示。选择指令可以通过用户点击勾选、双击等方式触发。
为了给用户呈现更好的视频画面场景，所述装置可以与VR/AR头盔相结合，提供相对封闭黑暗的操作空间，给用户身临其境之感，由于在黑暗条件下，可见光没有进入人眼，传感单元也就无法捕捉到用户反射回来的光信号，进而捕捉用户的眼球活动信息。因此，在某些实施例中，所述装置还包括红外光源，所述红外光源用于发出红外光。红外光源为具有发出红外光的电子元件，如LED红外器件，在设计时可以将红外光源设置在移动终端与显示面板相同一侧，以保证用户眼球反射回的红外光信号可以被显示屏下方的传感单元所捕捉。
如图6所示，在本实施方式中，所述传感单元包括TFT影像感测阵列薄膜。现有的液晶显示(LCD)面板或有机发光二极管(OLED)显示面板，皆是以TFT结构驱动扫描单一像素，以实现面板上像素阵列的显示功能。形成TFT开关功能的主要结构为金属氧化物半导体场效晶体管(MOSFET)，其中熟知的半导体层材料主要有非晶硅、多晶硅、氧化铟镓锌(IGZO)、或是混有碳纳米材料之有机化合物等等。由于光感测二极管的结构亦可采用此类半导体材料制备，且生产设备也兼容于TFT阵列的生产设备，因此近年来TFT光侦测二极管开始以TFT阵列制备方式进行生产。本实施方式所述的TFT影像感测阵列薄膜即为上述提到的TFT光侦测二极管(如图6中的光感测二极管区域部分)，具体结构可以参考美国专利US6943070B2、中华人民共和国专利CN204808361U中对传感单元结构的描述。TFT影像感测阵列薄膜的生产工艺与显示面板TFT结构不同的是：原本在显示面板的像素开口区域，在生产工艺上改为光感测区域。其TFT制备方式可以采用薄型玻璃为基材，亦可采用耐高温塑性材料为基材，如美国专利US6943070B2所述。
图6所示的传感单元易受周围环境光或者显示屏像素所发出的可见光的反 射、折射等因素影响，造成光学干扰，严重影响内嵌于显示面板下方的TFT影像感测阵列薄膜的信号噪声比(SNR)，为了提高信号噪声比，本发明的传感单元在图6所示的传感单元的基础上做了进一步改进，使得TFT影像感测阵列薄膜可以侦测识别用户眼球反射回的红外信号，所述传感单元包括红外感应层，所述红外感应层用于接收用户眼球反射的红外光信号以捕捉用户的眼球活动信息。
如图7所示，在某些实施例中，所述红外感应层为红外光敏电晶管所形成的阵列。为了将TFT影像感测阵列薄膜能够识别的光信号波长从可见光范围扩展至红外光范围，对图6的TFT影像感测阵列薄膜进行改进，具体包括：采取TFT光伏场效晶体管(PVFET)取代原有的阵列薄膜中的光侦测二极管，具体可以参考文献“Photovoltage field-effect transistors，Nature 542,324–327(16February2017)”。在本实施例中，红外感应层亦设计为场效晶体管(Field Effect Transistor)结构，为了制备此晶体管结构，且满足制备的晶体管结构完全兼容于图6所示的TFT影像感测阵列薄膜，可以通过以下方式来实现：以化学汽像沉积制备非晶硅层、物理溅镀制备金属层与透明电极层，在制备过程中以光罩微影蚀刻定义出各层器件需要之微影形状图层。
制备得出的红外光敏电晶管的漏极与源极之间的电位差即为将红外光转为电信号之操作参数。为达到此目的，红外光敏电晶管的栅极采用对红外光较为敏感的光伏材料制成，例如光伏材料可以采用以能隙工程(Bandgap Engineering)改质过之量子点、碳纳米材料、金属氧化物薄膜材料等，将这些材料混入有机或无机化合物搅拌成胶质或液态，再采用涂布、印刷等方式制备得到作为栅极的材料。
当用户眼球反射回的红外光射入上述红外光敏电晶管所形成的阵列时，光伏材料所激发的电子电洞对(Electron-Hole Pair)诱发红外光敏电晶管的漏极与源极之间的电子通道形成，漏极与源极之间的电位差会因此下降，以TFT作扫描驱动漏极输出电流升高，此开关特性可实现红外光影像电信号之读出，从而实 现对用户眼球活动信息的捕捉。
如图8所示，在某些实施例中，所述红外感应层为红外光敏二级管所形成的阵列。为了将TFT影像感测阵列薄膜能够识别的光信号波长从可见光范围扩展至红外光范围，对图6的TFT影像感测阵列薄膜进行改进，具体是采用红外光敏二级管替换图6中TFT影像感测阵列薄膜的光二极管层，红外光敏二极管包括微晶硅光电二极管或非晶硅光电二极管。
实施例一：将非结晶硅p型/i型/n型光电二极管结构(图6中的光二极管层)改由微晶硅p型/i型/n型光电二极管结构。在此实施例中，光电二极管的微结晶程度主要是在化学气象沉积过程中，以适当氢气浓度混入气体硅烷(SiH4)去控制氢原子键结非晶硅之悬空键(dangling bond)，以实现微晶硅p型/i型/n型光电二极管结构之镀膜。藉由调整化学气象沉积的氢气浓度，微晶光电二极管的操作波长范围可以扩展到光波长600nm到1000nm的范围。
在采用微晶光电二极管之实施例中，为了进一步地提高光电转换之量子效率，微晶硅光电二极管也可采用双结以上p型/i型/n型结构堆叠形成。该光电二极管第一结层p型/i型/n型材料仍然为非结晶结构，第二结层以上p型/i型/n型材料可以为微晶结构、多晶结构。
实施例二：将非结晶硅p型/i型/n型光电二极管结构(图6中的光二极管层)改为掺有可扩展光敏波长范围之非结晶硅化合物之p型/i型/n型光电二极管结构，优选之化合物实施例为非晶硅化锗。在此实施例中，光电二极管的本质层(i型)在以化学气象沉积镀膜过程中，通以气体锗烷(GeH4)混入硅烷(SiH4)，以实现非结晶硅化锗p型/i型/n型光电二极管之光敏范围达到光波长600nm到1000nm的范围。
在采用非结晶硅化合物光电二极管之实施例中，为了提高光电转换之量子效率，非晶硅光电二极管也可采用双结以上p型/i型/n型结构堆叠形成。该光电二极管第一结层p型/i型/n型材料仍然为非晶硅结构，第二结层以上p型/i型/n型材料可以为微晶结构、多晶结构或是掺有可扩展光敏波长范围之化合物 材料。
当红外感应层为红外光敏二级管所形成的阵列时，在实际应用过程中，可藉由TFT作扫描驱动外加一偏压在p型/i型/n型光电二极管之间，将使得红外光敏二极管处于侦测红外光信号状态，用户眼球反射回的红外光信号转换为红外光影像电信号并输出，而实现对用户眼球活动信息的捕捉。
在某些实施例中，所述操作指令为画面切换指令，所述“判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动相匹配的眼球活动信息，若是则处理单元用于执行操作配置信息中该眼球活动信息对应的操作指令，否则处理单元不执行所述操作指令”包括：判断单元用于判断捕捉到的用户的眼球活动信息与画面切换指令对应的眼球活动信息是否匹配，若是则处理单元用于对画面进行切换，否则处理单元不对画面进行切换。
在某些实施例中，步骤“判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息”具体包括：判断单元用于根据捕捉到的用户的眼球活动信息计算其特征值，并与配置信息中预设的眼球活动信息的特征值进行对比；当误差小于预设值时，判定为相匹配，否则判定为不匹配。
在某些实施例中，所述处理单元还用于在判断单元判定操作配置信息中不存在与所捕捉的眼球活动信息相匹配的眼球活动信息时，发出提示信息。所述提示信息包括声音提示信息、图像提示信息、光线提示信息、视频提示信息中的一种或多种。所述声音提示信息包括提示用户再次操作眼球活动项目的语音提示信息，所述图像提示信息包括提示用户再次操作眼球活动项目的弹窗提示信息，所述视频提示信息包括提示用户再次操作眼球活动项目的动画提示信息，光线提示信息包括改变屏幕亮度或者让显示屏发出不同颜色的光线等。
如图4所示，在某些实施例中，当所述显示单元为LCD液晶显示屏时，所述传感单元的下方还设置有背光单元，所述传感单元设置于背光单元和LCD液晶显示屏之间。由于LCD液晶显示屏不属于自发光元件，因而在安装时需要在 传感单元的下方增加背光单元103。背光单元可以为LCD背光模组，也可以为其他具有自发光功能的电子元件。在另一些实施例中，当所述显示单元为AMOLED显示屏时，由于OLED显示屏属于自发光元件，因而无需设置背光单元。通过上述两种方案的设置，可以有效满足不同厂家的生产需求，提高装置的适用范围。
在本实施方式中，所述眼球活动识别区包括多个眼球活动识别子区域，每一眼球活动识别子区域的下方对应设置一传感单元。所述装置还包括传感单元控制电路，所述方法还包括：接收用户对眼球活动识别子区域的启动指令，传感单元控制电路开启所述眼球活动识别子区域的下方的传感单元，以及接收用户对眼球活动识别子区域的关闭指令，传感单元控制电路关闭所述眼球活动识别子区域的下方的传感单元。
以眼球活动识别子区域的数量为两个为例，两个眼球活动识别子区域可以一上一下或一左一右均匀分布于屏幕中，也可以以其他排列方式分布于屏幕中。下面对具有两个眼球活动识别子区域的装置的应用过程做具体说明：在使用过程中，用户通过启动指令，开启将两个眼球活动识别子区域都设置成开启状态，优选的实施例中，两个眼球活动识别子区域构成的范围覆盖了整个显示屏，这样可以保证当两个眼球活动识别子区域都设置成开启状态时，用户眼球的成像投影始终位于传感单元范围内，有效提高对用户眼球特征的捕捉，提升用户体验。在其他实施例中，两个眼球活动识别子区域构成的范围也可以占整个显示屏面积的2/3、3/4等，只需满足眼球活动识别子区域的中心不偏离眼球成像的光轴即可。当然，用户也可以根据自身喜好，设置某一个眼球识别子区域开启，另一个眼球识别子区域关闭。在不需要对装置进行操作时，还可以将两个识别子区域均设置为关闭状态。
在其他实施例中，眼球活动识别子区域的数量还可以为其他数值，可以根据实际需要进行设置。各个眼球活动识别子区域下方的传感单元处于开启或关闭，可以根据用户自身喜好进行设置。
本发明具有以下优点：通过在显示单元的眼球识别区下方设置传感单元，用户眼球活动通过光学器件成像的投影位于所述眼球识别区上，传感单元设置于眼球成像光轴位置或是近轴位置，相较于摄像头独立于显示屏设置在边缘位置的结构，本发明可以更精准且及时捕捉到用户眼球活动信息，进而与预设的眼球活动信息进行比对，执行该眼球活动信息对应的操作指令，使得用户通过眼球活动就可快速对设备进行操控，有效提升用户体验。此外，将传感单元设置于显示单元的下方，相较于摄像头独立突出设置于显示屏外的结构，可以有效缩小移穿戴式装备或是动设备的整体厚度，使得穿戴式装备或是移动设备更加轻薄、满足市场的需求。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者终端设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者终端设备所固有的要素。在没有更多限制的情况下，由语句“包括……”或“包含……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者终端设备中还存在另外的要素。此外，在本文中，“大于”、“小于”、“超过”等理解为不包括本数；“以上”、“以下”、“以内”等理解为包括本数。
本领域内的技术人员应明白，上述各实施例可提供为方法、装置、或计算机程序产品。这些实施例可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。上述各实施例涉及的方法中的全部或部分步骤可以通过程序来指令相关的硬件来完成，所述的程序可以存储于计算机设备可读取的存储介质中，用于执行上述各实施例方法所述的全部或部分步骤。所述计算机设备，包括但不限于：个人计算机、服务器、通用计算机、专用计算机、网络设备、嵌入式设备、可编程设备、智能移动终端、智能家居设备、穿戴式 智能设备、车载智能设备等；所述的存储介质，包括但不限于：RAM、ROM、磁碟、磁带、光盘、闪存、U盘、移动硬盘、存储卡、记忆棒、网络服务器存储、网络云存储等。
上述各实施例是参照根据实施例所述的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到计算机设备的处理器以产生一个机器，使得通过计算机设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机设备以特定方式工作的计算机设备可读存储器中，使得存储在该计算机设备可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机设备上，使得在计算机设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已经对上述各实施例进行了描述，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改，所以以上所述仅为本发明的实施例，并非因此限制本发明的专利保护范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围之内。

Claims (26)

  1. 一种眼球追踪操作的方法，其特征在于，所述方法应用于眼球追踪操作的装置，所述装置包括显示单元和传感单元；所述显示单元上设置有眼球活动识别区，所述传感单元位于所述眼球活动识别区的下方，用于感知用户眼球反射的光信号，并捕捉用户的眼球活动信息，所述方法包括以下步骤：
    预先设置操作配置信息，所述操作配置信息包括眼球活动信息与操作指令的对应关系；
    捕捉用户在眼球活动识别区上的眼球活动信息，判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息，若是则执行操作配置信息中该眼球活动信息对应的操作指令，否则不执行所述操作指令。
  2. 如权利要求1所述的眼球追踪操作的方法，其特征在于，所述步骤“预先设置操作配置信息”包括：
    接收用户设置命令，显示眼球活动识别区；
    接收用户选择的眼球操作项目，以及捕捉用户在所述眼球识别区上的眼球活动信息并保存；
    显示一操作指令标识列表，所述操作指令标识列表中包含着一个或多个操作指令对应的标识，每一操作指令标识对应一操作指令；
    接收用户对操作指令标识的选择指令，建立所选中的操作指令标识对应的操作指令与所捕捉的用户的眼球活动信息的对应关系并保存。
  3. 如权利要求1或2所述的眼球追踪操作的方法，其特征在于，所述操作指令包括文字操作指令、图像操作指令、视频操作指令、应用操作指令中的一种或多种。
  4. 如权利要求1所述的眼球追踪操作的方法，其特征在于，所述装置还包括红外光源，所述红外光源用于发出红外光，所述传感单元包括红外感应层，所述红外感应层用于接收用户眼球反射的红外光信号以捕捉用户的眼球活动信息。
  5. 如权利要求4所述的眼球追踪操作的方法，其特征在于，所述操作指令 为画面切换指令，则步骤“判断操作配置信息中是否存在与所捕捉的眼球活动相匹配的眼球活动信息，若是则执行操作配置信息中该眼球活动信息对应的操作指令，否则不执行所述操作指令”包括：
    判断捕捉到的用户的眼球活动信息与画面切换指令对应的眼球活动信息是否匹配，若是则对画面进行切换，否则不对画面进行切换。
  6. 如权利要求4所述的眼球追踪操作的方法，其特征在于，所述传感单元包括TFT影像感测阵列薄膜，所述红外感应层为红外光敏二极管或红外光敏电晶管所形成的阵列。
  7. 如权利要求1所述的眼球追踪操作的方法，其特征在于，步骤“判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息”具体包括：
    根据捕捉到的用户的眼球活动信息计算其特征值，并与配置信息中预设的眼球活动信息的特征值进行对比；当误差小于预设值时，判定为相匹配，否则判定为不匹配。
  8. 如权利要求1或7所述的眼球追踪操作的方法，其特征在于，所述方法还包括步骤：
    当判定操作配置信息中不存在与所捕捉的眼球活动信息相匹配的眼球活动信息时，发出提示信息。
  9. 如权利要求8所述的眼球追踪操作的方法，其特征在于，所述提示信息包括声音提示信息、图像提示信息、光线提示信息、视频提示信息中的一种或多种。
  10. 如权利要求1所述的眼球追踪操作的方法，其特征在于，所述显示单元包括AMOLED显示屏或LCD液晶显示屏。
  11. 如权利要求10所述的眼球追踪操作的方法，其特征在于，当所述显示单元为LCD液晶显示屏时，所述传感单元的下方还设置有背光单元，所述传感单元设置于背光单元和LCD液晶显示屏之间。
  12. 如权利要求1所述的眼球追踪操作的方法，其特征在于，所述眼球活动识别区包括多个眼球活动识别子区域，每一眼球活动识别子区域的下方对应设置一传感单元。
  13. 如权利要求12所述的眼球追踪操作的方法，其特征在于，所述装置还包括传感单元控制电路，所述方法还包括：
    接收用户对眼球活动识别子区域的启动指令，传感单元控制电路开启所述眼球活动识别子区域的下方的传感单元，以及接收用户对眼球活动识别子区域的关闭指令，传感单元控制电路关闭所述眼球活动识别子区域的下方的传感单元。
  14. 一种眼球追踪操作的装置，其特征在于，所述装置包括显示单元和传感单元；所述显示单元上设置有眼球活动识别区，所述传感单元位于所述眼球活动识别区的下方，用于感知用户眼球反射的光信号，并捕捉用户的眼球活动信息；所述装置还包括操作信息设置单元、判断单元和处理单元；
    所述操作信息设置单元用于预先设置操作配置信息，所述操作配置信息包括眼球活动信息与操作指令的对应关系；
    所述传感单元用于捕捉用户在眼球活动识别区上的眼球活动信息，所述判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息，若是则处理单元用于执行操作配置信息中该眼球活动信息对应的操作指令，否则处理单元不执行所述操作指令。
  15. 如权利要求14所述的眼球追踪操作的装置，其特征在于，所述装置包括操作指令接收单元，所述“操作信息设置单元用于预先设置操作配置信息”包括：
    所述操作信息设置单元用于接收用户设置命令，所述显示单元用于显示眼球活动识别区；
    所述操作指令接收单元用于接收用户选择的眼球操作项目，所述传感单元用于捕捉用户在所述眼球识别区上的眼球活动信息并保存；
    所述显示单元用于显示一操作指令标识列表，所述操作指令标识列表中包含着一个或多个操作指令对应的标识，每一操作指令标识对应一操作指令；
    所述操作指令接收单元还用于接收用户对操作指令标识的选择指令，建立所选中的操作指令标识对应的操作指令与所捕捉的用户的眼球活动信息的对应关系并保存。
  16. 如权利要求14或15所述的眼球追踪操作的装置，其特征在于，所述操作指令包括文字操作指令、图像操作指令、视频操作指令、应用操作指令中的一种或多种。
  17. 如权利要求14所述的眼球追踪操作的装置，其特征在于，所述装置还包括红外光源，所述红外光源用于发出红外光，所述传感单元包括红外感应层，所述红外感应层用于接收用户眼球反射的红外光信号以捕捉用户的眼球活动信息。
  18. 如权利要求17所述的眼球追踪操作的装置，其特征在于，所述操作指令为画面切换指令，所述“判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动相匹配的眼球活动信息，若是则处理单元用于执行操作配置信息中该眼球活动信息对应的操作指令，否则处理单元不执行所述操作指令”包括：
    判断单元用于判断捕捉到的用户的眼球活动信息与画面切换指令对应的眼球活动信息是否匹配，若是则处理单元用于对画面进行切换，否则处理单元不对画面进行切换。
  19. 如权利要求17所述的眼球追踪操作的装置，其特征在于，所述传感单元包括TFT影像感测阵列薄膜，所述红外感应层为红外光敏二极管或红外光敏电晶管所形成的阵列。
  20. 如权利要求14所述的眼球追踪操作的装置，其特征在于，“判断单元用于判断操作配置信息中是否存在与所捕捉的眼球活动信息相匹配的眼球活动信息”具体包括：
    判断单元用于根据捕捉到的用户的眼球活动信息计算其特征值，并与配置 信息中预设的眼球活动信息的特征值进行对比；当误差小于预设值时，判定为相匹配，否则判定为不匹配。
  21. 如权利要求14或20所述的眼球追踪操作的装置，其特征在于，所述处理单元还用于在判断单元判定操作配置信息中不存在与所捕捉的眼球活动信息相匹配的眼球活动信息时，发出提示信息。
  22. 如权利要求21所述的眼球追踪操作的装置，其特征在于，所述提示信息包括声音提示信息、图像提示信息、光线提示信息、视频提示信息中的一种或多种。
  23. 如权利要求14所述的眼球追踪操作的装置，其特征在于，所述显示单元包括AMOLED显示屏或LCD液晶显示屏。
  24. 如权利要求23所述的眼球追踪操作的装置，其特征在于，当所述显示单元为LCD液晶显示屏时，所述传感单元的下方还设置有背光单元，所述传感单元设置于背光单元和LCD液晶显示屏之间。
  25. 如权利要求14所述的眼球追踪操作的装置，其特征在于，所述眼球活动识别区包括多个眼球活动识别子区域，每一眼球活动识别子区域的下方对应设置一传感单元。
  26. 如权利要求25所述的眼球追踪操作的装置，其特征在于，所述装置还包括传感单元控制电路和操作指令接收单元，所述操作指令接收单元用于接收用户对眼球活动识别子区域的启动指令，所述传感单元控制电路用于开启所述眼球活动识别子区域的下方的传感单元，以及所述操作指令接收单元用于接收用户对眼球活动识别子区域的关闭指令，所述传感单元控制电路用于关闭所述眼球活动识别子区域的下方的传感单元。
PCT/CN2018/078091 2017-04-20 2018-03-06 一种眼球追踪操作的方法和装置 WO2018192312A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710260321.9A CN108733204A (zh) 2017-04-20 2017-04-20 一种眼球追踪操作的方法和装置
CN201710260321.9 2017-04-20

Publications (1)

Publication Number Publication Date
WO2018192312A1 true WO2018192312A1 (zh) 2018-10-25

Family

ID=63855568

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/078091 WO2018192312A1 (zh) 2017-04-20 2018-03-06 一种眼球追踪操作的方法和装置

Country Status (3)

Country Link
CN (1) CN108733204A (zh)
TW (1) TWI681339B (zh)
WO (1) WO2018192312A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113504833A (zh) * 2021-09-10 2021-10-15 世纳微电子科技(成都)有限公司 数字光学色温传感器、眼球追踪装置及人机交互系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111639017B (zh) * 2020-05-29 2024-05-07 京东方科技集团股份有限公司 测量眼球追踪设备延迟量的方法、设备、眼球追踪系统
CN113325585B (zh) * 2021-05-27 2023-03-28 京东方科技集团股份有限公司 显示装置、可穿戴显示设备及注视点的位置的确定方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105759945A (zh) * 2014-12-17 2016-07-13 富泰华工业（深圳）有限公司 穿戴设备控制系统及方法
CN105867605A (zh) * 2015-12-15 2016-08-17 乐视致新电子科技（天津）有限公司 虚拟现实头盔的功能菜单翻页方法、装置及头盔
CN106503514A (zh) * 2016-09-28 2017-03-15 北京用友政务软件有限公司 基于虹膜识别的电子终端设备的解锁方法及系统
CN106557737A (zh) * 2016-09-27 2017-04-05 北京无线电计量测试研究所 一种便携式虹膜识别装置

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8602789B2 (en) * 2008-10-14 2013-12-10 Ohio University Cognitive and linguistic assessment using eye tracking
TW201310294A (zh) * 2011-08-24 2013-03-01 Acer Inc 手持裝置和調整位置資訊方法
TW201344554A (zh) * 2012-04-16 2013-11-01 Skymedi Corp 切換顯示模式的系統及其方法
CN103593051B (zh) * 2013-11-11 2017-02-15 百度在线网络技术（北京）有限公司 头戴式显示设备
CN103616953A (zh) * 2013-11-27 2014-03-05 福州瑞芯微电子有限公司 一种屏幕解锁和应用启动的方法及装置
TWI521384B (zh) * 2014-10-31 2016-02-11 佳世達科技股份有限公司 具有眼球追蹤功能之電子裝置及其控制方法
US9870049B2 (en) * 2015-07-31 2018-01-16 Google Llc Reflective lenses to auto-calibrate a wearable system
CN105975933A (zh) * 2016-05-05 2016-09-28 上海聚虹光电科技有限公司 基于透明屏幕的虹膜识别系统
CN106055102A (zh) * 2016-05-30 2016-10-26 北京奇艺世纪科技有限公司 一种虚拟现实设备控制方法及装置
CN105975136B (zh) * 2016-06-30 2019-03-26 京东方科技集团股份有限公司 显示基板及其制造方法和显示装置
CN106372587A (zh) * 2016-08-29 2017-02-01 乐视控股（北京）有限公司 一种指纹识别方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105759945A (zh) * 2014-12-17 2016-07-13 富泰华工业（深圳）有限公司 穿戴设备控制系统及方法
CN105867605A (zh) * 2015-12-15 2016-08-17 乐视致新电子科技（天津）有限公司 虚拟现实头盔的功能菜单翻页方法、装置及头盔
CN106557737A (zh) * 2016-09-27 2017-04-05 北京无线电计量测试研究所 一种便携式虹膜识别装置
CN106503514A (zh) * 2016-09-28 2017-03-15 北京用友政务软件有限公司 基于虹膜识别的电子终端设备的解锁方法及系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113504833A (zh) * 2021-09-10 2021-10-15 世纳微电子科技(成都)有限公司 数字光学色温传感器、眼球追踪装置及人机交互系统

Also Published As

Publication number Publication date
TWI681339B (zh) 2020-01-01
CN108733204A (zh) 2018-11-02
TW201839668A (zh) 2018-11-01

Similar Documents

Publication Publication Date Title
TWI696957B (zh) 同步採集指紋信息的方法和裝置
US11911133B2 (en) Operation method and device for physiological health detection
WO2018192312A1 (zh) 一种眼球追踪操作的方法和装置
US11137845B2 (en) Method and device for recognizing contact of foldable display screen
TWI681382B (zh) 終端及終端顯示亮度調節的方法
US20160035138A1 (en) Transparent display device and control method thereof
US11314962B2 (en) Electronic device and method for controlling fingerprint recognition-based electronic device
TW202004542A (zh) 一種同步驗證指紋資訊的螢幕解鎖方法和裝置
TWI730227B (zh) 一種眼球追蹤操作的方法和裝置
JP7493514B2 (ja) 機能パネル、表示装置、入出力装置、情報処理装置
CN112151583A (zh) 功能面板、半导体装置、显示装置、输入/输出装置、数据处理装置
US11250257B2 (en) Method and device for iris recognition
JP2021089423A (ja) 機能パネル、表示装置、入出力装置、情報処理装置
TWI715832B (zh) 一種虹膜識別的方法和裝置
CN108963015B (zh) 一种光侦测薄膜、器件、显示装置、光敏二极管的制备方法
US11914807B2 (en) Method and device for biometric recognition
JP7474256B2 (ja) 発光デバイス、機能パネル、表示装置、入出力装置、情報処理装置
CN108376531A (zh) 一种显示面板及其控制方法、显示装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18787441

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18787441

Country of ref document: EP

Kind code of ref document: A1