WO2006066474A1 - Procede et systeme de commande du curseur d'un afficheur - Google Patents

Procede et systeme de commande du curseur d'un afficheur Download PDF

Info

Publication number
WO2006066474A1
WO2006066474A1 PCT/CN2005/001020 CN2005001020W WO2006066474A1 WO 2006066474 A1 WO2006066474 A1 WO 2006066474A1 CN 2005001020 W CN2005001020 W CN 2005001020W WO 2006066474 A1 WO2006066474 A1 WO 2006066474A1
Authority
WO
WIPO (PCT)
Prior art keywords
cursor
display device
user
signal
unit
Prior art date
Application number
PCT/CN2005/001020
Other languages
English (en)
French (fr)
Inventor
Zhongqing Li
Dongyang Wang
Original Assignee
Lenovo (Beijing) Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lenovo (Beijing) Limited filed Critical Lenovo (Beijing) Limited
Priority to DE112005003221T priority Critical patent/DE112005003221B4/de
Priority to JP2007545815A priority patent/JP2008523517A/ja
Priority to US11/722,127 priority patent/US20090213071A1/en
Publication of WO2006066474A1 publication Critical patent/WO2006066474A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects

Definitions

  • the present invention relates to electronic information and data processing techniques, and more particularly to a control system and method for a cursor of a display device. Background technique
  • the current computer operating system supports up to eight display devices, and this number will increase with the advancement of technology; many workstations are equipped with up to six display devices, even more, even if Arrange the two lines, move the cursor from the upper left corner to the lower right corner, and the moving distance of the cursor will also reach about 1 meter.
  • Such a long-distance cursor movement if controlled by a conventional mouse or keyboard, will inevitably increase the fatigue of the user's wrist, and is not conducive to the improvement of work efficiency.
  • cursor movement For large-screen PC users and widescreen notebook users, there are similar problems with cursor movement.
  • the technical solution disclosed in the patent No. 03224405 is: by operating a moving point light source generator, the optical signal is taken at different positions of the light source propagation medium, and the signal is captured and converted by the CCD image sensor. Through the processor analysis processing, the movement of the cursor on the display screen of the output device is controlled by the direction detector.
  • this prior art still requires the user to operate the moving point source generator, and thus does not solve the aforementioned problems well.
  • the technical solution disclosed in the patent application No. 99111508 is: inputting a line image through an image input device and generating an image signal, the image signal is converted into position coordinates by the image processing circuit, and a cursor control signal is generated according to the position coordinates.
  • this prior art does not well meet the need for the cursor to quickly switch between multiple screens or to quickly switch between different areas on a display device, and may result in the occurrence of excessive computer resources.
  • the technical solution disclosed in the patent application No. 01110375 is: capturing an image by a photoelectric mechanism that receives an image signal, and the controller processes the captured image and controls the movement of the cursor on the screen or the display accordingly.
  • this prior art also does not well meet the need for the cursor to quickly switch between multiple screens or to quickly switch between different areas on a display device, and may result in the occurrence of excessive computer resources.
  • the technical problem to be solved by the present invention is to provide a control system and method for a cursor of a display device, which can realize fast switching of a cursor, thereby effectively improving user work efficiency and reducing fatigue of the wrist.
  • the present invention can also activate the application window in the target area when the cursor position is switched.
  • the technical solution of the present invention is to provide a control system for a cursor of a display device, where the display device is connected to the data processing device;
  • An image acquisition device configured to collect image information of a user and send a signal
  • a signal receiving unit configured to receive a signal from the image capturing device
  • a signal processing unit configured to parse the signal, determine whether a cursor needs to be switched, and a target area where the cursor arrives;
  • a cursor control unit configured to send a cursor control signal to the display device according to the determination result of the signal processing unit, to switch the cursor to the target area.
  • the system further comprises a window activation unit, configured to issue a window activation signal to the display device when the application window exists in the target area, and activate the application window.
  • a window activation unit configured to issue a window activation signal to the display device when the application window exists in the target area, and activate the application window.
  • the signal receiving unit, the signal processing unit, the cursor control unit and the window activation unit are separate devices or at least two units are integrated with each other.
  • the signal receiving unit, the signal processing unit, the cursor control unit, and the window activation unit are disposed within the data processing device.
  • the signal receiving unit, the signal processing unit, the cursor control unit and the window activation unit constitute a control software system of the data processing device.
  • the signal processing unit is disposed within the image acquisition device; the signal receiving unit, the cursor control unit, and the window activation unit are disposed within the data processing device.
  • the target area determined by the signal processing unit is one of a plurality of display devices or one of a plurality of screen areas of the display device.
  • the image information of the user acquired by the image acquisition device is a facial feature of the user; the signal processing unit performs the determination according to the current facial feature and the set reference facial feature.
  • the image acquisition device is disposed within a display device or a data processing device.
  • the present invention further provides a method for controlling a cursor of a display device, wherein the display device is connected to a data processing device; and the method includes the following steps:
  • step 2 2) comparing and analyzing the current image information, determining whether the range of cursor change is reached, and if yes, proceeding to step 3); if not, returning to step 1);
  • the step of collecting the reference image information of the user is further included; the comparing and analyzing in the step 2) is specifically determining the implementation of the user according to the difference between the current image information and the reference image information. Whether it has changed.
  • the method further comprises checking the number of display devices or setting a screen area of the display device.
  • the method further comprises activating an application window of the target area to which the cursor arrives.
  • the image information refers to a facial feature of the user.
  • the beneficial effects of the present invention are: Since the present invention controls the cursor on the display device according to the processing information by collecting and processing the image information of the user, such as quickly switching the cursor, the user can quickly display the plurality of display devices. Switching the cursor between them can also quickly change the cursor position on a display device, thereby saving the user's time to move the mouse over a long distance, improving work efficiency, and also reducing the fatigue of the user repeatedly operating the mouse over long distances and promoting physical health.
  • the application window can be activated to further improve the work efficiency, and at the same time give the user a good feeling that the computer can comprehend the user's intention, because the direction of the facial features such as the user's eyes represents the user's desire.
  • the object of the operation is also determined in real time whether an application window exists in the target area, If it exists, the application window can be activated to further improve the work efficiency, and at the same time give the user a good feeling that the computer can comprehend the user's intention, because the direction of the facial features such as the user's eyes represents the user's desire.
  • the cursor can be switched over a long distance between one or more screens without increasing the hardware cost, and the application window in the target area is activated. Function, suitable for a large number of promotion.
  • the present invention is a computer aided tool for switching the position of a cursor and activating an application window within a destination area by processing the orientation of the user's face. Further, the present invention has high practicability, a large user group, and can be used by people of all ages.
  • FIG. 1 is a block diagram of a control system for a cursor position of a display device of the present invention
  • FIG. 2 is a block diagram of a data processing device in the control system of the present invention.
  • FIG. 3 is a flow chart of the control method of the present invention.
  • FIG. 4 is a schematic diagram showing the composition of a first embodiment of the control system of the present invention.
  • FIG. 5 is a block diagram showing the composition of the control software system in the computer host shown in Figure 4;
  • Figure 6 is a schematic view of the display partition shown in Figure 4.
  • Figure 7 is a schematic illustration of a screen and a camera in a second embodiment of the present invention. detailed description
  • the invention is a computer-aided system, the main idea of which is: processing user image information by collecting user image information, and controlling the cursor position on the display device according to the processing information; on the basis of the present invention, the invention can also activate the target Application window within the zone.
  • the present invention relates to electronic information, digital signal processing.
  • a control system for a cursor position of a display device of the present invention is used to control a cursor displayed by the display device 120, including a data processing device 110 and an image capture device 130.
  • the data processing device 110 connects and controls the display device 120 and Image acquisition device 130.
  • the display device 120 receives the data transmitted by the data processing device 110 and implements the display under the control of the data processing device 110.
  • the image capture device 130 then captures the user's image information and sends it to the data processing device 110.
  • the data processing device 110 includes a signal receiving unit 111, a signal processing unit 112, and a cursor control unit 113.
  • the signal receiving unit 111 receives the signal from the image capturing device 130 and sends it to the signal processing unit 112; the signal processing unit 112 performs a ST determination on the target area where the user desires the cursor to arrive, and sends a message to the cursor control unit 113;
  • the control unit 113 transmits a cursor control signal to the display device 120 according to the message to adjust the cursor to the corresponding area.
  • the data processing device 110 further includes a window activation unit 114.
  • the window activation unit 114 determines whether the target area has an application window, and if so, issues a window activation signal to the display device 120 to activate the application window; if not, Then the system is running normally.
  • the number of the display devices 120 may be one or more, and the data processing device 110 controls the cursor to be displayed in the target area of the display device 120.
  • the number of image capture devices 130 may be one or more, each of which is controlled by the data processing device 110 to communicate with it.
  • the signal receiving unit 111, the signal processing unit 112, the cursor control unit 113, and the window activation unit 114 may be implemented by hardware, or may be operated by the data processing device 110 by using software. To achieve the corresponding function. And the units may be discrete or may be integrated with each other by two or more.
  • the image capture device 130 may be a standalone device, or may be integrated into the display device 120 or integrated into the data processing device 110.
  • the signal processing unit 112 can also be integrated in the image acquisition device 130. At this time, the signal receiving unit 111 receives the data processed by the signal processing unit 112.
  • the data processing device refers to an electronic device having data processing capability, including but not limited to a desktop computer, a notebook computer, etc.
  • the display device refers to an electronic device having display capability, including It is not limited to a cold cathode ray tube display, a liquid crystal display, a plasma display, a field effect transistor, etc.
  • the image capturing device refers to an electronic device having image capturing capability, including but not limited to a digital video camera, a camera, a CCD, and the like.
  • the image information of the user may be the face orientation of the user, and the signal processing unit determines the target area of the cursor based on the face orientation of the user.
  • the image information may also be facial features such as a user's perspective.
  • FIG. 3 is a flow chart of the control method of the present invention.
  • Step S110 recording the front and side images of the user.
  • the main role of this step is to determine the basis for subsequent steps of user image information comparison and analysis.
  • Step S120 Collect a current facial feature of the user.
  • Step S130 comparing and analyzing the current facial features.
  • step S140 it is determined whether the range of the change is reached. If yes, the process goes to step S150; if not, the process returns to step S120.
  • the judging step is to judge the difference between the current facial feature and the image information recorded in step S110. When the user's head steering changes, such as a large amplitude deflection, it is confirmed that the cursor position needs to be adjusted.
  • Step S150 changing the cursor position. Adjust the display position of the cursor to the target area corresponding to the user's current facial features.
  • step S160 it is determined whether the process is ended. If not, the process returns to step S120; if so, the process ends.
  • the present invention can also automatically activate the application window of the target area, which is not described here.
  • the current wide-screen computer can achieve the purpose of fast cursor switching by the technical solution of the present invention.
  • the system of this embodiment includes a computer host 910, a display 920 connected to the host computer, and a camera 930 connected to the host computer; the host computer 910 has a control software system (not shown).
  • the camera 930 is disposed above the display 920 to collect facial features of the user in real time.
  • the control software system runs in the host computer 910.
  • the main functions are image acquisition, image pattern recognition, controlling the position of the cursor on the display 920, and activating the corresponding application window.
  • the camera 930 first records the facial features of the user's head-up display color.
  • the image captured by the camera 930 is processed by the control software system to determine the target area that the user needs to reach, and the cursor is adjusted to corresponding The area of the application, if there is an application window in the area, activates the application window.
  • the cursor is displaced. During the implementation process, the movement of the cursor is delayed, so that the response is not too sensitive, resulting in unnecessary trouble.
  • the control software system includes an underlying driver portion 810, an image acquisition and processing portion 820, and a cursor control portion 830.
  • the bottom driving part 810 comprises two parts, a camera device driving module and a display driving module, and mainly provides a corresponding call connection, and provides an image acquisition and processing part of the upper layer, and realizes control of the camera and the display.
  • control software system When the control software system starts working, you first need to collect the front image of the user and check the number of displays (for a display, you can select the screen partition, as there are 4 or more display devices), as shown in Figure 6. Shown.
  • the control software system divides the screen into 4 areas by default.
  • the cursor When the cursor is switched from area 1 to area 4, the moving distance also has a problem that the moving distance is too long.
  • the user's facial features collected by the camera are processed by the control software system, and the position of the user's cursor is switched by the processed signal, which greatly improves the work efficiency.
  • the resolution and pixel value of the display screen are the main reference factors for dividing the screen into four regions or more. According to these reference factors, we can divide the screen into a large number of areas, but considering the accuracy problem, this embodiment is divided into four.
  • control software system records the image when the user faces the respective display areas, and processes them accordingly as a reference.
  • the image of the user's face is collected in real time, and compared with the recorded reference. After the comparison result reaches the range of switching the cursor, after the set cursor control strategy is processed, if it is necessary to switch the position of the cursor, The position of the cursor is changed by the cursor control portion 830, and if the target area has the presence of the application window, it is activated.
  • the cursor control strategy mainly implements the processing of some actual problems encountered during the user's processing, such as the user's inadvertent head movement, rapid rotation, and the like. In image processing, the characteristics between the user's eyes and the nose are mainly used as a reference.
  • the camera is in front of the user, and there is only one camera.
  • the number of cameras connected to the host computer can be multiple and can be controlled by the control software system.
  • control software system works basically the same, but it will be numbered in the software initialization phase. If the angle of deflection of the camera is large, the characteristics of the user's eyes, ears and nose can be used as a reference during image processing.
  • the workstation needs to monitor more tasks. During the use, multiple displays are used. Each display corresponds to different tasks. In this embodiment, six screens are taken as an example, and are evenly arranged around the camera.
  • the moving distance will reach about 1 meter, and such long distance movement is a very troublesome way; if the technique of processing the user's facial features is adopted, when the user's face is facing from the screen 1 changes to screen 6, only takes a short time, plus a predetermined delay, can switch the cursor to the destination in less than 0.5 seconds, and activate the application window of the target area.
  • the present invention provides a computer-aided system that allows a user to quickly switch between a plurality of display devices and activate an application window in a target area; or to quickly change a cursor position on a display device. Set and activate the application window in the target zone.
  • the technical idea is that most of the laptops currently on the market are equipped with built-in cameras, many desktop computers, and cameras are installed because of various user needs such as video chat, video conferencing, etc. When the user switches the screen, it is inevitably directed toward the screen, which in turn causes a change in the facial features of the user photographed by the camera. The image taken by the camera combined with the current powerful image processing technology can make a more accurate analysis and comparison of the facial features of the person.
  • the destination of the user's cursor needs to be determined, and the cursor is quickly switched to the destination. Ground.
  • the user does not need to use the peripheral control cursor such as a mouse or a keyboard to easily move the cursor over a long distance, which not only relieves the fatigue of the user's wrist, but also improves work efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)
  • User Interface Of Digital Computer (AREA)

Description

一种显示设备的光标的控制系统及方法
技术领域
本发明涉及电子信息及数据处理技术，尤其是涉及一种显示设备的光标的控制系 统及方法。 背景技术
随着工业自动化的发展， 网络覆盖面越来越广， 计算机的数据处理能力越来越发 达， 人们需要接受的信息量也越来越大， 使用一个或多个较大的显示设备成为提高工 作效率的手段之一， 但是由此也会导致光标移动距离过长， 从而产生操作不方便和手 腕容易疲劳等问题。
目前的计算机操作系统支持的显示设备数量可以多达 8个，而且该数量还会随着 科技的进展有所增加； 很多工作站配备的显示设备的数量也多达 6个， 甚至更多， 即 使分两行排列， 将光标从左上角移动右下角， 光标的移动距离也将达到 1米左右。 如 此长距离的光标移动， 如果采用传统的鼠标或键盘来进行控制， 必然会增加用户手腕 的疲劳程度， 并且不利于工作效率的提高。对于大屏幕的个人电脑用户及宽屏笔记本 用户， 光标的移动也存在类似的问题。
为解决光标移动距离过长的问题， 部分用户采用热键切换的方法， 但此方法仍然 需要用户多使用一只手腕去控制， 而且可能需要眼睛辅助来看一下键盘上的热键， 这 样会影响操作效率， 因此并不能很好的解决所有的问题。
现有技术中， 存在一些通过设计新型的装置来控制光标的技术方案， 例如申请号 为 02101980、 02141365、 99800299的中国专利所揭示的技术方案， 此类装置类似于 鼠标和游戏手柄， 都需要用户使用手去实现装置的运动， 根据装置的运动来控制光标 的位置， 因此并不能很好地解决前述问题。
现有技术中， 还存在一些采用图像处理技术来控制光标的技术方案， 如申请号为 03224405、 01110375、 99111508的中国专利所揭示的技术方案。
其中， 申请号为 03224405的专利揭示的技术方案是： 通过操作移动点光源发生 器， 使其光信号在光源传播媒介的不同位置上， 由 CCD图像传感器拍摄并转换信号， 经处理器分析处理， 通过方向检测器控制输出设备的显示屏上的光标的移动。 但是， 该现有技术仍然需要用户操作移动点光源发生器， 因此并不能很好地解决前述问题。
申请号为 99111508的专利揭示的技术方案是： 通过影像输入装置输入线形影像 并产生影像信号， 由影像处理电路将该影像信号转换成位置坐标， 再根据位置坐标产 生光标控制信号。但是， 该现有技术不能很好地满足光标在多个屏幕间快速切换或者 在一个显示设备上的不同区域之间快速切换的需求，并且可能导致出现占用过多电脑 资源的情况。
申请号为 01110375 的专利揭示的技术方案是： 通过接收影像信号的光电机构来 撷取影像， 控制器处理所撷取的影像并相应控制屏幕或显示器上的光标移动。 但是， 该现有技术同样不能很好地满足光标在多个屏幕间快速切换或者在一个显示设备上 的不同区域之间快速切换的需求， 并且可能导致出现占用过多电脑资源的情况。
此外， 上述的现有技术均只能控制光标位置的移动， 不能满足在光标移动的同时 激活目标区内应用程序窗口的需求。 发明内容
本发明解决的技术问题在于提供一种显示设备的光标的控制系统及方法，可以实 现光标的快速切换， 从而有效地提高用户的工作效率， 并降低手腕的疲劳感。
在此基础上， 本发明还可以在切换光标位置时激活目标区内的应用程序窗口。 为此，本发明解决技术问题的技术方案是：提供一种显示设备的光标的控制系统， 所述显示设备连接数据处理设备； 包括：
图像采集设备， 用于采集用户的图像信息并发送信号；
信号接收单元， 用于接收来自图像采集设备的信号；
信号处理单元， 用于解析所述信号， 判断是否需要切换光标以及光标到达的目标 区域；
光标控制单元，用于根据信号处理单元的判断结果向所述显示设备发送光标控制 信号， 将光标切换到所述目标区域。
优选地， 该系统还包括窗口激活单元， 用于在所述目标区域中存在应用程序窗口 时向显示设备发出窗口激活信号， 激活该应用程序窗口。
优选地， 所述信号接收单元、 信号处理单元、 光标控制单元和窗口激活单元是分 立的器件或者至少二个单元相互集成。 优选地， 所述信号接收单元、 信号处理单元、 光标控制单元和窗口激活单元设置 在数据处理设备内。
优选地， 所述信号接收单元、 信号处理单元、 光标控制单元和窗口激活单元构成 数据处理设备的控制软件系统。
优选地， 所述信号处理单元设置在图像采集设备内； 所述信号接收单元、 光标控 制单元和窗口激活单元设置在数据处理设备内。
优选地，所述信号处理单元确定的目标区域是多个显示设备之一或显示设备的多 个屏幕区域之一。
优选地， 所述图像采集设备采集的用户的图像信息是用户的面部特征； 所述信号 处理单元根据当前的面部特征和设定的基准面部特征来进行判断。
优选地， 所述图像采集设备设置在显示设备或数据处理设备内。
本发明另提供一种显示设备的光标的控制方法， 所述显示设备连接数据处理设 备； 包括步骤：
1 ) 采集用户的当前图像信息；
2 ) 比较和分析所述当前图像信息， 判断是否达到光标变动的范围， 如果是， 则 进入步骤 3 ); 如果否， 则返回步骤 1 );
3 ) 将光标显示在用户的当前图像信息所对应的目标区域。
优选地， 在所述步骤 1 ) 之前还包括采集用户的基准图像信息的步骤； 所述步骤 2) 中的比较和分析具体是根据当前图像信息和基准图像信息的之间的差别判断用户 的实现是否发生变化。
优选地， 在所述步骤 1 ) 之前还包括检查显示设备数量或设定显示设备的屏幕区 域。
优选地， 该方法还包括激活光标到达的目标区域的应用程序窗口。
优选地， 所述图像信息是指用户的面部特征。
相对于现有技术， 本发明的有益效果是： 由于本发明通过采集和处理用户的图像 信息， 并根据处理信息控制显示设备上的光标， 如快速切换光标， 使得用户可以快速 在多个显示设备间切换光标， 也可以在一个显示设备上快速改变光标位置， 从而节省 用户长距离移动鼠标的时间， 提高工作效率， 还可以减轻用户长距离来回反复操作鼠 标的疲劳， 促进身体健康。
本发明的优选实施方案中， 还实时判断所述目标区域中是否存在应用程序窗口， 如果存在， 则可以激活该应用程序窗口， 进一步改进工作效率， 同时给用户一个很好 的感受， 就是计算机能够领悟使用者的意图， 因为使用者的眼睛等脸部特征的方向代 表了该用户想操作的对象。
在本发明的基础上， 对于目前己经内置摄像头的笔记本电脑， 可以在不增加硬件 成本的情况下， 实现光标在一个或多个屏幕间长距离的切换， 并激活目标区内应用程 序窗口的功能， 适合于大量推广。
总之， 本发明是一种通过对用户面部朝向的处理， 来切换光标位置和激活目的区 内应用程序窗口的计算机辅助工具。 此外， 本发明的实用性强， 用户群较大， 各个年 龄层的人都能使用。 附图说明
图 1是本发明的显示设备的光标位置的控制系统的框图；
图 2是本发明控制系统中数据处理设备的框图；
图 3是本发明控制方法的流程图；
图 4是本发明控制系统第一实施例的组成示意图；
图 5是图 4所示电脑主机中的控制软件系统的组成框图；
图 6是图 4所示显示器分区的示意图；
图 7是本发明第二实施例中屏幕与摄像头的示意图。 具体实施方式
本发明是一种计算机辅助系统， 其主要构思在于： 通过采集用户图像信息， 对用 户图像信息进行处理， 并根据处理信息控制显示设备上的光标位置； 在此基础上， 本 发明还可以激活目标区内应用程序窗口。按照 IPC的分类， 本发明涉及电子信息， 数 字信号处理类。
请参阅图 1， 本发明的显示设备的光标位置的控制系统用于控制显示设备 120显 示的光标， 包括数据处理设备 110、 图像采集设备 130; 该数据处理设备 110连接并 控制该显示设备 120和图像采集设备 130。
显示设备 120接收数据处理设备 110发送的数据，并在数据处理设备 110的控制 下实现显示。
图像采集设备 130则捕捉用户的图像信息， 将其发送至数据处理设备 110。 一并参阅图 2, 该数据处理设备 110包括信号接收单元 111、 信号处理单元 112、 光标控制单元 113。 信号接收单元 111接收来自图像采集设备 130的信号， 并发送至 信号处理单元 112; 信号处理单元 112对所述信号进行处 ST 确定用户希望光标到达 的目标区域， 向光标控制单元 113发送消息； 光标控制单元 113根据所述消息向显示 设备 120发送光标控制信号， 将光标调整到相应的区域。
在此基础上， 为自动激活目标区域的应用程序窗口， 所述数据处理设备 110进一 步包括窗口激活单元 114。 在信号处理单元 112确定光柝到达的目标区域后， 窗口激 活单元 114判断该目标区域是否存在应用程序窗口， 如果是， 则向显示设备 120发出 窗口激活信号， 激活该应用程序窗口； 如果否， 则系统正常运行。
需要说明的是， 该显示设备 120的数量可以是一个或多个， 数据处理设备 110控 制光标在所述显示设备 120的目标区域进行显示。该图像采集设备 130的数量可以是 一个或多个， 均接受数据处理设备 110的控制， 与其进行通讯。
需要进一步说明的是， 所述信号接收单元 111、 信号处理单元 112、 光标控制单 元 113和窗口激活单元 114可以采用硬件的方式实现， 也可以采用软件的方式， 通过 数据处理设备 110 运行相应的软件来实现对应的功能。 并且所述各单元可以是分立 的， 也可以两个或多个相互集成在一起。
可以理解的是， 该图像采集设备 130可以是独立的器件， 也可以集成在显示设备 120中或者集成在数据处理设备 110中。
可以理解的是， 也可以将信号处理单元 112集成在图像采集设备 130中， 此时， 信号接收单元 111接收的将是经过信号处理单元 112处理后的数据。
本发明的实施方式中， 所述数据处理设备是指具有数据处理能力的电子设备， 包 括但不限于台式机算机、笔记本电脑等；所述显示设备是指具有显示能力的电子设备， 包括但不限于冷阴极射线管显示器、 液晶显示器、 等离子显示器、 场效应管等； 所述 图像采集设备是指具有图像拍摄能力的电子设备，包括但不限于数码摄像机、摄像头、 CCD等。
所述用户的图像信息可以是用户的面部朝向，信号处理单元根据用户的面部朝向 来确定光标的目标区域。 当然， 所述图像信息还可以是用户的视角等各面部特征等。
请参阅图 3， 是本发明控制方法的流程图。
步骤 S110， 记录用户的正面和侧面图像。 该步骤的主要作用在于确定后续步骤 进行用户图像信息比较和分析的基准。 步骤 S120， 采集用户当前面部特征。
步骤 S130, 比较和分析当前的面部特征。
步骤 S140, 判断是否达到变动的范围， 如果是， 则进入步骤 S150; 如果否， 则 返回步骤 S120。所述判断步骤就是判断当前的面部特征与步骤 S110所记录的图像信 息之间的差别， 当用户头部转向发生变化， 如较大幅度的偏转时， 则确认光标位置需 要调整。
步骤 S150, 改变光标位置。 将光标的显示位置调整到用户的当前面部特征所对 应的目标区域。
步骤 S160, 判断是否结束流程， 如果否， 则返回步骤 S120; 如果是， 则结束流 程。
此外， 本发明还可以自动激活目标区域的应用程序窗口， 此不赘述。
为便于理解本发明， 下面结合实施例进行介绍。
请参阅图 4， 目前的宽屏电脑可以通过本发明的技术方案实现光标快速切换的目 的。
本实施例的系统包括电脑主机 910、 电脑主机连接的显示器 920， 电脑主机连接 的摄像头 930; 该电脑主机 910 中具有控制软件系统 （图未示）。 其中， 摄像头 930 设置在显示器 920的上方， 可以实时采集用户的面部特征。
控制软件系统运行在电脑主机 910中， 主要功能为图像采集、 图像模式识别、 控 制光标在显示器 920上的位置及激活相应的应用程序窗口。
摄像头 930首先记录用户平视显示色别的面部特征，当用户头部做较大幅度的偏 转时， 摄像头 930采集的图像经过控制软件系统的处理， 确定用户需要达到的目标区 域， 将光标调整到相应的区域， 如该区域内有应用程序窗口存在， 则激活此应用程序 窗口。为防止用户头部随意转动一下就导致光标移位的问题， 实现过程中对光标的移 动进行一定延迟， 使其反应不会过于灵敏， 而导致一些不必要的麻烦。
一并参阅图 5， 是控制软件系统的组成框图。 控制软件系统包括底层驱动部分 810、 图像采集和处理部分 820以及光标控制部分 830。 底层驱动部分 810包含摄像 头设备驱动模块和显示器驱动模块两部分， 主要提供相应的调用接， 给上层的图像采 集和处理部分， 并实现对摄像头和显示器的控制。
当控制软件系统开始工作时， 首先需要采集用户的正面的图像， 检査显示器的个 数 （对于一个显示器， 可以选择屏幕分区， 当作有 4个或更多的显示设备）， 如图 6 所示。 控制软件系统默认将屏幕分成 4个区域， 当光标从区域 1切换到区域 4， 移动 距离也存在移动距离过长的问题。如通过控制软件系统处理摄像头采集的用户面部特 征， 并通过处理的信号切换用户光标所在的位置， 会极大的提高工作效率。
需要说明的是，显示器屏幕的解析度和象素值作为将屏幕分成 4个区域或是更多 的区域的主要参考因素。根据这些参考因素我们可以将屏幕分成非常多的区域， 但考 虑到精度问题， 本实施例分成 4个比较合适。
随后，控制软件系统记录用户正面朝向各个显示区域时的图像，经过相应的处理， 以作为基准。
控制软件系统工作过程中，会实时采集用户面部的图像，和记录的基准进行比较， 比较的结果达到切换光标的范围后， 经过设定的光标控制策略处理， 如果的确需要切 换光标的位置， 则通过光标控制部分 830改变光标的位置， 此时如果目标区有应用程 序窗口的存在， 则激活之。光标控制策略主要实现对用户处理过程中碰到的一些实际 的问题的处理， 如用户不经意的头部运动、 快速的转动等等。 图像处理中， 主要通过 用户的双眼和鼻子三点之间的特征作为参考。
需要说明的是， 以上所述是默认方式下， 摄像头处在用户的正前方， 并只有一个 摄像头的情况。 实际情况中， 电脑主机连接的摄像头的数量可以多个， 都可被控制软 件系统控制。
当用户使用的摄像头不在用户的正前方或是多个摄像头，控制软件系统的工作过 程基本一样， 只是在软件初始化阶段会对其进行一定的编号。如果摄像头的偏转角度 较大， 图像处理过程中可以采用用户的眼睛， 耳朵和鼻子之间的特征作为参考。
请参阅图 7， 工作站实行监控时可以通过本发明的技术方案实现光标快速切换的 目的。
工作站需要监控的任务比较多， 在使用过程中会使用多个显示屏， 每个显示屏对 应不同任务， 本实施例以 6个屏幕为例， 较为均匀地排列在摄像头的周围。
如果用户通过移动鼠标将光标从屏幕 1移动到屏幕 6,移动距离会达到 1米左右， 如此长距离的移动是非常麻烦的方式； 如果采用通过处理用户面部特征的技术， 当用 户面部朝向从屏幕 1变到屏幕 6， 只需要较短的时间， 加上预定的延迟， 可以在不到 0.5秒的时间内将光标切换到目的地， 并激活目标区的应用程序窗口。
总之， 本发明在于提供一种计算机辅助系统， 让用户可以快速在多个显示设备间 切换光标并激活目标区内的应用程序窗口；也可用在一个显示设备上快速改变光标位 置， 并激活目标区内的应用程序窗口。 其技术构思在于： 目前上市的大部分笔记本电 脑都已经配备了内置摄像头， 很多台式电脑， 因为用户的各种需要如视频聊天、 视频 会议等， 也安装了摄像头。 而用户在切换屏幕时， 必然朝向该屏幕， 随之引起摄像头 拍摄的用户的面部特征的变化。摄像头拍摄的图像结合目前强大的图像处理技术， 可 以对人的面部特征做较为精确的分析和比较， 通过这些分析和处理的信息， 判断用户 光标需要到达的目的地， 并将光标快速切换到目的地。 采用此方式， 用户不需要使用 乎去操作鼠标或键盘等外设控制光标， 就可以轻易达到将光标进行长距离移动的目 的， 不但可在缓解用户手腕的疲劳， 并提高工作效率。
以上所述仅是本发明的优选实施方式， 应当指出， 对于本技术领域的普通技术人 员来说， 在不脱离本发明原理的前提下， 还可以作出若千改进和润饰， 这些改进和润 饰也应视为本发明的保护范围。

Claims

权 利 要 求
1 . 一种显示设备的光标的控制系统， 所述显示设备连接数据处理设备； 其特征 在于， 包括：
图像采集设备， 用于采集用户的图像信息并发送信号；
信号接收单元， 用于接收来自图像采集设备的信号；
信号处理单元， 用于解析所述信号， 判断是否需要切换光标以及光标到达的目标 区域；
光标控制单元，用于根据信号处理单元的判断结果向所述显示设备发送光标控制 信号， 将光标切换到所述目标区域。
2. 根据权利要求 1所述的显示设备的光标的控制系统， 其特征在于： 还包括窗 口激活单元，用于在所述目标区域中存在应用程序窗口时向显示设备发出窗口激活信 号， 激活该应用程序窗口。
3. 根据权利要求 2所述的显示设备的光标的控制系统， 其特征在于： 所述信号 接收单元、信号处理单元、光标控制单元和窗口激活单元是分立的器件或者至少二个 单元相互集成。
4. 根据权利要求 2所述的显示设备的光标的控制系统， 其特征在于： 所述信号 接收单元、 信号处理单元、 光标控制单元和窗口激活单元设置在数据处理设备内。
5. 根据权利要求 4所述的显示设备的光标的控制系统， 其特征在于： 所述信号 接收单元、信号处理单元、光标控制单元和窗口激活单元构成数据处理设备的控制软 件系统。
6. 根据权利要求 1所述的显示设备的光标的控制系统， 其特征在于： 所述信号 处理单元设置在图像采集设备内； 所述信号接收单元、光标控制单元和窗口激活单元 设置在数据处理设备内。
7. 根据权利要求 1所述的显示设备的光标的控制系统， 其特征在于： 所述信号 处理单元确定的目标区域是多个显示设备之一或显示设备的多个屏幕区域之一。
8. 根据权利要求 1至 7任一项所述的显示设备的光标的控制系统， 其特征在于： 所述图像采集设备采集的用户的图像信息是用户的面部特征；所述信号处理单元根据 当前的面部特征和设定的基准面部特征来进行判断。
9. 根据权利要求 8所述的显示设备的光标的控制系统， 其特征在于： 所述图像 采集设备设置在显示设备或数据处理设备内。
10. 一种显示设备的光标的控制方法， 所述显示设备连接数据处理设备； 其特征 在于， 包括步骤-
1 ) 采集用户的当前图像信息；
2) 比较和分析所述当前图像信息， 判断是否达到光标变动的范围， 如果是， 则 进入步骤 3 ); 如果否， 则返回步骤 1 );
3 ) 将光标显示在用户的幸前图像信息所对应的目标区域。
11 . 根据权利要求 9所述的显示设备的光标的控制方法， 其特征在于： 在所述步 骤 1 ) 之前还包括采集用户的基准图像信息的步骤； 所述步骤 2) 中的比较和分析具 体是根据当前图像信息和基准图像信息的之间的差别判断用户的实现是否发生变化。
12. 根据权利要求 9所述的显示设备的光标的控制方法， 其特征在于： 在所述步 骤 1 ) 之前还包括检查显示设备数量或设定显示设备的屏幕区域。
13. 根据权利要求 9至 11所述的显示设备的光标的控制方法， 其特征在于： 还 包括激活光标到达的目标区域的应用程序窗口。
14. 根据权利要求 9至 11任一项所述的显示设备的光标的控制方法， 其特征在. 于： 所述图像信息是指用户的面部特征。
PCT/CN2005/001020 2004-12-22 2005-07-11 Procede et systeme de commande du curseur d'un afficheur WO2006066474A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
DE112005003221T DE112005003221B4 (de) 2004-12-22 2005-07-11 Steuersystem und -verfahren für einen Cursor in einer Mehrzahl von Anzeigevorrichtungen
JP2007545815A JP2008523517A (ja) 2004-12-22 2005-07-11 表示装置のカーソルの制御システム及び方法
US11/722,127 US20090213071A1 (en) 2004-12-22 2005-07-11 Control system and method for a cursor in display device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CNB2004101024175A CN100399247C (zh) 2004-12-22 2004-12-22 一种显示设备的光标的控制系统及方法
CN200410102417.5 2004-12-22

Publications (1)

Publication Number Publication Date
WO2006066474A1 true WO2006066474A1 (fr) 2006-06-29

Family

ID=36601365

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2005/001020 WO2006066474A1 (fr) 2004-12-22 2005-07-11 Procede et systeme de commande du curseur d'un afficheur

Country Status (5)

Country Link
US (1) US20090213071A1 (zh)
JP (1) JP2008523517A (zh)
CN (1) CN100399247C (zh)
DE (1) DE112005003221B4 (zh)
WO (1) WO2006066474A1 (zh)

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9235262B2 (en) 2009-05-08 2016-01-12 Kopin Corporation Remote control of host application using motion and voice commands
CN101533314B (zh) * 2008-03-11 2011-11-02 联想(北京)有限公司 通过光标定位终端的方法、系统及终端控制器
CN102016975A (zh) 2008-03-28 2011-04-13 寇平公司 适合用作移动式互联网装置的具有高分辨率显示器的手持式无线显示装置
CN102339124A (zh) * 2010-07-22 2012-02-01 宏碁股份有限公司 控制光标的方法
US10013976B2 (en) 2010-09-20 2018-07-03 Kopin Corporation Context sensitive overlays in voice controlled headset computer displays
US9122307B2 (en) * 2010-09-20 2015-09-01 Kopin Corporation Advanced remote control of host application using motion and voice commands
US9316827B2 (en) 2010-09-20 2016-04-19 Kopin Corporation LifeBoard—series of home pages for head mounted displays (HMD) that respond to head tracking
US9377862B2 (en) 2010-09-20 2016-06-28 Kopin Corporation Searchlight navigation using headtracker to reveal hidden or extra document data
WO2012154938A1 (en) 2011-05-10 2012-11-15 Kopin Corporation Headset computer that uses motion and voice commands to control information display and remote devices
US8879801B2 (en) 2011-10-03 2014-11-04 Qualcomm Incorporated Image-based head position tracking method and system
WO2013101438A1 (en) 2011-12-29 2013-07-04 Kopin Corporation Wireless hands-free computing head mounted video eyewear for local/remote diagnosis and repair
CN103324816B (zh) * 2012-03-19 2017-03-01 联想(北京)有限公司 一种信息处理方法及电子设备
WO2013163293A1 (en) 2012-04-25 2013-10-31 Kopin Corporation Instant translation system
US8929954B2 (en) 2012-04-25 2015-01-06 Kopin Corporation Headset computer (HSC) as auxiliary display with ASR and HT input
KR20130130453A (ko) * 2012-05-22 2013-12-02 엘지전자 주식회사 영상표시장치 및 그 동작 방법
US9301085B2 (en) 2013-02-20 2016-03-29 Kopin Corporation Computer headset with detachable 4G radio
CN103605621B (zh) * 2013-11-26 2016-09-21 三星电子（中国）研发中心 单个输入设备控制两个以上终端设备的方法和装置
CN105094504B (zh) * 2014-05-07 2019-08-13 鸿合科技股份有限公司 仿真显示控制方法、装置及遥控装置
DE102014017355A1 (de) * 2014-11-18 2016-05-19 DIPAX e. K. Dirk Pachael Verfahren und Anordnung zur intuitiven Auswahl eines Programms, Programmteils oder eines Computers, insbesondere bei Leitständen, wobei jedes Programm, Programmteil oder jeder Computer durch eine eigene Anzeige oder ein Bereich innerhalb einer Anzeige im
CN105929954B (zh) * 2016-04-19 2019-10-18 京东方科技集团股份有限公司 一种控制光标的方法及装置、显示设备
CN106791794A (zh) * 2016-12-30 2017-05-31 重庆卓美华视光电有限公司 一种显示设备、图像处理方法及装置
CN110580104A (zh) * 2018-05-22 2019-12-17 群光电子股份有限公司 光标移动辅助系统及方法
CN110851030A (zh) * 2019-10-18 2020-02-28 苏州佳世达电通有限公司 显示设备及其图像移动方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1060940A (zh) * 1990-07-24 1992-05-06 生物控制系统有限公司 眼睛操纵的控制器
CN1379318A (zh) * 2001-04-09 2002-11-13 华邦电子股份有限公司 撷取影像和操控光标的装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0423027A (ja) * 1990-05-17 1992-01-27 Fujitsu Ltd ウインドウ選択方式
US5825360A (en) * 1995-04-07 1998-10-20 Apple Computer, Inc. Method for arranging windows in a computer workspace
JPH09204287A (ja) * 1996-01-25 1997-08-05 Canon Inc 処理選択方法及び装置
JP3050808B2 (ja) * 1996-06-28 2000-06-12 財団法人大阪科学技術センター 位置指示装置
DE19819961A1 (de) * 1998-05-05 1999-11-11 Dirk Kukulenz Automatische Blickpunktanalyse mit Methoden der Bilderkennung zur Computersteuerung
US6243076B1 (en) * 1998-09-01 2001-06-05 Synthetic Environments, Inc. System and method for controlling host system interface with point-of-interest data
US6127663A (en) * 1998-10-09 2000-10-03 Ericsson Inc. Electronics cabinet cooling system
US20020126090A1 (en) * 2001-01-18 2002-09-12 International Business Machines Corporation Navigating and selecting a portion of a screen by utilizing a state of an object as viewed by a camera
US7333071B2 (en) * 2001-05-11 2008-02-19 Xerox Corporation Methods of using mixed resolution displays
JP3876671B2 (ja) * 2001-09-26 2007-02-07 ティアック株式会社 入力装置およびこれを備える情報処理装置ならびにカーソル表示制御方法
JP2003280805A (ja) * 2002-03-26 2003-10-02 Gen Tec:Kk データ入力装置
CN2609054Y (zh) * 2003-03-18 2004-03-31 赵志强 图像传感器识别驱动电脑指点设备系统
US7391888B2 (en) * 2003-05-30 2008-06-24 Microsoft Corporation Head pose assessment methods and systems
EP3190546A3 (en) * 2003-06-12 2017-10-04 Honda Motor Co., Ltd. Target orientation estimation using depth sensing
JP4494002B2 (ja) * 2003-12-19 2010-06-30 富士通コンポーネント株式会社 切替器および切替器の切替方法
US7545362B2 (en) * 2004-02-26 2009-06-09 Microsoft Corporation Multi-modal navigation in a graphical user interface computing system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1060940A (zh) * 1990-07-24 1992-05-06 生物控制系统有限公司 眼睛操纵的控制器
CN1379318A (zh) * 2001-04-09 2002-11-13 华邦电子股份有限公司 撷取影像和操控光标的装置

Also Published As

Publication number Publication date
US20090213071A1 (en) 2009-08-27
DE112005003221T5 (de) 2007-11-15
JP2008523517A (ja) 2008-07-03
CN1797299A (zh) 2006-07-05
CN100399247C (zh) 2008-07-02
DE112005003221B4 (de) 2011-06-09

Similar Documents

Publication Publication Date Title
WO2006066474A1 (fr) Procede et systeme de commande du curseur d'un afficheur
JP4689684B2 (ja) 動作に基づくトラッキング
US9195345B2 (en) Position aware gestures with visual feedback as input method
US9547379B2 (en) Method, air mouse, and set top box for controlling air mouse
US20070216642A1 (en) System For 3D Rendering Applications Using Hands
TW200945174A (en) Vision based pointing device emulation
JP2002526867A (ja) データ入力方法
JP2004521551A (ja) 遠隔カメラ制御装置
TW200949609A (en) Method and system for controlling multiple computers
CN103677569A (zh) 一种电子设备的用户界面的处理方法及电子设备
US20050283544A1 (en) Method and system for reducing latency in transferring captured image data
WO2023030519A1 (zh) 投屏处理方法及相关设备
JP6982277B2 (ja) 情報端末装置、情報処理システムおよび表示制御プログラム
TW200949618A (en) Input device and the control method thereof
WO2015062248A1 (zh) 显示装置及其控制方法、和手势识别方法
TWI437464B (zh) 頭戴式個人電腦及其互動式系統
US11249315B2 (en) Augmented reality system and method of displaying virtual screen using augmented reality glasses
TW201939921A (zh) 遠端管理系統及方法
CN104423568A (zh) 显示画面的控制系统、输入装置及控制方法
TWI499938B (zh) 觸控系統
JPH10336505A (ja) 画像表示装置及び画像表示方法
TWI726252B (zh) 多螢幕操作方法與使用此方法的電子系統
WO2006058488A1 (fr) Dispositif et procede de synthese d'affichage
WO2018205745A1 (zh) 用于便携设备的输入界面显示系统及方法
CN101071350A (zh) 一种通过识别动态轨迹操作光标、视窗的装置

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2007545815

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 1120050032213

Country of ref document: DE

RET De translation (de og part 6b)

Ref document number: 112005003221

Country of ref document: DE

Date of ref document: 20071115

Kind code of ref document: P

122 Ep: pct application non-entry in european phase

Ref document number: 05766566

Country of ref document: EP

Kind code of ref document: A1

WWW Wipo information: withdrawn in national office

Ref document number: 5766566

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8607

WWE Wipo information: entry into national phase

Ref document number: 11722127

Country of ref document: US