CN102426509A - Method, device and system for displaying hand input - Google Patents
Method, device and system for displaying hand input Download PDFInfo
- Publication number
- CN102426509A CN102426509A CN2011103518803A CN201110351880A CN102426509A CN 102426509 A CN102426509 A CN 102426509A CN 2011103518803 A CN2011103518803 A CN 2011103518803A CN 201110351880 A CN201110351880 A CN 201110351880A CN 102426509 A CN102426509 A CN 102426509A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- dimensional
- user
- axis
- user input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 claims description 40
- 239000013598 vector Substances 0.000 claims description 25
- 238000003909 pattern recognition Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 7
- 238000013507 mapping Methods 0.000 description 6
- 238000013519 translation Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000005484 gravity Effects 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a method, a device and a system for displaying hand input. The method comprises the following steps of: receiving three-dimensional coordinates of a user input trace; determining a fitting plane of the user input trace; selecting a straight line, parallel with a horizontal plane in a three-dimensional world coordinate system which uses a user as a reference system, from the fitting plane as an X axis in a two-dimensional coordinate system; selecting a straight line, vertical to the straight line which serves as the X axis in the two-dimensional coordinate system, from the fitting plane as a Y axis in the two-dimensional coordinate system; and converting the three-dimensional coordinates of the user input trace to the two-dimensional coordinate system, and displaying and outputting the user input trace. By adoption of the method, according to the three-dimensional coordinates of the user input trace, an expected output trace of the user can be effectively output, deformation of input contents can be avoided, and an experience effect of the user is better.
Description
Technical Field
The invention belongs to the technical field of computer vision, image processing and pattern recognition.
Background
Most of common computer input devices such as a keyboard, a mouse, a writing pad, a touch screen and the like become standard peripherals or common peripherals of a computer, and people cannot simultaneously receive respective defects of the devices when applying the computer at present. Summarizing the characteristics of these conventional computer input devices, they have some disadvantages as follows.
Keyboard: it is often used in conjunction with a mouse, and it is difficult to move the cursor to any position on the screen quickly through the keyboard, and also difficult to perform continuous operations such as drawing with the keyboard. The keyboard is from western, which is beneficial to inputting alphabetic writing letters and numerical symbols, and can realize some compound operations by combining with a limited number of function keys. There is fundamentally a great obstacle to the input of pictographic characters such as Chinese characters. In the existing input methods for inputting Chinese characters through a keyboard, on one hand, the input efficiency is influenced by multi-key operation, and on the other hand, part of input rules need to be remembered. For the input of pictographic characters such as Chinese characters, the keyboard cannot be intuitive and convenient. The keyboard is also often kept clean and maintained.
Mouse: the movement of the mouse is a linear uninterrupted movement. If the mouse on the screen is moved to a target from one point, the hands and eyes of a user are required to be always in contact with the mouse (sometimes, the left and right keys or the scroll wheel simultaneously participate), so that more time is occupied for the user, the attention of the user is required to be kept, and the freedom of movement and the use mode of the hands are limited. The mouse is also often kept clean and maintained.
Touch panel: the existing touch pad functions to replace a traditional rolling type mouse, a point-shaped or column-shaped mouse, and has the defects of low moving efficiency, limitation of free hand movement of a user, more attention consumption of the user and the like of the traditional mouse. In addition, it only collects the little information of user's hand, can not decipher the multiple action of user's hand, and blocks operator's sight. Only single-point identification can be realized, and multi-point identification cannot be realized.
Writing on a handwriting board: the keyboard is not rapid and intuitive when inputting specified symbols, letters and numbers. The handwriting board can only accept single-point information, and a single point or single line on the plane is input at one time, so that the information touching multiple discontinuous positions on the handwriting board at the same time is not reflected. The amount of information received or reflected by the user is small, a large amount of information on the user's hand is not utilized, and the level of human-computer communication is limited. Meanwhile, the problem of character input recognition rate is also solved.
A touch screen: when a user points on the touch screen with a hand, the hand can block the touch screen to block the sight of the operator. Pointing on a touch screen, the user must first lift the upper arm, and lifting the arm for a long time can also cause fatigue. The device can be used as an auxiliary service device only in a few public places, and can only receive single-point information.
Some computer input device manufacturers have noticed such problems and have produced products with combinations of functions of the above different types of input devices, such as: keyboards with writing pads, etc., but it is still difficult for such products to fundamentally change their respective defects.
The prior art is often limited by time and space, and the fingers of a user are uncomfortable due to long-time holding of the equipment and touch; the written track information is often inaccurate by using a mouse and the like.
Disclosure of Invention
The invention aims to solve the technical problem of providing a display method, a device and a system for handwriting input, wherein a user only needs to draw a track of input content in the air during input, the input is not limited by media and space, the content expected to be input by the user can be well displayed, and the user experience is high.
In order to solve the above technical problem, the present invention provides a display method of handwriting input, comprising:
receiving a three-dimensional coordinate of a user input track;
determining a fitting plane of the user input track;
selecting a straight line parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system from the fitting plane as an X axis in a two-dimensional coordinate system;
selecting a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system from the fitting plane as a Y axis in the two-dimensional coordinate system;
and converting the three-dimensional coordinates of the user input track into the two-dimensional coordinate system, and displaying and outputting the three-dimensional coordinates.
Further, the method can also have the following characteristics:
and determining a three-dimensional world coordinate system taking the user as a reference system by adopting a mode identification mode.
Further, the method can also have the following characteristics:
and determining a three-dimensional world coordinate system taking the user as a reference system by identifying the pre-marked reference points.
Further, the method can also have the following characteristics:
calculating the product of the normal vector of the fitting plane and the normal vector of a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system;
and selecting a straight line with a direction vector as the product of the normal vectors from the fitting plane to obtain a straight line on the fitting plane, wherein the straight line is parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system.
Further, the method can also have the following characteristics:
and when the fitting plane is coincident with a horizontal plane in the three-dimensional world coordinate system taking the user as the reference system, selecting a straight line which is consistent with the horizontal direction in the three-dimensional world coordinate system taking the user as the reference system from the fitting plane as an X axis in the two-dimensional coordinate system.
Further, the method can also have the following characteristics:
and calculating an intersection line of a plane taking the straight line as the X axis in the two-dimensional coordinate system as a normal line and the fitting plane to obtain a straight line perpendicular to the straight line as the X axis in the two-dimensional coordinate system on the fitting plane.
Further, the method can also have the following characteristics:
and selecting an X axis and a Y axis in a two-dimensional coordinate system, and enabling the X coordinate and the Y coordinate of the user input track converted into the two-dimensional coordinate system to be greater than or equal to 0.
Further, the method can also have the following characteristics:
converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system with the user as a reference system;
and selecting an X axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum Y coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that the X coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
Further, the method can also have the following characteristics:
converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system with the user as a reference system;
and selecting a Y axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum X coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that Y coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
Further, the method can also have the following characteristics:
the user input trajectory converted into the two-dimensional coordinate system is also scaled and/or translated prior to displaying the output.
In order to solve the above technical problem, the present invention further provides a display method of handwriting input, including:
measuring and calculating the three-dimensional coordinates of the input track of the user;
the method of any of claims 1-10, wherein the user input trajectory is transformed from a three-dimensional coordinate system to a two-dimensional coordinate system, and the output is displayed.
Further, the method can also have the following characteristics:
and measuring and calculating the three-dimensional coordinates of the input track of the user by adopting a stereoscopic vision identification mode or a radar sensing mode or an infrared sensing mode.
In order to solve the above technical problem, the present invention further provides a display device for handwriting input, including:
the receiving module is used for receiving the three-dimensional coordinates of the track input by the user;
the first processing module is connected with the receiving module and used for determining a fitting plane of the user input track according to the three-dimensional coordinates of the user input track; selecting a straight line parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system from the fitting plane as an X axis in a two-dimensional coordinate system; selecting a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system from the fitting plane as a Y axis in the two-dimensional coordinate system, so as to establish the two-dimensional coordinate system;
the second processing module is connected with both the receiving module and the first processing module and is used for converting the three-dimensional coordinates of the user input track into the two-dimensional coordinate system;
and the output module is connected with the second processing module and is used for displaying and outputting according to the two-dimensional coordinates of the user input track.
Further, the device can also have the following characteristics:
the first processing module is used for determining a three-dimensional world coordinate system which takes a user as a reference system in a mode of pattern recognition.
Further, the device can also have the following characteristics:
the first processing module is used for determining a three-dimensional world coordinate system taking a user as a reference system by identifying a pre-marked reference point.
Further, the device can also have the following characteristics:
the first processing module is used for selecting a straight line with a direction vector as the product of the normal vectors from the fitting plane by calculating the product of the normal vectors of the fitting plane and the normal vectors of a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system, so as to obtain the straight line on the fitting plane, which is parallel to the horizontal plane in the three-dimensional world coordinate system taking the user as the reference system.
Further, the device can also have the following characteristics:
and the first processing module selects a straight line which is consistent with the horizontal direction in the three-dimensional world coordinate system taking the user as the reference system from the fitting plane of the user input track as the X axis in the two-dimensional coordinate system when judging that the fitting plane of the user input track is coincident with the horizontal plane in the three-dimensional world coordinate system taking the user as the reference system.
Further, the device can also have the following characteristics:
the first processing module is configured to obtain a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system on the fitting plane by calculating an intersection line between the plane using the straight line serving as the X axis in the two-dimensional coordinate system as a normal line and the fitting plane.
Further, the device can also have the following characteristics:
the first processing module is used for selecting an X axis and a Y axis in a two-dimensional coordinate system, and enabling the X coordinate and the Y coordinate of the user input track converted into the two-dimensional coordinate system to be larger than or equal to 0.
Further, the device can also have the following characteristics:
the first processing module is used for converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system taking a user as a reference system, and selecting a straight line of which the X axis in the two-dimensional coordinate system is the minimum point of the Y coordinate in the three-dimensional world coordinate system taking the user as the reference system through the user input track, so that the X coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
Further, the device can also have the following characteristics:
the first processing module is used for converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system with a user as a reference system; and selecting a Y axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum X coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that Y coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
Further, the device can also have the following characteristics:
and the output module is used for zooming and/or translating the two-dimensional coordinates of the track input by the user and then displaying and outputting the two-dimensional coordinates.
In order to solve the above technical problem, the present invention further provides a display system for handwriting input, including:
the measuring and calculating module is used for measuring and calculating the three-dimensional coordinates of the input track of the user;
a display device for handwriting input according to any of claims 13-22 and arranged to convert a user input trajectory from a three-dimensional coordinate system into said two-dimensional coordinate system for display output thereof.
Further, the system can also have the following characteristics:
the measuring and calculating module is a stereoscopic vision recognition device or a radar sensing device or an infrared sensing device.
The display method, the device and the system for handwriting input provided by the invention have the following advantages:
1. no medium is needed, and the device is not limited by space. The conventional touch screen method must contact a corresponding medium in order for a computer to obtain information of a point, and the range of contact is limited only to the size of the touch screen. The display method of the handwriting input only needs to be within the identification range of measurement (sensing or image acquisition), does not need to contact any medium, and has a large range.
2. The scheme for measuring and calculating the space points by adopting the stereoscopic vision provided by the invention is only required to be capable of shooting the target object by the camera, is not limited by the position of the camera, and can ensure that the relation between the space points mapped to the user terminal cannot be distorted even if the space points are not right opposite to the camera. And the cost is lower. Only 2 or more than 2 common cameras are needed to form the stereoscopic vision information.
Drawings
FIG. 1 is a flow chart of a method for identifying three-dimensional coordinates of a user input trajectory in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a parallel binocular visual distance sensor model;
FIG. 3 is a schematic diagram of the distance measurement principle of a parallel binocular vision distance sensor;
FIG. 4(a) is a first schematic diagram illustrating a mapping relationship between a three-dimensional coordinate system and a two-dimensional coordinate system;
FIG. 4(b) is a schematic diagram of two-dimensional coordinates of a user input trace point in the mapping relationship shown in FIG. 4 (a);
FIG. 4(c) is a second schematic diagram illustrating the mapping relationship between the three-dimensional coordinate system and the two-dimensional coordinate system;
FIG. 4(d) is a schematic diagram of two-dimensional coordinates of a user input trace point in the mapping relationship shown in FIG. 4 (c);
FIG. 5 is a flowchart of a display method of handwriting input according to an embodiment of the invention;
FIG. 6(a) is a schematic diagram of a mapping relationship between a three-dimensional coordinate system and a two-dimensional coordinate system according to an embodiment of the present invention;
FIG. 6(b) is a schematic diagram of two-dimensional coordinates of a user input trace point under the mapping relationship shown in FIG. 6 (a);
FIG. 7 is a schematic diagram of the correspondence between the three-dimensional coordinate system of the camera and the three-dimensional coordinate system of the display device;
FIG. 8 is a diagram of a display device for handwriting input according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a display system for handwriting input according to an embodiment of the present invention.
Detailed Description
The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, process, and other changes. The examples merely typify possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in or substituted for those of others. The scope of embodiments of the invention encompasses the full ambit of the claims, as well as all available equivalents of the claims. Embodiments of the invention may be referred to herein, individually or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed.
The invention provides a display method for handwriting input, which displays and outputs an input track of a user by measuring and calculating three-dimensional coordinates of the input track of the user. The user does not need a medium when writing, and the user can display the input content expected by the user only by simply waving a finger to mark the content to be input in the measuring and calculating identification range, so that the user has a good user experience effect.
The three-dimensional coordinate of the user input track refers to a space point coordinate of the user input track in a three-dimensional world coordinate system, the three-dimensional space coordinate system comprises an X axis, a Y axis and a Z axis which are vertical to each other, and the X axis, the Y axis and the Z axis respectively represent a horizontal direction, a vertical direction and a depth of field direction, wherein: the three-axis intersection point is the origin and is marked as the O point; a plane formed by the X axis and the Z axis is marked as an XOZ plane; a plane formed by the X axis and the Y axis is marked as an XOY plane; the plane formed by the Y axis and the Z axis is denoted as a YOZ plane.
In order to further detail the implementation of the present invention, the following specific examples are given as examples.
First embodiment
The embodiment of the invention provides a method for measuring and calculating three-dimensional coordinates of a user input track by adopting a stereoscopic vision identification method, which comprises the following steps of:
step S101: collecting a target image;
step S102: extracting feature points of a target image;
step S103: measuring and calculating the three-dimensional coordinates of the characteristic points in the current image;
step S104: and collecting the frame images to generate track information of the characteristic points.
In order to facilitate the operation of a user, two or more cameras are adopted to collect target images so as to obtain depth of field information, and a track plane waved by the user is not required to be perpendicular to the direction of the cameras so as to correctly identify input contents expected by the user.
After removing interference information and background information from the collected image, extracting characteristic points:
extracting feature points from the image, wherein the feature points of the target are the gravity center of the target in the embodiment; since the position of the center of gravity of the object is constant as long as the shape of the object does not change greatly, the position of the center of gravity of the object at a certain time is selected as one point in the trajectory. The center of gravity is calculated as follows:
wherein:
x, y are coordinates in the image, and f (x, y) is a color value of the image at point (x, y).
The following describes a method for determining three-dimensional coordinates of a user input trajectory by taking an example of acquiring a target image by using two cameras.
As shown in fig. 2, the two focal lengths arefThe cameras are arranged in parallel, and the distance between the optical axes isTThe two rectangles respectively represent imaging planes of the left camera and the right camera,O l andO r the focus of left and right camerasO l Is taken as the origin point of the image,O l O r in a straight line ofXAxis, left camera optic axisZAnd the straight line vertical to both the X axis and the Z axis is the Y axis. For any point in the scenePThe imaging points on the imaging planes of the left camera and the right camera are respectivelyp l Andp r 。
with reference to figure 3 of the drawings,p l andp r imaging coordinates (image coordinates) on the imaging plane are respectivelyx l Andx r parallax is defined asd = x l - x r (as shown in fig. 2).PIs spotted onO l The coordinates in the coordinate system may be calculated as follows:
the (X, Y, Z) coordinates are actual coordinates of the object in the real world with the camera as the origin. The Z value is the actual distance between the camera and the object, here the distance between the lens and the center of gravity of the waving target. Stereoscopic information of the track can be obtained.
The characteristic points of the target are determined in the frame images at different moments, and the track information of the characteristic points is generated by collecting the frame images.
Preferably, when the three-dimensional coordinates of the user input track are measured and calculated by adopting the stereoscopic vision identification method, the target image can be collected by adopting the multi-view camera, the visual angle is wider, and the calculated three-dimensional coordinates of the user input track are more accurate.
In another embodiment, distance sensing may also be used to measure the three-dimensional coordinates of the user input trajectory, for example, radar sensing or infrared sensing.
Second embodiment
For a display device whose display mode is two-dimensional display, it is also necessary to convert the three-dimensional coordinates of the user input trajectory into a two-dimensional coordinate system.
Since in three-dimensional space, the orientation of the XY axis defining the two-dimensional space also directly affects the angular relationship between the spatial points. As shown in fig. 4(a), two-dimensional space points are obtained by selecting AB as the X axis in the three-dimensional space, the X axis direction is from a to B, and the direction perpendicular to the AB axis direction is the Y axis, as shown in fig. 4 (B). Also as shown in fig. 4(c), an arbitrary direction is taken as the X-axis, and the resulting two-dimensional space point is shown as 4 (d). It can be seen that although the positional relationship between three points ABC of the spatial points in the two obtained two-dimensional images is not changed, the angular relationship changes from one spatial point to another spatial point in different coordinate axes due to the change of the coordinate system. Therefore, if the angle relation between the spatial points is not required, the coordinate axes can be selected optionally, otherwise, the selection is carried out according to specific conditions, and the input track expected by the user can be displayed correctly.
To solve the above problem, an embodiment of the present invention provides a display method for handwriting input, as shown in fig. 5, including the steps of:
step S501: receiving a three-dimensional coordinate of a user input track;
the three-dimensional world coordinates using the device for measuring and calculating the user input trajectory as a reference system may be the same as or different from the three-dimensional world coordinates using the user as a reference system. That is, there is no requirement for establishing a three-dimensional coordinate system for measuring and calculating a user input trajectory, and an output trajectory expected by a user can be obtained regardless of the three-dimensional coordinate system of the received user input trajectory calculated in any three-dimensional world coordinate system.
Step S502: determining a fitting plane of the user input track;
step S503: selecting a straight line parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system from a fitting plane of a user input track as an X axis in a two-dimensional coordinate system;
pattern recognition, such as face recognition or body feature recognition, may be employed to determine a three-dimensional world coordinate system with the user as a reference frame. Alternatively, a three-dimensional world coordinate system with the user as a reference system may also be determined by identifying pre-marked reference points.
In a three-dimensional world coordinate system with a user as a reference system, an X axis represents a horizontal direction, a Y axis represents a vertical direction, and a Z axis represents a depth direction. The horizontal plane in the three-dimensional world coordinate system with the user as the reference system is the XOZ plane in the three-dimensional world coordinate system with the user as the reference system. For convenience, the following description will all describe the horizontal plane in the three-dimensional world coordinate system with the user as a reference frame in terms of the "XOZ plane".
Step S504: selecting a straight line vertical to the straight line as the X axis in the two-dimensional coordinate system from a fitting plane of the user input track as a Y axis in the two-dimensional coordinate system;
step S505: and converting the three-dimensional coordinates of the user input track into the two-dimensional coordinate system, and displaying and outputting the three-dimensional coordinates.
When the fitting plane of the user input track coincides with a horizontal plane (XOZ plane) in the three-dimensional world coordinate system using the user as a reference system, a straight line which is consistent with the horizontal direction (namely, the X-axis direction) in the three-dimensional world coordinate system using the user as the reference system can be selected from the fitting plane of the user input track to be used as the X-axis in the two-dimensional coordinate system.
By adopting the method, the embodiment of the invention considers efficient identification, optimally presents the input track of the user and fully considers the three-dimensional space angle of the user, thereby ensuring that the displayed content is the input track expected by the user.
In a specific implementation, according to the three-dimensional coordinates of the trajectory input by the user, the trajectory of the points obtained in the process of waving the hand can be converted into the same plane by adopting the following method:
let the general equation for the plane be:
(4)
the coordinates of the points to be fitted are set as:
if the coordinates of all points satisfy the above formula, the points are considered to be in the same plane, otherwise, the points are not in the same plane. This requires fitting the points to obtain an optimal plane. The following description is given by taking the least square method as an example of fitting a plane, and can be obtained according to equation (4):
for equation (5), let:
(6)
the new plane equation is obtained as:
application pointFitting the above-mentioned plane equation, minimizes equation (7):
(8)
so that S is minimal, it should satisfy:
namely:
solving the linear equation system of the formula (10) to obtain
Three parameters, thus obtaining a plane equation fitted by a least square method. With the plane equation, it is also necessary to project trace points that are not on the plane equation onto the plane. Let the point P to be projected be
The specific projection method can be used for making a linear equation of the passing point P perpendicular to the plane, and the linear equation obtained according to the formula (4) is as follows:
(11)
solving a system of equations:
(12)
i.e. the coordinates of its projected point can be obtained. Thus, the user input tracks on the same fitting plane can be obtained. The points on the user input trajectory are in the same plane, and the coordinates of each point are three-dimensional coordinate points represented by (x, y, z).
Referring to fig. 6(a), the fitting plane is PQR, a straight line parallel to the XOZ plane is made in the plane PQR as the X-axis in the two-dimensional coordinate system, and the specific calculation process is as follows:
first according to the known PQR plane
Equation and XOZ plane equation
The normal vectors of the obtained plane are respectively:and
and due to the direction vector of the straight line to be solved as the X axis in the two-dimensional coordinate systemNormal vectors perpendicular to the PQR and XOZ planes, so the normal of the PQR plane fitting the user input trajectory is calculated
And the XOZ plane in the three-dimensional coordinate system
Of the normal line ofThe product is obtained to obtain the direction vector of the straight line to be solved
：
The equation of a straight line parallel to PR is calculated as:
selecting a point in the PQR plane according to equation (14)
The linear equation to be solved can be obtained by substituting the formula (14).
Preferably, in order to reduce the amount of calculation in the display process, the specific position of the X axis in the two-dimensional coordinate system is selected such that the X coordinates of the user input trajectory converted into the two-dimensional coordinate system are all greater than or equal to 0, so that it is not necessary to process the polarity (positive or negative) of the X coordinate symbol of the user input trajectory in the two-dimensional coordinate system during display.
That is, one point on the PQR plane is selected so that all other points are above the X-axis. The three-dimensional coordinates of the received user input track can be converted into coordinates in a three-dimensional world coordinate system with a user as a reference system; an X-axis in the two-dimensional coordinate system is selected as a straight line passing through a point (e.g., a point B shown in fig. 6 (a)) where the Y coordinate of the user input trajectory is minimum in the three-dimensional world coordinate system using the user as a reference system, so that the X coordinates of the user input trajectory converted into the two-dimensional coordinate system are all greater than or equal to 0.
With the linear equation of the X axis, the linear equation of the Y axis can be obtained according to the relationship that the X axis and the Y axis are mutually perpendicular. Specifically, the method may be implemented by calculating an intersection line of a plane using the straight line as the X axis in the two-dimensional coordinate system as a normal and a fitting plane of the user input trajectory. That is, assuming that a linear equation of the X axis in the three-dimensional coordinates is taken as a normal vector, a plane equation perpendicular to the plane PQR is taken as:
the equation of the line for the Y axis is obtained from the fitted plane and equation (15) as:
preferably, in order to reduce the amount of calculation in the display process, the specific position of the Y axis in the two-dimensional coordinate system is selected such that the Y coordinates of the user input trajectory converted into the two-dimensional coordinate system are all greater than or equal to 0, so that it is not necessary to process the polarity (positive or negative) of the Y coordinate symbol of the user input trajectory in the two-dimensional coordinate system during display.
That is, one point on the PQR plane needs to be selected so that the other points are all to the right of the Y-axis or Y-axis. The specific calculation method can convert the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system taking a user as a reference system; the Y axis in the two-dimensional coordinate system is selected as a straight line passing through a point (point a shown in fig. 6 (a)) where the X coordinate of the user input trajectory is minimum in the three-dimensional world coordinate system using the user as a reference system, so that the Y coordinates of the user input trajectory converted into the two-dimensional coordinate system are all greater than or equal to 0.
The X, Y axes have been previously determined, but it is also necessary to determine the origin of coordinates, i.e. the intersection of the X and Y axes. The intersection point obtained by simultaneous equations of two straight lines is set as a point
. As shown in figure 6(b) of the drawings,
the coordinates of the point in the two-dimensional coordinate system are (0, 0). B is
In two-dimensional coordinates of
Wherein
Comprises the following steps:
point A of the same theory
In two-dimensional coordinates of
WhereinComprises the following steps:
determine
And
after the coordinates of the two-dimensional coordinate axis, any point C in the three-dimensional coordinate
New coordinates
The calculation method comprises the following steps:
the coordinate of any point on the two-dimensional plane can be obtained by solving the equation set
Absolute value of (a). Since it has been mentioned above that the determination of the X-axis and Y-axis ensures that the spatial points are positive
The absolute value of (A) is the coordinate value thereof.
The obtained two-dimensional plane coordinates of the space points are based on the space coordinates, and the tracks of the space points need to be displayed by zooming and translating according to specific conditions.
The specific scaling formula is as follows:
wherein,
is the scaling factor.
The translation formula is:
wherein,
is the amount of translation.
In another embodiment, mutually perpendicular straight lines may also be arbitrarily selected on the PQR plane as XY axes of the two-dimensional space, and when the trajectory is displayed, translation processing needs to be performed, so that the coordinates of the user input trajectory in the two-dimensional space are all values greater than or equal to 0, and rotation processing needs to be performed, the display direction of the user input trajectory is adjusted, and scaling processing needs to be performed according to display requirements.
Third embodiment
For a display device whose display mode is three-dimensional display, when displaying output, since a three-dimensional coordinate system used when measuring and calculating a three-dimensional coordinate of a target may not coincide with a coordinate system of the display device, it is necessary to modify the coordinate system. As shown in fig. 7, the left side is a three-dimensional coordinate system used for measuring the object, and the Z axis represents the depth of field and corresponds to the X axis in the display device. The right side is the coordinate system of the display device, and the Z-axis represents the height of the object, corresponding to the Y-axis in the three-dimensional coordinate system. Therefore, the problem can be solved only by exchanging the values of the coordinate system according to the difference of the coordinate system.
Similarly, the value of (x, y, z) in real three-dimensional coordinates is generally much larger than the range represented by the display device, so the spatial point is set as the point in space by zooming as necessaryThe scaling formula is:
(25)
wherein,
is a scaling factor.
In addition, the position in the display device and the three-dimensional coordinate may be different, so that a translation operation is required, and the translation transformation formula is as follows:
wherein,
is the translation factor.
Fourth embodiment
An embodiment of the present invention further provides a display device for handwriting input, as shown in fig. 8, including:
the receiving module is used for receiving the three-dimensional coordinates of the track input by the user;
the first processing module is connected with the receiving module and used for determining a fitting plane of the user input track; selecting a straight line parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system from the fitting plane as an X axis in a two-dimensional coordinate system; selecting a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system from the fitting plane as a Y axis in the two-dimensional coordinate system, so as to establish the two-dimensional coordinate system;
the second processing module is connected with both the receiving module and the first processing module and is used for converting the three-dimensional coordinates of the user input track into the two-dimensional coordinate system;
and the output module is connected with the second processing module and is used for displaying and outputting according to the two-dimensional coordinates of the user input track.
The first processing module can determine a three-dimensional world coordinate system taking a user as a reference system in a mode of pattern recognition.
The first processing module can also determine a three-dimensional world coordinate system taking the user as a reference system by identifying a pre-marked reference point.
The device of the embodiment of the invention provides a solution for converting the spatial point information of the recognized user input track into the two-dimensional output display information aiming at the two-dimensional display output mode, and fully considers the three-dimensional spatial angle of the user while considering the efficient recognition and optimally presenting the user input track, thereby ensuring that the displayed content is the input track expected by the user.
Preferably, the first processing module is configured to select a straight line with a direction vector as a product of the normal vectors from the fitting plane by calculating a product of the normal vector of the fitting plane and a normal vector of a horizontal plane in a three-dimensional world coordinate system with a user as a reference system, so as to obtain a straight line on the fitting plane, which is parallel to the horizontal plane in the three-dimensional world coordinate system with the user as the reference system.
Preferably, when it is determined that the fitting plane of the user input trajectory coincides with the horizontal plane in the three-dimensional world coordinate system using the user as the reference system, the first processing module selects a straight line, which is consistent with the horizontal direction in the three-dimensional world coordinate system using the user as the reference system, from the fitting plane of the user input trajectory as the X-axis in the two-dimensional coordinate system.
Preferably, the first processing module is configured to obtain a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system on the fitting plane by calculating an intersection line between the plane using the straight line serving as the X axis in the two-dimensional coordinate system as a normal line and the fitting plane.
Preferably, the first processing module is configured to select an X axis and a Y axis in a two-dimensional coordinate system, and enable both the X coordinate and the Y coordinate of the user input track converted into the two-dimensional coordinate system to be greater than or equal to 0.
Preferably, the first processing module is configured to convert the three-dimensional coordinates of the received user input trajectory into coordinates in a three-dimensional world coordinate system using a user as a reference system, and select a straight line, where an X-axis in the two-dimensional coordinate system is a point where a Y-coordinate of the user input trajectory in the three-dimensional world coordinate system using the user as the reference system is the minimum, so that the X-coordinates of the user input trajectory converted into the two-dimensional coordinate system are all greater than or equal to 0.
Preferably, the first processing module is configured to convert the three-dimensional coordinates of the received user input trajectory into coordinates in a three-dimensional world coordinate system using a user as a reference system; and selecting a Y axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum X coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that Y coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
Preferably, the output module is configured to scale and/or translate the two-dimensional coordinate of the user input track, and then perform display output.
Fifth embodiment
The embodiment of the invention also provides a display system for handwriting input, which comprises:
the measuring and calculating module is used for measuring and calculating the three-dimensional coordinates of the input track of the user;
and the display device for handwriting input is used for converting the input track of the user from the three-dimensional coordinate system into the two-dimensional coordinate system and displaying and outputting the input track.
For the specific structure of the display device for handwriting input, refer to the fourth embodiment, and details are not repeated here.
The measuring and calculating module is a stereoscopic vision recognition device or a radar sensing device or an infrared sensing device. And measuring and calculating the three-dimensional coordinates of the user input track through a stereoscopic vision recognition method, or measuring and calculating the three-dimensional coordinates of the user input track through sensing the distance.
The disclosed embodiments are provided to enable those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope or spirit of the invention. The above-described embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (24)
1. A display method of a handwriting input, comprising:
receiving a three-dimensional coordinate of a user input track;
determining a fitting plane of the user input track;
selecting a straight line parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system from the fitting plane as an X axis in a two-dimensional coordinate system;
selecting a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system from the fitting plane as a Y axis in the two-dimensional coordinate system;
and converting the three-dimensional coordinates of the user input track into the two-dimensional coordinate system, and displaying and outputting the three-dimensional coordinates.
2. The method of claim 1, wherein:
and determining a three-dimensional world coordinate system taking the user as a reference system by adopting a mode identification mode.
3. The method of claim 1, wherein:
and determining a three-dimensional world coordinate system taking the user as a reference system by identifying the pre-marked reference points.
4. A method according to any of claims 1-3, characterized by:
calculating the product of the normal vector of the fitting plane and the normal vector of a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system;
and selecting a straight line with a direction vector as the product of the normal vectors from the fitting plane to obtain a straight line on the fitting plane, wherein the straight line is parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system.
5. The method of any of claims 1-4, wherein:
and when the fitting plane is coincident with a horizontal plane in the three-dimensional world coordinate system taking the user as the reference system, selecting a straight line which is consistent with the horizontal direction in the three-dimensional world coordinate system taking the user as the reference system from the fitting plane as an X axis in the two-dimensional coordinate system.
6. The method of any of claims 1-5, wherein:
and calculating an intersection line of a plane taking the straight line as the X axis in the two-dimensional coordinate system as a normal line and the fitting plane to obtain a straight line perpendicular to the straight line as the X axis in the two-dimensional coordinate system on the fitting plane.
7. The method of any of claims 1-6, wherein:
and selecting an X axis and a Y axis in a two-dimensional coordinate system, and enabling the X coordinate and the Y coordinate of the user input track converted into the two-dimensional coordinate system to be greater than or equal to 0.
8. The method of claim 7, wherein:
converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system with the user as a reference system;
and selecting an X axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum Y coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that the X coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
9. The method of claim 7, wherein:
converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system with the user as a reference system;
and selecting a Y axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum X coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that Y coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
10. The method of claim 1, wherein:
the user input trajectory converted into the two-dimensional coordinate system is also scaled and/or translated prior to displaying the output.
11. A display method of a handwriting input, comprising:
measuring and calculating the three-dimensional coordinates of the input track of the user;
the method of any of claims 1-10, wherein the user input trajectory is transformed from a three-dimensional coordinate system to a two-dimensional coordinate system, and the output is displayed.
12. The method of claim 11, wherein:
and measuring and calculating the three-dimensional coordinates of the input track of the user by adopting a stereoscopic vision identification mode or a radar sensing mode or an infrared sensing mode.
13. A display device for handwriting input, comprising:
the receiving module is used for receiving the three-dimensional coordinates of the track input by the user;
the first processing module is connected with the receiving module and used for determining a fitting plane of the user input track according to the three-dimensional coordinates of the user input track; selecting a straight line parallel to a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system from the fitting plane as an X axis in a two-dimensional coordinate system; selecting a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system from the fitting plane as a Y axis in the two-dimensional coordinate system, so as to establish the two-dimensional coordinate system;
the second processing module is connected with both the receiving module and the first processing module and is used for converting the three-dimensional coordinates of the user input track into the two-dimensional coordinate system;
and the output module is connected with the second processing module and is used for displaying and outputting according to the two-dimensional coordinates of the user input track.
14. The apparatus of claim 13, wherein:
the first processing module is used for determining a three-dimensional world coordinate system which takes a user as a reference system in a mode of pattern recognition.
15. The apparatus of claim 13, wherein:
the first processing module is used for determining a three-dimensional world coordinate system taking a user as a reference system by identifying a pre-marked reference point.
16. The apparatus of any of claims 13-15, wherein:
the first processing module is used for selecting a straight line with a direction vector as the product of the normal vectors from the fitting plane by calculating the product of the normal vectors of the fitting plane and the normal vectors of a horizontal plane in a three-dimensional world coordinate system taking a user as a reference system, so as to obtain the straight line on the fitting plane, which is parallel to the horizontal plane in the three-dimensional world coordinate system taking the user as the reference system.
17. The apparatus of any of claims 13-16, wherein:
and the first processing module selects a straight line which is consistent with the horizontal direction in the three-dimensional world coordinate system taking the user as the reference system from the fitting plane of the user input track as the X axis in the two-dimensional coordinate system when judging that the fitting plane of the user input track is coincident with the horizontal plane in the three-dimensional world coordinate system taking the user as the reference system.
18. The apparatus of any of claims 13-17, wherein:
the first processing module is configured to obtain a straight line perpendicular to the straight line serving as the X axis in the two-dimensional coordinate system on the fitting plane by calculating an intersection line between the plane using the straight line serving as the X axis in the two-dimensional coordinate system as a normal line and the fitting plane.
19. The apparatus of any of claims 13-18, wherein:
the first processing module is used for selecting an X axis and a Y axis in a two-dimensional coordinate system, and enabling the X coordinate and the Y coordinate of the user input track converted into the two-dimensional coordinate system to be larger than or equal to 0.
20. The apparatus of claim 19, wherein:
the first processing module is used for converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system taking a user as a reference system, and selecting a straight line of which the X axis in the two-dimensional coordinate system is the minimum point of the Y coordinate in the three-dimensional world coordinate system taking the user as the reference system through the user input track, so that the X coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
21. The apparatus of claim 19, wherein:
the first processing module is used for converting the three-dimensional coordinates of the received user input track into coordinates in a three-dimensional world coordinate system with a user as a reference system; and selecting a Y axis in the two-dimensional coordinate system as a straight line passing through a point with the minimum X coordinate of the user input track in a three-dimensional world coordinate system taking a user as a reference system, so that Y coordinates of the user input track converted into the two-dimensional coordinate system are all larger than or equal to 0.
22. The apparatus of claim 13, wherein:
and the output module is used for zooming and/or translating the two-dimensional coordinates of the track input by the user and then displaying and outputting the two-dimensional coordinates.
23. A display system for handwriting input, comprising:
the measuring and calculating module is used for measuring and calculating the three-dimensional coordinates of the input track of the user;
a display device for handwriting input according to any of claims 13-22 and arranged to convert a user input trajectory from a three-dimensional coordinate system into said two-dimensional coordinate system for display output thereof.
24. The system of claim 23, wherein:
the measuring and calculating module is a stereoscopic vision recognition device or a radar sensing device or an infrared sensing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103518803A CN102426509A (en) | 2011-11-08 | 2011-11-08 | Method, device and system for displaying hand input |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103518803A CN102426509A (en) | 2011-11-08 | 2011-11-08 | Method, device and system for displaying hand input |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102426509A true CN102426509A (en) | 2012-04-25 |
Family
ID=45960499
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011103518803A Pending CN102426509A (en) | 2011-11-08 | 2011-11-08 | Method, device and system for displaying hand input |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102426509A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103150022A (en) * | 2013-03-25 | 2013-06-12 | 深圳泰山在线科技有限公司 | Gesture identification method and gesture identification device |
| CN103674018A (en) * | 2013-12-20 | 2014-03-26 | 河北汉光重工有限责任公司 | Two-dimensional displaying method for three-dimensional flight path |
| CN103823554A (en) * | 2014-01-12 | 2014-05-28 | 青岛科技大学 | Digital virtual-real interaction system and digital virtual-real interaction method |
| CN104571521A (en) * | 2015-01-21 | 2015-04-29 | 京东方科技集团股份有限公司 | Device and method of handwritten record |
| CN106125994A (en) * | 2016-06-17 | 2016-11-16 | 深圳迪乐普数码科技有限公司 | Coordinate matching method and use control method and the terminal of this coordinate matching method |
| CN106774974A (en) * | 2016-11-29 | 2017-05-31 | 网易（杭州）网络有限公司 | The method and apparatus of output information |
| CN107179839A (en) * | 2017-05-23 | 2017-09-19 | 三星电子（中国）研发中心 | Information output method, device and equipment for terminal |
| CN108459782A (en) * | 2017-02-17 | 2018-08-28 | 阿里巴巴集团控股有限公司 | A kind of input method, device, equipment, system and computer storage media |
| CN112199015A (en) * | 2020-09-15 | 2021-01-08 | 安徽鸿程光电有限公司 | Intelligent interaction all-in-one machine and writing method and device thereof |
| CN114745579A (en) * | 2022-03-18 | 2022-07-12 | 阿里巴巴（中国）有限公司 | Interaction method based on space writing interface, terminal and storage medium |
-
2011
- 2011-11-08 CN CN2011103518803A patent/CN102426509A/en active Pending
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103150022A (en) * | 2013-03-25 | 2013-06-12 | 深圳泰山在线科技有限公司 | Gesture identification method and gesture identification device |
| CN103674018A (en) * | 2013-12-20 | 2014-03-26 | 河北汉光重工有限责任公司 | Two-dimensional displaying method for three-dimensional flight path |
| CN103823554A (en) * | 2014-01-12 | 2014-05-28 | 青岛科技大学 | Digital virtual-real interaction system and digital virtual-real interaction method |
| CN104571521B (en) * | 2015-01-21 | 2018-09-11 | 京东方科技集团股份有限公司 | Hand-written recording equipment and hand-written recording method |
| CN104571521A (en) * | 2015-01-21 | 2015-04-29 | 京东方科技集团股份有限公司 | Device and method of handwritten record |
| CN106125994B (en) * | 2016-06-17 | 2019-04-23 | 深圳迪乐普数码科技有限公司 | Coordinate matching method and the control method and terminal for using the coordinate matching method |
| CN106125994A (en) * | 2016-06-17 | 2016-11-16 | 深圳迪乐普数码科技有限公司 | Coordinate matching method and use control method and the terminal of this coordinate matching method |
| CN106774974A (en) * | 2016-11-29 | 2017-05-31 | 网易（杭州）网络有限公司 | The method and apparatus of output information |
| CN106774974B (en) * | 2016-11-29 | 2019-08-13 | 网易（杭州）网络有限公司 | The method and apparatus of output information |
| CN108459782A (en) * | 2017-02-17 | 2018-08-28 | 阿里巴巴集团控股有限公司 | A kind of input method, device, equipment, system and computer storage media |
| CN107179839A (en) * | 2017-05-23 | 2017-09-19 | 三星电子（中国）研发中心 | Information output method, device and equipment for terminal |
| CN112199015A (en) * | 2020-09-15 | 2021-01-08 | 安徽鸿程光电有限公司 | Intelligent interaction all-in-one machine and writing method and device thereof |
| CN112199015B (en) * | 2020-09-15 | 2022-07-22 | 安徽鸿程光电有限公司 | Intelligent interaction all-in-one machine and writing method and device thereof |
| CN114745579A (en) * | 2022-03-18 | 2022-07-12 | 阿里巴巴（中国）有限公司 | Interaction method based on space writing interface, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102426509A (en) | Method, device and system for displaying hand input | |
| TW452723B (en) | Method and apparatus for three-dimensional input entry | |
| CN103472916B (en) | A kind of man-machine interaction method based on human body gesture identification | |
| CN104199550B (en) | Virtual keyboard operation device, system and method | |
| TWI540461B (en) | Gesture input method and system | |
| JP5656514B2 (en) | Information processing apparatus and method | |
| CN102868811B (en) | Mobile phone screen control method based on real-time video processing | |
| CN109145802B (en) | Kinect-based multi-person gesture man-machine interaction method and device | |
| JP6344530B2 (en) | Input device, input method, and program | |
| WO1999040562A1 (en) | Video camera computer touch screen system | |
| CN104423569A (en) | Pointing position detecting device, method and computer readable recording medium | |
| CN109471533B (en) | Student end system in VR/AR classroom and use method thereof | |
| US20150009119A1 (en) | Built-in design of camera system for imaging and gesture processing applications | |
| JP2004265222A (en) | Interface method, system, and program | |
| TWI486815B (en) | Display device, system and method for controlling the display device | |
| CN102609734A (en) | Machine vision-based handwriting recognition method and system | |
| Su et al. | Virtual keyboard: A human-computer interaction device based on laser and image processing | |
| Malik | An exploration of multi-finger interaction on multi-touch surfaces | |
| US20130187890A1 (en) | User interface apparatus and method for 3d space-touch using multiple imaging sensors | |
| KR101535738B1 (en) | Smart device with touchless controlling operation function and the control method of using the same | |
| CN114740997A (en) | Interaction control device and interaction control method | |
| CN103869941A (en) | Electronic device with virtual touch service and virtual touch immediate correction method | |
| CN104102332B (en) | Display device and its control system and method | |
| Yang et al. | Perceptual issues of a passive haptics feedback based MR system | |
| Karray et al. | A survey of ar systems and a case study of virtual keyboard based camera projector system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| ASS | Succession or assignment of patent right |
Owner name: BEIJING NUFRONT MOBILE MULTIMEDIA TECHNOLOGY CO., Free format text: FORMER OWNER: BEIJING NUFRONT SOFTWARE TECHNOLOGY CO., LTD. Effective date: 20121024 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20121024 Address after: 100084 Beijing, Zhongguancun, East Road, No. 1, Tsinghua Science and Technology Park, building science and Technology Park, building No. 8, block A, floor 16 Applicant after: Beijing Nufront Mobile Multimedia Technology Co., Ltd. Address before: 100084 Beijing, Zhongguancun, East Road, No. 1, Tsinghua Science and Technology Park, building science and Technology Park, building No. 8, block A, floor 16 Applicant before: Beijing Nufront Software Technology Co., Ltd. |
|
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120425 |