CN112419412A - Cursor positioning method, handheld cursor control equipment and screen control system - Google Patents
Cursor positioning method, handheld cursor control equipment and screen control system Download PDFInfo
- Publication number
- CN112419412A CN112419412A CN202011404093.6A CN202011404093A CN112419412A CN 112419412 A CN112419412 A CN 112419412A CN 202011404093 A CN202011404093 A CN 202011404093A CN 112419412 A CN112419412 A CN 112419412A
- Authority
- CN
- China
- Prior art keywords
- cursor
- display area
- coordinates
- reference point
- current frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20068—Projection on vertical or horizontal image axis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Position Input By Displaying (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application provides a cursor positioning method, a handheld cursor control device and a screen control system, wherein the cursor positioning method comprises the following steps: acquiring a current frame image through a camera, wherein the current frame image comprises a display area image; acquiring mark point acquisition coordinates of at least four display area mark points; acquiring mark point reference coordinates of at least four display area mark points; acquiring cursor reference point acquisition coordinates of a cursor reference point in a current frame image; and calculating to obtain relative coordinates of the cursor reference point projected to the display area by the cursor reference point according to the marker point acquisition coordinates, the marker point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, wherein the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area. The cursor positioning method provided by the application does not need positioning and calibration, is accurate in positioning and small in calculation amount, and solves the problem of high hysteresis caused by large calculation amount in the prior art.
Description
Technical Field
The application relates to the technical field of air-spaced control, in particular to a cursor positioning method, a handheld cursor control device and a screen control system.
Background
With the popularization of intelligent interactive large-screen and intelligent television and other equipment, touch technology is also updated and advanced, and in the prior art, various equipment has the capacity of realizing remote touch operation on the intelligent television, the large-screen and other equipment. Common devices such as an air mouse acquire relative movement information through sensors such as an accelerator and a gyroscope, and coordinates are calculated through accumulation of relative movement, so that acquisition errors can be gradually accumulated, and meanwhile, the current cursor position cannot be sensed when the device is started, and the cursor needs to be moved to the edge of a screen to correct the cursor position through the movement distance.
To solve this problem, a method for positioning a cursor using an image captured by a camera is proposed in the prior art, and specifically, a position of the cursor is determined by comparing an actually displayed picture in a display screen with a corresponding picture in the image captured by the camera, so that the calculation amount is large, the hysteresis of cursor movement is high, and the user experience is poor.
Content of application
Based on the above situation, a main objective of the present application is to provide a cursor positioning method, a handheld cursor control device and a screen control system, so as to solve the technical problem of high hysteresis of cursor movement in the prior art.
In order to achieve the above object, in a first aspect, the present application adopts the following technical solutions:
a cursor positioning method is used for positioning a cursor in a display area of a display device through a handheld cursor control device, the handheld cursor control device comprises a camera, the display device is provided with at least four display area mark points used for marking the position of the display area of the display device, and the connecting lines of the four display area mark points form a quadrangle, and the cursor positioning method comprises the following steps:
acquiring a current frame image through the camera, determining a cursor reference point in the current frame image, wherein the current frame image comprises the display area image, and identifying the at least four display area mark points in the current frame image;
constructing a first reference coordinate system by using the current frame image, acquiring coordinates of at least four display area mark points in the display area image in the first reference coordinate system, and taking the coordinates as mark point acquisition coordinates;
constructing a second reference coordinate system by using the display area, acquiring coordinates of the mark points of the at least four display areas in the second reference coordinate system, and taking the coordinates as reference coordinates of the mark points;
acquiring coordinates of a cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
and calculating to obtain relative coordinates of the cursor reference point projected to the display area by the cursor reference point according to the marker point acquisition coordinates, the marker point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, wherein the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area.
Optionally, the method for obtaining the relative coordinate of the cursor reference point projected onto the display area by calculating according to the marker point collecting coordinate, the marker point reference coordinate, and the cursor reference point collecting coordinate includes:
calculating to obtain a transformation matrix according to the marker point acquisition coordinates and the marker point reference coordinates, wherein the transformation matrix is used for projecting the pixel points of the current frame image to the display area;
and calculating to obtain the relative coordinates of the cursor reference point according to the transformation matrix and the cursor reference point acquisition coordinates.
Optionally, the cursor positioning method further includes:
sending out the relative coordinates of the cursor reference points obtained by calculation; alternatively, the first and second electrodes may be,
and after the relative coordinates of the cursor reference point are obtained through calculation, judging whether a display area of a current screen is in a rotating state, if so, converting the relative coordinates of the cursor reference point into pixel coordinates of the cursor in the display area based on an affine transformation rule, otherwise, directly taking the relative coordinates of the cursor reference point as the pixel coordinates of the cursor in the display area.
Optionally, before calculating the relative coordinate of the cursor reference point projected onto the display area by the cursor reference point according to the marker point collecting coordinate, the marker point reference coordinate, and the cursor reference point collecting coordinate, first determining whether the cursor reference point is located in the display area image, and if the cursor reference point is not located in the display area image, directly processing the next frame of image.
Optionally, the display area is a rectangle, and the display area mark points are four vertices of the display area.
Optionally, an infrared emission element is arranged at the edge of a display area of the display device, the camera is an infrared camera, and the position of the image of the display area in the current frame image is determined by infrared light emitted by the infrared emission element and shot by the infrared camera; alternatively, the first and second electrodes may be,
and determining the position of the image of the display area in the current frame image by comparing the color difference between the pixel points in the current frame image.
In a second aspect, the technical solution adopted by the present application is as follows:
a computer-readable storage medium storing a computer program which, when executed, implements a cursor positioning method as described above.
In a third aspect, the technical solution adopted by the present application is as follows:
a hand-held cursor control device for positioning a cursor in a display area of a display device, comprising:
a camera;
the communication module is used for being in communication connection with a system control end of the display device;
the key is used for sending a control instruction to the system control end through the communication module; and
a processing module comprising a receiving unit, a first obtaining unit, a second obtaining unit, a third obtaining unit, and a calculating unit, wherein,
the receiving unit is used for receiving a current frame image which is shot by the camera and comprises a display area image, determining a cursor reference point in the current frame image and identifying the at least four display area mark points in the current frame image;
a first obtaining unit, configured to construct a first reference coordinate system with the current frame image, obtain coordinates of at least four display area mark points in the display area image in the first reference coordinate system, and use the coordinates as mark point acquisition coordinates, where the display area mark points are used to identify the position of the display area, and a connecting line of four display area mark points in the at least four display area mark points forms a quadrangle;
the second acquisition unit is used for constructing a second reference coordinate system by using the display area, acquiring the coordinates of the mark points of the display area in the second reference coordinate system and taking the coordinates as the reference coordinates of the mark points;
the third acquisition unit is used for acquiring the coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
and the calculating unit is used for calculating to obtain the relative coordinate of the cursor reference point projected to the display area by the cursor reference point according to the mark point acquisition coordinate, the mark point reference coordinate and the cursor reference point acquisition coordinate based on a perspective transformation rule, wherein the relative coordinate of the cursor reference point is used for determining the pixel coordinate of the cursor in the display area.
In a fourth aspect, the technical solution adopted by the present application is as follows:
a screen control system comprising:
the handheld cursor control device comprises a camera, a communication module and a key;
the system control end of the display device can be in communication connection with the communication module of the handheld cursor control equipment, the display device is provided with at least four display area mark points for marking the display area position of the display device, and the connecting lines of the four display area mark points form a quadrangle;
a processing module comprising a receiving unit, a first obtaining unit, a second obtaining unit, a third obtaining unit, and a calculating unit, wherein,
the receiving unit is used for receiving a current frame image which is shot by the camera and comprises a display area image, determining a cursor reference point in the current frame image and identifying the at least four display area mark points in the current frame image;
the first acquisition unit is used for constructing a first reference coordinate system by using the current frame image, acquiring the coordinates of at least four display area mark points in the display area image in the first reference coordinate system, and taking the coordinates as mark point acquisition coordinates;
the second acquisition unit is used for constructing a second reference coordinate system by using the display area, acquiring the coordinates of the mark points of the display area in the second reference coordinate system and taking the coordinates as the reference coordinates of the mark points;
the third acquisition unit is used for acquiring the coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
the computing unit is used for computing relative coordinates of a cursor reference point projected to the display area by the cursor reference point according to the mark point acquisition coordinates, the mark point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, and the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area;
the processing module is arranged in the hand-held cursor control device, or in the system control end, or one part is arranged in the hand-held cursor control device, and the other part is arranged in the system control end.
Optionally, the display device is rectangular, infrared emitting elements are arranged at four corners of the display area, the camera is an infrared camera, and the position of the image of the display area in the current frame image is determined by infrared light emitted by the infrared emitting elements and shot by the infrared camera.
According to the cursor positioning method, the coordinates of the display area mark points in the current frame image and the coordinates of the display area mark points in the display area actually are directly utilized to carry out cursor positioning calculation, positioning and calibration are not needed, positioning is accurate, the calculation amount is small, and the problem of high hysteresis caused by large calculation amount in the prior art is solved.
Other advantages of the present application will be described in the detailed description, and those skilled in the art will understand the technical features and technical solutions presented in the description.
Drawings
Preferred embodiments according to the present application will be described below with reference to the accompanying drawings. In the figure:
fig. 1 is a flowchart of a cursor positioning method according to an embodiment of the present application;
fig. 2 is a diagram illustrating an arrangement of an infrared emission unit according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of another arrangement of an IR emitting unit according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a first reference coordinate system in a cursor positioning method according to an embodiment of the present invention;
FIG. 5 is a diagram of a second reference coordinate system in the cursor positioning method according to the embodiment of the present invention;
FIG. 6 is a block diagram of a handheld cursor control device according to an embodiment of the present invention;
fig. 7 is a block diagram of a screen control system according to an embodiment of the present invention;
fig. 8 is a block diagram of a screen control system according to another embodiment of the present invention.
In the figure, 10, a hand-held cursor control device; 11. a camera; 12. a communication module; 13. pressing a key; 14. a processing module;
10', a handheld cursor control device; 11', a camera; 12', a communication module; 13', a key;
20. a display device;
30. a system control end; 31. and a processing module.
Detailed Description
The present application is described below based on examples, but the present application is not limited to only these examples. In the following detailed description of the present application, certain specific details are set forth in order to avoid obscuring the nature of the present application, well-known methods, procedures, and components have not been described in detail.
Further, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale.
Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is, what is meant is "including, but not limited to".
In the description of the present application, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Aiming at the problems of large operation amount and high movement hysteresis existing in the existing cursor positioning technology, the application provides a cursor positioning method, which is used for positioning a cursor in a display area of a display device through a handheld cursor control device, namely, the cursor is displayed in the display area of the display device, the handheld cursor control device is used as a pointing device similar to a mouse, and the position of the cursor in the display area is controlled and changed by moving the handheld cursor control device, namely, the cursor moves along with the movement of the handheld cursor control device.
The hand-held cursor control device comprises a camera, the display device is provided with at least four display area mark points for marking the display area position of the display device, the connecting lines of the four display area mark points in the at least four display area mark points form a quadrangle, namely, no three points of the four display area mark points are positioned on the same straight line, as shown in fig. 1, the cursor positioning method comprises the following steps:
s10, acquiring a current frame image through the camera, determining a cursor reference point in the current frame image, wherein the current frame image comprises the image of the display area, and identifying the at least four display area mark points in the current frame image;
s20, constructing a first reference coordinate system by using the current frame image, acquiring coordinates of at least four display area mark points in the image of the display area in the first reference coordinate system, and taking the coordinates as mark point acquisition coordinates;
s30, constructing a second reference coordinate system by the display area, acquiring coordinates of the mark points of the at least four display areas in the second reference coordinate system, and taking the coordinates as the reference coordinates of the mark points;
s40, acquiring coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
and S50, calculating to obtain relative coordinates of the cursor reference point projected to the display area by the cursor reference point according to the marker point acquisition coordinates, the marker point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, wherein the relative coordinates of the cursor reference point are used for determining the pixel coordinates of the cursor in the display area.
The positions of the display area mark points can be set according to specific situations, and since a perspective transformation rule is used in the subsequent calculation, the number of the display area mark points needs to be set to be at least 4. To facilitate the acquisition of the coordinates of the display region indicating point in the current frame image in step S20, the calculation process is simplified, and optionally, the display region indicating point is set as a boundary point of the display region, for example, when the display region is rectangular, four vertices of the display region are optionally used as the display region indicating points.
In step S20, the method for identifying the mark point of the display area in the current frame image can be arbitrarily selected, for example, because the frame of the display area is usually pure color, such as pure black, and is clearly different from the surrounding background, the boundary of the display area image can be determined by comparing the color difference between the pixel points in the current frame image, so as to obtain the position of the image of the display area in the current frame image, and further obtain the position of the mark point of the display area in the current frame image. For another example, an infrared emitting element is disposed at an edge of a display area of the display device, and the camera is set as an infrared camera, so that the infrared camera can shoot infrared light emitted by the infrared emitting element, so as to determine a position of an image of the display area in the current frame image, and further obtain a position of a display area mark point in the current frame image, where the display area is generally rectangular, optionally, as shown in fig. 2, infrared emitting elements are disposed at four corners of the display area, and a position where the infrared camera shoots infrared light at this time is a position of the display area mark point in the current frame image. When the display device is an infrared touch screen, the infrared emission elements are arranged in the infrared touch frame of the infrared touch screen to emit infrared light, for example, as shown in fig. 3, rows of infrared emission elements are arranged on the top edge and the right edge of the infrared touch frame, and in this case, the infrared emission elements do not need to be additionally arranged, and the image position of the display area can be determined by using the infrared light emitted by the infrared emission elements.
In step S20, the coordinates of the display area marker point in the first reference coordinate system may be obtained from the image pixel position, that is, each pixel point in the current frame image has its own coordinate position, and the coordinate position of the pixel point corresponding to the display area marker point is the display area marker point collection coordinate. For example, as shown in fig. 4, in A, B, C, D a first reference coordinate system, four display area mark points are identified as E, F, G, and H respectively at four vertices of a current frame image, and coordinates of the E, F, G, and H points in the first reference coordinate system are P respectivelyE，PF，PG，PH。
In step S30, since the position of the mark point in the display area is fixed, the reference coordinate of the mark point can be directly pre-stored in the memory, and can be directly retrieved when used. For example, as shown in fig. 5, when the width of the display area is w and the height is h, the coordinates of the four display area mark points in the second reference coordinate system are P'E(0，0)，P′F(w，O)，P′G(w，h)，P′H(O，h)。
In step S40, the cursor reference point is a point for indicating the cursor position, that is, the cursor position that actually converts the cursor reference point into the display area during calculation. It can select any point in the current frame image, for example, the cursor reference point is the middle point of the current frame image. Because the position of the point is fixed, the collected coordinates of the cursor reference point can be directly pre-stored in a memory, and the collected coordinates can be directly called when in use.
In step S50, since the camera image conforms to a Perspective Transformation (Perspective Transformation) rule, that is, the display area plane is projected onto the photo-sensitive chip plane of the camera, the relative coordinates of the cursor reference point can be calculated based on the Perspective Transformation rule.
Specifically, the perspective transformation needs to satisfy simultaneously:
due to PE，PF，PG，PH，PE’，PF’，PG’，PHThe coordinates of' are known, from which a transformation matrix can be derived:
the transformation matrix is used for projecting the pixel points of the current frame image to the display area.
Therefore, the relative coordinates of the cursor reference point can be obtained through calculation according to the transformation matrix and the cursor reference point collection coordinates.
It is understood that S20, S30 and S40 are not limited to a sequential order, and may be performed in any order, or any two or three of them may be performed simultaneously.
However, in the case where the display content of the screen is rotatable, if the display area is actually rotated, the relative coordinates of the cursor reference point are still not aligned with the apparent position of the pixel coordinates of the cursor, and therefore it is necessary to determine whether the display area is in a rotated state, and if the display area is in a rotated state, the relative coordinates of the cursor reference point need to be converted based on an affine transformation rule to obtain the pixel coordinates of the cursor in the display area. Specifically, the transformation needs to use an affine transformation matrix, which can be calculated in real time according to the rotation angles of the display area, and since the rotation angles of the display area are usually determined to be a few, the affine transformation matrix corresponding to each rotation angle can also be directly pre-stored in the memory and directly called when in use.
It can be understood that the determination of whether the rotation occurs and the conversion of the coordinates may not belong to the cursor positioning method described in the present application, that is, the cursor positioning is completed by sending out the calculated relative coordinates of the cursor reference point, or the determination of the rotation and the conversion of the coordinates may be included.
Optionally, before calculating the relative coordinate of the cursor reference point projected onto the display area by the cursor reference point according to the marker point collecting coordinate, the marker point reference coordinate, and the cursor reference point collecting coordinate, first determining whether the cursor reference point is located in the display area image, and if the cursor reference point is not located in the display area image, it indicates that the camera does not move to the effective position, at this time, the next frame of image may be directly processed, so that the processing efficiency may be improved, and unnecessary calculation may be avoided.
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed, implements a cursor positioning method as described above. Note that the computer readable storage medium described in the embodiments of the present disclosure may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The present application further provides a handheld cursor control device 10 for positioning a cursor in a display area of a display device, as shown in fig. 6, which includes a camera 11, a communication module 12, a key 13 and a processing module 14. The camera 11 is used for acquiring images. The communication module 12 is used for communicating with a system control end of the display device, the system control end is, for example, a host of the display device, and the handheld cursor control device 10 communicates with the system control end through the communication module 12 to transmit information. The key 13 is used for sending a control instruction to the system control end through the communication module 12, where the control instruction is, for example, a click, a shortcut key, and the like. The processing module 14 comprises a receiving unit, a first acquiring unit, a second acquiring unit, a third acquiring unit and a calculating unit, wherein,
the receiving unit is configured to receive a current frame image including a display area image captured by the camera 11, determine a cursor reference point in the current frame image, and identify the at least four display area marker points in the current frame image.
The first obtaining unit is used for establishing a first reference coordinate system by using the current frame image, obtaining coordinates of at least four display area mark points in the display area image in the first reference coordinate system, taking the coordinates as mark point collecting coordinates, wherein the display area mark points are used for marking the position of the display area, and connecting lines of the four display area mark points form a quadrangle.
The second acquisition unit is used for constructing a second reference coordinate system by using the display area, acquiring the coordinates of the display area mark points in the second reference coordinate system, and taking the coordinates as the mark point reference coordinates.
The third acquiring unit is used for acquiring coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates.
The calculation unit is used for calculating to obtain relative coordinates of the cursor reference point projected to the display area by the cursor reference point according to the marker point acquisition coordinates, the marker point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, and the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area.
The specific processing of the processing module 14 is the same as that in the cursor positioning method, and is not described herein again.
The pixel coordinates of the display area obtained by the processing module 14 may be directly transmitted to the system control end of the display device through the communication module 12, and the determination of the pixel coordinates of the display area is performed through the system control end, for example, when the display device can rotate the display area, the system control end determines whether the rotation occurs and converts the coordinates. For example, the processing module 14 further includes a judging module and a transforming module, where the judging module is configured to judge whether the display area rotates according to the display area angle information sent by the system control terminal and received by the communication module 12, and the transforming module is configured to transform the relative coordinate of the cursor reference point when the display area is judged to rotate, so as to obtain the pixel coordinate of the cursor in the display area.
The present application further provides a screen control system, as shown in fig. 7, which includes a handheld cursor control device 10 and a display device 20, in an embodiment, the handheld cursor control device 10 may be the structure shown in fig. 6, that is, the handheld cursor control device 10 includes a camera 11, a communication module 12, a key 13 and a processing module 14, the camera 11 is used for acquiring an image, the communication module 12 is used for being in communication connection with a system control end 30 of the display device 20, the system control end 30 is, for example, a host of the display device 20, and the handheld cursor control device 10 communicates with the system control end 30 through the communication module 12 to transmit information. The key 13 is used for sending a control instruction to the system control end 30 through the communication module 12, where the control instruction is, for example, a click, a shortcut key, and the like. The display device 20 has at least four display area mark points for marking the display area position, and the connecting lines of four display area mark points in the at least four display area mark points form a quadrangle.
The processing module 14 comprises a receiving unit, a first acquiring unit, a second acquiring unit, a third acquiring unit and a calculating unit, wherein,
the receiving unit is configured to receive a current frame image including a display area image captured by the camera 11, determine a cursor reference point in the current frame image, and identify the at least four display area marker points in the current frame image.
The first acquisition unit is used for constructing a first reference coordinate system by using the current frame image, acquiring the coordinates of at least four display area mark points in the display area image in the first reference coordinate system, and taking the coordinates as mark point acquisition coordinates.
The second acquisition unit is used for constructing a second reference coordinate system by using the display area, acquiring the coordinates of the display area mark points in the second reference coordinate system, and taking the coordinates as the mark point reference coordinates.
The third acquiring unit is used for acquiring coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates.
The calculation unit is used for calculating to obtain relative coordinates of the cursor reference point projected to the display area by the cursor reference point according to the marker point acquisition coordinates, the marker point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, and the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area.
When the display device 20 can rotate the display area, the processing module 14 further determines whether the display area is rotated and converts the coordinate, and specifically, the processing module 14 further includes a determining module and a converting module, where the determining module is configured to determine whether the display area is rotated according to the display area angle information sent by the system control terminal 30 and received by the communication module 12, and the converting module is configured to convert the relative coordinate of the cursor reference point when the display area is determined to be rotated, so as to obtain the pixel coordinate of the cursor in the display area.
In another embodiment, as shown in fig. 8, the processing module 31 is integrally disposed in the system control end 30, that is, the handheld cursor control device 10 ' is only responsible for acquiring and transmitting images, and the calculation process is completed at the system control end 30, specifically, the handheld cursor control device 10 ' includes a camera 11 ', a communication module 12 ' and a key 13 ', the camera 11 ' is used for acquiring images, the communication module 12 ' is used for communicating with the system control end 30 of the display device 20, the system control end 30 is, for example, a host of the display device 20, and the handheld cursor control device 10 ' communicates with the system control end 30 through the communication module 12 ' to transmit information. The key 13 'is used for sending a control instruction to the system control terminal 30 through the communication module 12', where the control instruction is, for example, a click, a shortcut key, and the like. The handheld cursor control device 10 'sends the image information acquired by the camera 11' to the processing module 31 of the system control end 30, where the processing module 31 includes a receiving unit, a first acquiring unit, a second acquiring unit, a third acquiring unit and a calculating unit, and a processing process of the processing module 31 on the image is similar to a processing process of the processing module 14 in the embodiment shown in fig. 7, and is not described herein again. Of course, it is understood that a part of the processing module may be disposed in the handheld cursor control device, and another part may be disposed in the system control end, that is, a part of the operation is performed in the handheld cursor control device, and another part of the operation is performed in the system control end.
Optionally, the display device is rectangular, infrared emitting elements are arranged at four corners of the display area, the camera is an infrared camera, and the position of the image of the display area in the current frame image is determined by infrared light emitted by the infrared emitting elements and shot by the infrared camera.
It will be appreciated by those skilled in the art that the alternatives described above may be freely combined, superimposed without conflict.
It will be understood that the embodiments described above are illustrative only and not restrictive, and that various obvious or equivalent modifications and substitutions for details described herein may be made by those skilled in the art without departing from the basic principles of the present application.
Claims (10)
1. A cursor positioning method is used for positioning a cursor in a display area of a display device through a handheld cursor control device, the handheld cursor control device is provided with a camera, the display device is provided with at least four display area mark points used for marking the position of the display area of the display device, and the connecting line of four display area mark points in the at least four display area mark points forms a quadrangle, the cursor positioning method comprises the following steps:
acquiring a current frame image through the camera, determining a cursor reference point in the current frame image, wherein the current frame image comprises the display area image, and identifying the at least four display area mark points in the current frame image;
constructing a first reference coordinate system by using the current frame image, acquiring coordinates of the at least four display area mark points in the display area image in the first reference coordinate system, and taking the coordinates as mark point acquisition coordinates;
constructing a second reference coordinate system by using the display area, acquiring coordinates of the mark points of the at least four display areas in the second reference coordinate system, and taking the coordinates as reference coordinates of the mark points;
acquiring coordinates of a cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
and calculating to obtain relative coordinates of the cursor reference point projected to the display area by the cursor reference point according to the marker point acquisition coordinates, the marker point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, wherein the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area.
2. The cursor positioning method according to claim 1, wherein the method of calculating relative coordinates of the cursor reference point projected onto the display area from the marker point collecting coordinates, the marker point reference coordinates, and the cursor reference point collecting coordinates comprises:
calculating to obtain a transformation matrix according to the marker point acquisition coordinates and the marker point reference coordinates, wherein the transformation matrix is used for projecting the pixel points of the current frame image to the display area;
and calculating to obtain the relative coordinates of the cursor reference point according to the transformation matrix and the cursor reference point acquisition coordinates.
3. The cursor positioning method of claim 1, further comprising:
sending out the relative coordinates of the cursor reference points obtained by calculation; alternatively, the first and second electrodes may be,
and after the relative coordinates of the cursor reference point are obtained through calculation, judging whether a display area of a current screen is in a rotating state, if so, converting the relative coordinates of the cursor reference point into pixel coordinates of the cursor in the display area based on an affine transformation rule, otherwise, directly taking the relative coordinates of the cursor reference point as the pixel coordinates of the cursor in the display area.
4. The cursor positioning method according to any one of claims 1 to 3, wherein before calculating the relative coordinates of the cursor reference point projected onto the display area from the marker point collection coordinates, the marker point reference coordinates, and the cursor reference point collection coordinates, it is first determined whether the cursor reference point is located in the display area image, and if the cursor reference point is not located in the display area image, the next frame image is directly processed.
5. The cursor positioning method of any one of claims 1 to 3, wherein the display area is rectangular, and the display area marker points are four vertices of the display area.
6. The cursor positioning method according to any one of claims 1 to 3, wherein an infrared emitting element is disposed at an edge of a display area of the display device, the camera is an infrared camera, and a position of the display area image in the current frame image is determined by infrared light emitted from the infrared emitting element captured by the infrared camera; alternatively, the first and second electrodes may be,
and determining the position of the display area image in the current frame image by comparing the color difference between the pixel points in the current frame image.
7. A computer-readable storage medium, storing a computer program, characterized in that the computer program, when executed, implements a cursor positioning method as claimed in any one of claims 1-6.
8. A hand-held cursor control device for positioning a cursor in a display area of a display device, comprising:
a camera;
the communication module is used for being in communication connection with a system control end of the display device;
the key is used for sending a control instruction to the system control end through the communication module; and
a processing module comprising a receiving unit, a first obtaining unit, a second obtaining unit, a third obtaining unit, and a calculating unit, wherein,
the receiving unit is used for receiving a current frame image which is shot by the camera and comprises a display area image, determining a cursor reference point in the current frame image and identifying the at least four display area mark points in the current frame image;
a first obtaining unit, configured to construct a first reference coordinate system with the current frame image, obtain coordinates of at least four display area mark points in the display area image in the first reference coordinate system, and use the coordinates as mark point acquisition coordinates, where the display area mark points are used to identify the position of the display area, and a connecting line of four display area mark points in the at least four display area mark points forms a quadrangle;
the second acquisition unit is used for constructing a second reference coordinate system by using the display area, acquiring the coordinates of the mark points of the display area in the second reference coordinate system and taking the coordinates as the reference coordinates of the mark points;
the third acquisition unit is used for acquiring the coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
and the calculating unit is used for calculating to obtain the relative coordinate of the cursor reference point projected to the display area by the cursor reference point according to the mark point acquisition coordinate, the mark point reference coordinate and the cursor reference point acquisition coordinate based on a perspective transformation rule, wherein the relative coordinate of the cursor reference point is used for determining the pixel coordinate of the cursor in the display area.
9. A screen control system, comprising:
the handheld cursor control device comprises a camera, a communication module and a key;
the system control end of the display device can be in communication connection with the communication module of the handheld cursor control equipment, the display device is provided with at least four display area mark points for marking the display area position of the display device, and the connecting lines of the four display area mark points form a quadrangle;
a processing module comprising a receiving unit, a first obtaining unit, a second obtaining unit, a third obtaining unit, and a calculating unit, wherein,
the receiving unit is used for receiving a current frame image which is shot by the camera and comprises a display area image, determining a cursor reference point in the current frame image and identifying the at least four display area mark points in the current frame image;
the first acquisition unit is used for constructing a first reference coordinate system by using the current frame image, acquiring the coordinates of at least four display area mark points in the display area image in the first reference coordinate system, and taking the coordinates as mark point acquisition coordinates;
the second acquisition unit is used for constructing a second reference coordinate system by using the display area, acquiring the coordinates of the mark points of the display area in the second reference coordinate system and taking the coordinates as the reference coordinates of the mark points;
the third acquisition unit is used for acquiring the coordinates of the cursor reference point in the current frame image in the first reference coordinate system as cursor reference point acquisition coordinates;
the computing unit is used for computing relative coordinates of a cursor reference point projected to the display area by the cursor reference point according to the mark point acquisition coordinates, the mark point reference coordinates and the cursor reference point acquisition coordinates based on a perspective transformation rule, and the relative coordinates of the cursor reference point are used for determining pixel coordinates of the cursor in the display area;
the processing module is arranged in the hand-held cursor control device, or in the system control end, or one part is arranged in the hand-held cursor control device, and the other part is arranged in the system control end.
10. The screen control system according to claim 9, wherein the display device is rectangular, infrared emitting elements are disposed at four corners of the display area, the cameras are infrared cameras, and the position of the image of the display area in the current frame image is determined by infrared light emitted from the infrared emitting elements and captured by the infrared cameras.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011404093.6A CN112419412A (en) | 2020-12-04 | 2020-12-04 | Cursor positioning method, handheld cursor control equipment and screen control system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011404093.6A CN112419412A (en) | 2020-12-04 | 2020-12-04 | Cursor positioning method, handheld cursor control equipment and screen control system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112419412A true CN112419412A (en) | 2021-02-26 |
Family
ID=74830087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011404093.6A Pending CN112419412A (en) | 2020-12-04 | 2020-12-04 | Cursor positioning method, handheld cursor control equipment and screen control system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112419412A (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106527762A (en) * | 2016-11-10 | 2017-03-22 | 深圳市鹰眼在线电子科技有限公司 | Cursor coordinate determining method, cursor coordinate determining device and mouse control system |
-
2020
- 2020-12-04 CN CN202011404093.6A patent/CN112419412A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106527762A (en) * | 2016-11-10 | 2017-03-22 | 深圳市鹰眼在线电子科技有限公司 | Cursor coordinate determining method, cursor coordinate determining device and mouse control system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7733404B2 (en) | Fast imaging system calibration | |
| JP5740884B2 (en) | AR navigation for repeated shooting and system, method and program for difference extraction | |
| US7161596B2 (en) | Display location calculation means | |
| CN110111388B (en) | Three-dimensional object pose parameter estimation method and visual equipment | |
| WO2021136386A1 (en) | Data processing method, terminal, and server | |
| JP4710734B2 (en) | Pointing device, pointing position detection method, and program | |
| CN112556685B (en) | Navigation route display method and device, storage medium and electronic equipment | |
| CN110619662A (en) | Monocular vision-based multi-pedestrian target space continuous positioning method and system | |
| CN110737414B (en) | Interactive display method, device, terminal equipment and storage medium | |
| JP2020178248A (en) | Projection control device, projection control method, projection system, program, and storage medium | |
| JP2006352495A (en) | Remote instruction system | |
| WO2020024845A1 (en) | Positioning method and apparatus | |
| JP5200800B2 (en) | Imaging apparatus and imaging system | |
| JP5152281B2 (en) | Image processing apparatus, method, and program | |
| CN112073640B (en) | Panoramic information acquisition pose acquisition method, device and system | |
| WO2024055531A1 (en) | Illuminometer value identification method, electronic device, and storage medium | |
| JP2003036142A (en) | System and method for presentation | |
| CN112419412A (en) | Cursor positioning method, handheld cursor control equipment and screen control system | |
| CN110533577B (en) | Fisheye image correction method and device | |
| US20180061135A1 (en) | Image display apparatus and image display method | |
| WO2018195973A1 (en) | Method for positioning cursor using handheld terminal, handheld terminal, and electronic device | |
| CN111581322B (en) | Method, device and equipment for displaying region of interest in video in map window | |
| JP2008123090A (en) | Camera pointer device, labeling method and program | |
| CN114466143A (en) | Shooting angle calibration method and device, terminal equipment and storage medium | |
| CN112308924B (en) | Method, device, equipment and storage medium for calibrating camera in augmented reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |