CN113822795B - Virtual key projection method and device based on millimeter wave radar and electronic equipment - Google Patents
Virtual key projection method and device based on millimeter wave radar and electronic equipment Download PDFInfo
- Publication number
- CN113822795B CN113822795B CN202111095410.5A CN202111095410A CN113822795B CN 113822795 B CN113822795 B CN 113822795B CN 202111095410 A CN202111095410 A CN 202111095410A CN 113822795 B CN113822795 B CN 113822795B
- Authority
- CN
- China
- Prior art keywords
- key
- operation target
- projection
- millimeter wave
- wave radar
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000012937 correction Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 9
- 230000009471 action Effects 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 abstract description 9
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 241001122767 Theaceae Species 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
Classifications
-
- G06T3/08—
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/06—Systems determining position data of a target
- G01S13/42—Simultaneous measurement of distance and other co-ordinates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/14—Fourier, Walsh or analogous domain transformations, e.g. Laplace, Hilbert, Karhunen-Loeve, transforms
- G06F17/141—Discrete Fourier transforms
- G06F17/142—Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Abstract
The embodiment of the invention provides a virtual key projection method and device based on millimeter wave radar, a storage medium and electronic equipment. The key projection is generated, key information corresponding to the operation target is determined based on the distance and the angle between the operation target and the electronic equipment, which are acquired by the millimeter wave radar, and finally corresponding control is performed according to the determined key information. The embodiment of the application replaces the entity keys of the traditional electronic equipment, ensures the operation accuracy of the generated virtual keys, is beneficial to the integrated production of the electronic equipment, reduces the production and operation costs, and simultaneously reduces the use cost of a user.
Description
Technical Field
The application relates to the technical field of projection, in particular to a virtual key projection method and device based on millimeter wave radar, a storage medium and electronic equipment.
Background
In the existing electronic products, various entity keys, such as remote controllers of televisions, computer keyboards, mouse keys and the like, often exist as accessories of main products, and additional manufacturing cost is required, so that the integrated design of the products is hindered. In addition, the additional key fittings often cause additional consumption and inconvenience to users, such as use cost caused by replacement of the battery, replacement cost caused by loss of the fittings, and the like.
Disclosure of Invention
The invention aims to provide a virtual key projection method and device based on millimeter wave radar, a storage medium and electronic equipment, so as to solve the problems.
In order to achieve the above object, the embodiments of the present invention are realized as follows:
in a first aspect, an embodiment of the present invention provides a virtual key projection method based on millimeter wave radar, which is applied to an electronic device, and the method includes:
generating a key projection;
acquiring the distance and the angle between an operation target and electronic equipment when a user operates on key projection through a millimeter wave radar;
determining key information corresponding to the operation target through the distance and the angle;
and responding to the key information to perform corresponding control.
Further, the method further comprises:
displaying a guide key;
acquiring the distance and the angle between an operation target and the electronic equipment by the millimeter wave radar when a user operates a corresponding key on key projection under the information guidance of the guide key;
determining the position information of the operation target according to the distance and the angle;
and associating the key projection area corresponding to the position information with the guide key.
Further, the method further comprises:
acquiring the distance and the angle between an operation target and the electronic equipment when a user operates a corresponding key on key projection under the guidance of a preset correction sequence through the millimeter wave radar;
determining the position information of the operation target according to the distance and the angle;
and associating the key projection area corresponding to the position information with a preset key.
Further, the step of acquiring the distance and the angle between the operation target and the electronic device when the user operates on the key projection through the millimeter wave radar further comprises: identifying action information of an operation target when a user operates on the key projection;
the step of responding to the key information for corresponding control comprises the following steps: and performing corresponding control according to the operation information indicated by the action information of the operation target and the key information corresponding to the operation target.
Further, the step of generating a key projection includes:
identifying gesture information of a user;
and generating key projections when the gesture information meets preset gesture information.
Further, the method further comprises:
judging whether a calibration record exists or not;
when a calibration record is present, the calibration record is imported.
In a second aspect, an embodiment of the present invention provides a virtual key projection device based on millimeter wave radar, which is applied to an electronic device, and the device includes:
a projection generation module configured to generate a key projection;
the acquiring module is configured to acquire the distance and the angle between the operation target and the electronic equipment when the user operates on the key projection through the millimeter wave radar;
the key determining module is configured to determine key information corresponding to the operation target through the distance and the angle;
and the execution module is configured to respond to the key information to perform corresponding control.
Further, the device also comprises a correction module, which is configured to display the guide key, acquire the distance and the angle between the operation target and the electronic equipment when the user operates the corresponding key on the key projection under the information guide of the guide key through the millimeter wave radar, determine the position information of the operation target according to the distance and the angle, and correlate the key projection area corresponding to the position information with the guide key; or the device is configured to acquire the distance and the angle between the operation target and the electronic equipment when the user operates the corresponding key on the key projection under the guidance of the preset correction sequence through the millimeter wave radar, determine the position information of the operation target according to the distance and the angle, and correlate the key projection area corresponding to the position information with the preset key.
In a third aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon a computer program which when loaded by a processor performs steps in a method as provided in the first aspect of embodiments of the present invention.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, where the memory stores a computer program, and the processor is configured to execute steps in the virtual key projection method based on millimeter wave radar according to the first aspect of the present invention by loading the computer program.
According to the virtual key projection method based on the millimeter wave radar, key projection is generated, key information corresponding to an operation target is determined according to the distance and the angle between the operation target and the electronic equipment, which are acquired by the millimeter wave radar, and finally corresponding control is performed according to the determined key information, so that the physical keys of the traditional electronic equipment are replaced, the operation accuracy of the generated virtual keys is ensured, the integrated production of the electronic equipment is facilitated, the production and operation costs are reduced, and meanwhile, the use cost of a user is reduced.
In the following description, other features will be partially set forth. Upon review of the ensuing disclosure and the accompanying figures, those skilled in the art will in part discover these features or will be able to ascertain them through production or use thereof. The features of the present application may be implemented and obtained by practicing or using the various aspects of the methods, tools, and combinations that are set forth in the detailed examples described below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered limiting the scope, and that other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
The methods, systems, and/or programs in the accompanying drawings will be described further in terms of exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein reference numerals represent similar mechanisms throughout the several views of the drawings.
Fig. 1 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a frame of an electronic device according to an embodiment of the present invention.
Fig. 3 is a flowchart of a virtual key projection method based on millimeter wave radar according to an embodiment of the present application.
Fig. 4 is a schematic diagram of an architecture of a virtual key projection device based on millimeter wave radar according to an embodiment of the present invention.
Detailed Description
In order to better understand the technical solutions described above, the following detailed description of the technical solutions of the present application is provided through the accompanying drawings and specific embodiments, and it should be understood that the specific features of the embodiments and embodiments of the present application are detailed descriptions of the technical solutions of the present application, and not limit the technical solutions of the present application, and the technical features of the embodiments and embodiments of the present application may be combined with each other without conflict.
In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it will be apparent to one skilled in the art that the present application may be practiced without these details. In other instances, well-known methods, procedures, systems, components, and/or circuits have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present application.
These and other features, together with the functions, acts, and combinations of parts and economies of manufacture of the related elements of structure, all of which form part of this application, may become more apparent upon consideration of the following description with reference to the accompanying drawings. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the application. It should be understood that the drawings are not to scale. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the application. It should be understood that the figures are not to scale.
The flowcharts are used in this application to describe implementations performed by systems according to embodiments of the present application. It should be clearly understood that the execution of the flowcharts may be performed out of order. Rather, these implementations may be performed in reverse order or concurrently. Additionally, at least one other execution may be added to the flowchart. One or more of the executions may be deleted from the flowchart.
The virtual key projection method based on millimeter wave radar provided by the embodiment of the invention is applied to the electronic equipment 100, and the electronic equipment 100 is any electronic equipment which needs to be controlled in the traditional sense and has keys, such as a television, a computer, a mouse and the like. Referring to fig. 1, a schematic diagram of an electronic device 100 according to an embodiment of the invention is provided, and compared with a conventional electronic device, the electronic device 100 omits physical keys, and the electronic device 100 includes a projection generating device 140 and a millimeter wave radar 150.
The projection generating means 140 can generate a projection image, such as a key projection image, on a plane by a projection imaging technique. The millimeter wave radar 150 is a special radar capable of using short wavelength electromagnetic waves, and can emit signals having a wavelength on the order of millimeters. The millimeter wave radar 150 is capable of distance measurement, speed measurement and angle detection, and the projection imaging technology and the millimeter wave radar technology are currently mature prior technologies, and the principles thereof are not repeated here.
Through the projection generating device 140 and the millimeter wave radar 150, the electronic device 100 may generate a virtual key by projection on a plane, and then acquire a distance between an operation target of the user and the electronic device 100 when the user operates on the virtual key. For example, the electronic device 100 is a television, the projection generating apparatus 140 projects a virtual key on a plane such as a wall, a tea table, a floor, etc., the user clicks the virtual key by a finger, and the millimeter wave radar 150 detects the distance and angle of the user's finger, thereby determining the key type, and the electronic device 100 performs control corresponding to the key.
The implementation of this process is described in detail below.
Fig. 2 is a block diagram of an electronic device 100 according to an embodiment of the invention, where the electronic device 100 includes a virtual key projection device 110, a memory 120, and a processor 130. The memory 120 and the processor 130 are electrically connected to each other, either directly or indirectly, to enable transmission or interaction of data. For example, these components may be electrically connected to each other via one or more communication buses or signal lines, and virtual key projection device 110 includes at least one software function module that may be stored in memory 120 in the form of software or firmware (firmware) or cured in an Operating System (OS) of electronic device 100. The processor 130 is configured to execute executable modules stored in the memory 120, such as software functional modules and computer programs included in the virtual key projection device 110.
The Memory 120 may be, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), an electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc. The memory 120 is used for storing a program, and the processor 130 executes the program after receiving an execution instruction.
The processor may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also Digital Signal Processors (DSPs)), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It is to be understood that the configuration shown in fig. 2 is merely illustrative, and that the control system of the electronic device 100 may also include more or fewer components than shown in fig. 2, or have a different configuration than shown in fig. 2. The components shown in fig. 2 may be implemented in hardware, software, or a combination thereof.
Fig. 3 is a flowchart of a virtual key projection method based on millimeter wave radar according to some embodiments of the present application, and the method is applied to the virtual key projection device 110 in fig. 2, and may specifically include the following steps S1 to S4. Some alternative embodiments will be described on the basis of the following steps S1-S4, which should be understood as examples and should not be interpreted as essential features for implementing the present solution.
And S1, generating key projections.
In this embodiment, the key projection is a virtual key generated by the projection generating device 140 through a projection technology, and the key projection may be projected at any position where the user can easily operate, for example, a wall panel, a ground surface, or a tea table. In some embodiments, the key projection is not always maintained and is only generated when needed, for example, the user may wake up by a specific gesture, the electronic device 100 identifies gesture information of the user, and when the gesture information satisfies preset gesture information, the key projection is generated, and the preset gesture information is gesture information set by the user, for example, slides to the right. The gesture recognition process may be implemented based on the millimeter wave radar 150 of the electronic device 100, for example, a set of chirp signals transmitted by the millimeter wave radar 150, and perform a two-dimensional fast fourier transform (2D-FFT) on the data received by each receiving antenna, so as to obtain a two-dimensional grid including a distance dimension and a doppler dimension. Peaks in the grid indicate that an object is present at a distance. The angle of arrival of the target is obtained by processing the corresponding peaks in the grid from the multiple receive antennas, i.e., the angle FFT. And then carrying out algorithm calculation of gesture recognition, carrying out 2D-FFT operation on ADC data of each receiving channel to obtain a 2D-FFT matrix containing distance and Doppler dimensions, carrying out incoherent accumulation on the 2D-FFT matrices of a plurality of receiving channels to obtain a distance Doppler heat map, carrying out feature extraction in the heat map, generating a time sequence for each feature, classifying the extracted features in a specific time window through a machine learning algorithm, and finally outputting corresponding gesture types.
And S2, acquiring the distance and the angle between the operation target and the electronic equipment when a user operates on the key projection through the millimeter wave radar.
The key projection displays a plurality of virtual keys, and different virtual keys correspond to different operation information, such as next, volume up, homepage return, etc., and no matter how the user operates on the key projection, the electronic device 100 needs to determine which key corresponds to the current operation of the user, so as to execute the operation information corresponding to the key.
In the present embodiment, the distance and angle between the operation target and the electronic apparatus 100 at the time of user operation are acquired by the millimeter wave radar 150. The operation target is a target used when a user operates key projection, such as a finger, a pen and the like. The millimeter wave radar 150 detects the position of the operation target on the key projection, and acquires the corresponding distance and angle.
Millimeter-wave radar 150 may include a combiner, a transmit antenna, a receive antenna, and a mixer. When detecting the distance and angle of an operation target, the synthesizer generates a linear frequency modulation pulse, the linear frequency modulation pulse is transmitted through the transmitting antenna, the electromagnetic signal is blocked by an object on a transmitting path in the transmitting process, the receiving antenna captures the reflected signal, the mixer combines the signals of the transmitting antenna and the receiving antenna to generate an intermediate frequency signal with new frequency, and therefore the delay of the signal is obtained, and the distance of a measured object can be obtained. When measuring angles, at least 2 receiving antennas are needed, and the difference between the distances between the object and the two receiving antennas can cause the FFT peak to generate phase change, and the phase change can help to calculate the angle of the transmitted signal of the measured object. And S3, determining key information corresponding to the operation target through the distance and the angle.
By the distance and the angle, the electronic device 100 can be assisted to accurately locate the position of the operation target, which corresponds to the key information, and thus the operation target corresponds to the key information. For example, after the electronic device 100 calculates according to the distance and the angle, it determines that the position corresponding to the operation target is a prestored stop key, and the key information corresponding to the operation target is stop.
In some prior arts, an infrared laser is used to form a light spot on an operation target, then an image sensor detects the emitted laser, captures an image of a projection keyboard, and then processes and analyzes the image to obtain a virtual key currently operated by a user. In the prior art, an image is acquired by means of a sensor and is analyzed to infer the operation position of a user, and because the captured image is limited by resolution and analysis efficiency, the inferred speed and accuracy are not ideal, the virtual key projection method based on the millimeter wave radar provided by the embodiment of the application can directly position key information corresponding to an operation target by acquiring the distance and angle of the operation target of the user, does not need an additional camera and image analysis, has high analysis efficiency, saves cost, and the accuracy of millimeter wave measurement is also the accuracy of key positioning brought by the virtual key projection method of the embodiment of the application, so that fine actions can be identified.
And S4, responding to the key information to perform corresponding control.
It should be noted that, before using the key projection, each virtual key on the key projection needs to be accurately positioned, so that the corresponding position of each virtual key is associated with the operation information corresponding to the virtual key, and the electronic device 100 will not perform erroneous judgment when the user clicks the corresponding position. In addition, when the position of the electronic device 100 is changed, for example, moved, the position of the key projection projected by the projection generating device 140 of the electronic device 100 may also be changed, and the distance between the position of the key projection and the electronic device 100 may be changed, so when the user operates the key projection, the distance and the angle acquired by the millimeter wave radar 150 may deviate, and misoperation may occur to the calculated key information, so that correction is required, so that each virtual key in the key projection area can accurately correspond to the key information.
In some embodiments, when the user operates on the key projection, double-click, continuous-click, sliding or the like may be used, in this scenario, when the user operates on the key projection through the millimeter wave radar, it is necessary to obtain not only the distance and angle between the operation target and the electronic device, but also the motion information of the operation target when the user operates on the key projection, and the manner of identifying the motion information may refer to the above-mentioned gesture identification process through the millimeter wave radar. After the action information is identified, corresponding control is performed according to the operation information indicated by the action information of the operation target and the key information corresponding to the operation target. For example, when it is recognized that the operation target is double-clicked and the operation target corresponds to the key information to increase the volume, the corresponding control is to control the volume to increase by two. Or, if the action information of the operation target is identified as double-click and the key information corresponding to the operation target is fast-forward, the corresponding control is to control the video to fast forward by 4 times.
In the embodiment of the present invention, the correction of the key projection may include two ways, one is that the electronic device 100 guides the user, the user performs interactive correction according to the guidance, and the other is that the user directly performs correction according to a predetermined correction sequence.
For interactive correction, the following steps may be included:
step S5a, displaying a guide key.
The electronic device 100 first displays a guide key on the screen, guides the user to click a corresponding key on the key projection, for example, the guide key is displayed on the screen as a "back" key, the key projection includes the "back" key, and the user clicks the "back" key on the key projection.
Step S6a, acquiring the distance and angle between the operation target and the electronic equipment when the user operates the corresponding key on the key projection under the information guidance of the guide key through the millimeter wave radar.
When the user clicks, the millimeter wave radar 150 acquires the distance and angle between the operation target and the electronic device 100, which is easily understood, that is, the distance and angle between the "return" key on the key projection and the electronic device 100.
Step S7a, position information of the operation target is determined according to the distance and the angle.
In step S8a, the key projection area corresponding to the position information is associated with the guide key.
The steps S5 a-S8 a are repeated until all keys are corrected. After the association is completed, when the user clicks the position corresponding to the position information again, the electronic device 100 performs operation control corresponding to the key information.
For another correction mode, the following steps may be included:
step S5b, acquiring the distance and the angle between the operation target and the electronic equipment when the user operates the corresponding key on the key projection under the guidance of the preset correction sequence through the millimeter wave radar.
The preset correction sequence is a correction sequence that informs the user in advance that the user needs to operate sequentially on the key projection, for example, in the instruction sheet of the electronic device 100, the virtual keys on the key projection are clicked sequentially from top to bottom, from left to right during the correction period.
Step S6b, determining the position information of the operation target according to the distance and the angle.
Step S7b, associating the key projection area corresponding to the position information with a preset key.
Compared with the interactive correction method, the correction method has higher correction efficiency, but is easy to make mistakes, and the embodiment is not limited by the specific correction method.
Before the electronic device 100 generates the key projection, it may be determined whether or not there is a calibration record, and if there is a calibration record, if there is a change in the position of the millimeter wave radar 150 or the projection, the calibration is not required again, and the previous calibration record may be directly used as the key projection record.
In summary, according to the virtual key projection method based on the millimeter wave radar provided by the embodiment of the invention, key projection is generated, key information corresponding to an operation target is determined based on the distance and the angle between the operation target and the electronic device acquired by the millimeter wave radar, and finally corresponding control is performed according to the determined key information. The physical keys of the traditional electronic equipment are replaced, the operation accuracy of the generated virtual keys is guaranteed, the integrated production of the electronic equipment is facilitated, the production and operation costs are reduced, and the use cost of a user is reduced.
Fig. 4 is a schematic diagram of an architecture of a virtual key projection device 110 according to an embodiment of the present invention, where the virtual key projection device 110 may be used to execute a virtual key projection method based on millimeter wave radar, and the virtual key projection device 110 includes:
a projection generation module 111 configured to generate a key projection;
an acquisition module 112 configured to acquire, by means of a millimeter wave radar, a distance and an angle between an operation target and the electronic device when a user operates on the key projection;
a key determination module 113 configured to determine key information corresponding to the operation target by a distance and an angle;
the execution module 114 is configured to perform corresponding control in response to the key information.
The projection generation module 111 may be used to perform step S1, the acquisition module 112 may be used to perform step S2, the key determination module 113 may be used to perform step S3, and the execution module 114 may be used to perform step S4.
Since in the above embodiments, the method for projecting a virtual key based on a millimeter wave radar provided in the embodiments of the present invention has been described in detail, and the principle of the virtual key projection device 110 is the same as that of the method, the execution principle of each module of the virtual key projection device 110 will not be described in detail here.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners as well. The apparatus embodiments described above are merely illustrative, for example, of the flowcharts and block diagrams in the figures that illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Furthermore, the order in which the processing elements and sequences are described, the use of numerical letters, or other designations are used is not intended to limit the order in which the processes and methods of the present application are performed, unless specifically indicated in the claims. While in the foregoing disclosure there has been discussed, by way of various examples, some embodiments of the invention which are presently considered to be useful, it is to be understood that this detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements that are within the spirit and scope of the embodiments of this application. For example, while the system components described above may be implemented by hardware devices, they may also be implemented solely by software solutions, such as installing the described system on an existing server or mobile device.
It should also be appreciated that in the foregoing description of the embodiments of the present application, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of at least one of the embodiments of the invention. This method of disclosure, however, is not intended to imply that more features than are presented in the claims are required for the subject application. Indeed, less than all of the features of a single embodiment disclosed above.
Claims (9)
1. The virtual key projection method based on the millimeter wave radar is applied to electronic equipment and is characterized by comprising the following steps of:
generating a key projection, the key projection being generated on an operable entity;
displaying a guide key on a screen of the electronic equipment;
acquiring the distance and angle between an operation target and the electronic equipment when a user operates a corresponding key on the key projection under the information guidance of the guide key through a millimeter wave radar;
determining position information of the operation target according to the distance and the angle;
associating a key projection area corresponding to the position information with the guide key;
acquiring the distance and the angle between an operation target and the electronic equipment when a user operates on the key projection through a millimeter wave radar based on the association;
determining key information corresponding to the operation target according to the distance and the angle obtained after the association;
and responding to the key information to perform corresponding control.
2. The method according to claim 1, wherein the method further comprises:
acquiring the distance and the angle between an operation target and the electronic equipment when a user operates a corresponding key on key projection under the guidance of a preset correction sequence through the millimeter wave radar;
determining the position information of the operation target according to the distance and the angle;
and associating the key projection area corresponding to the position information with a preset key.
3. The method according to any one of claims 1-2, wherein the acquiring, by the millimeter wave radar, the distance and the angle between the operation target and the electronic device when the user operates on the key projection, further comprises:
identifying action information of an operation target when a user operates on the key projection;
the response to the key information carries out corresponding control, which comprises the following steps:
and corresponding control is carried out according to the operation information indicated by the action information of the operation target and the key information corresponding to the operation target.
4. A method according to claim 3, wherein generating a key projection comprises:
identifying gesture information of a user;
and generating key projection when the gesture information meets preset gesture information.
5. The method according to claim 4, wherein the method further comprises:
judging whether a calibration record exists or not;
when a calibration record is present, the calibration record is imported.
6. A virtual key projection device based on millimeter wave radar, applied to an electronic device, characterized in that the device comprises:
a projection generation module configured to generate a key projection, the key projection generated on an operable entity;
the correction module is configured to display a guide key on a screen of the electronic equipment, acquire a distance and an angle between an operation target and the electronic equipment when a user operates a corresponding key on key projection under the information guide of the guide key through the millimeter wave radar, determine the position information of the operation target according to the distance and the angle, and associate a key projection area corresponding to the position information with the guide key;
the acquiring module is configured to acquire the distance and the angle between an operation target and the electronic equipment when a user operates on the key projection through the millimeter wave radar;
the key determining module is configured to determine key information corresponding to the operation target through the distance and the angle;
and the execution module is configured to respond to the key information to perform corresponding control.
7. The apparatus of claim 6, wherein the correction module further comprises:
the method comprises the steps of acquiring distance and angle between an operation target and electronic equipment when a user operates a corresponding key on key projection under the guidance of a preset correction sequence through the millimeter wave radar, determining position information of the operation target according to the distance and the angle, and associating a key projection area corresponding to the position information with a preset key.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the steps in the millimeter wave radar based virtual key projection method according to any one of claims 1-5 are performed when the computer program is loaded by a processor.
9. An electronic device comprising a processor and a memory, the memory storing a computer program, the processor being configured to perform the steps in the millimeter wave radar-based virtual key projection method according to any one of claims 1 to 5 by loading the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111095410.5A CN113822795B (en) | 2021-09-17 | 2021-09-17 | Virtual key projection method and device based on millimeter wave radar and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111095410.5A CN113822795B (en) | 2021-09-17 | 2021-09-17 | Virtual key projection method and device based on millimeter wave radar and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113822795A CN113822795A (en) | 2021-12-21 |
| CN113822795B true CN113822795B (en) | 2024-02-09 |
Family
ID=78914877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111095410.5A Active CN113822795B (en) | 2021-09-17 | 2021-09-17 | Virtual key projection method and device based on millimeter wave radar and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113822795B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101685342A (en) * | 2008-09-26 | 2010-03-31 | 联想(北京)有限公司 | Method and device for realizing dynamic virtual keyboard |
| WO2017001578A1 (en) * | 2015-06-30 | 2017-01-05 | Softathome | Virtual remote control |
| CN106980362A (en) * | 2016-10-09 | 2017-07-25 | 阿里巴巴集团控股有限公司 | Input method and device based on virtual reality scenario |
| CN107025019A (en) * | 2017-01-12 | 2017-08-08 | 瑞声科技(新加坡)有限公司 | The exchange method and terminal device of virtual key |
| CN107967058A (en) * | 2017-12-07 | 2018-04-27 | 联想(北京)有限公司 | Information processing method, electronic equipment and computer-readable recording medium |
| CN111028494A (en) * | 2019-11-04 | 2020-04-17 | 珠海格力电器股份有限公司 | Virtual remote control method of electrical equipment, computer readable storage medium and intelligent household appliance |
| CN111301311A (en) * | 2020-02-27 | 2020-06-19 | 科世达(上海)机电有限公司 | Steering wheel button based on camera |
| CN112034446A (en) * | 2020-08-27 | 2020-12-04 | 南京邮电大学 | Gesture recognition system based on millimeter wave radar |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9921657B2 (en) * | 2014-03-28 | 2018-03-20 | Intel Corporation | Radar-based gesture recognition |
| CN106959814A (en) * | 2017-03-27 | 2017-07-18 | 联想(北京)有限公司 | A kind of display methods of dummy keyboard, device and terminal |
-
2021
- 2021-09-17 CN CN202111095410.5A patent/CN113822795B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101685342A (en) * | 2008-09-26 | 2010-03-31 | 联想(北京)有限公司 | Method and device for realizing dynamic virtual keyboard |
| WO2017001578A1 (en) * | 2015-06-30 | 2017-01-05 | Softathome | Virtual remote control |
| CN106980362A (en) * | 2016-10-09 | 2017-07-25 | 阿里巴巴集团控股有限公司 | Input method and device based on virtual reality scenario |
| CN107025019A (en) * | 2017-01-12 | 2017-08-08 | 瑞声科技(新加坡)有限公司 | The exchange method and terminal device of virtual key |
| CN107967058A (en) * | 2017-12-07 | 2018-04-27 | 联想(北京)有限公司 | Information processing method, electronic equipment and computer-readable recording medium |
| CN111028494A (en) * | 2019-11-04 | 2020-04-17 | 珠海格力电器股份有限公司 | Virtual remote control method of electrical equipment, computer readable storage medium and intelligent household appliance |
| CN111301311A (en) * | 2020-02-27 | 2020-06-19 | 科世达(上海)机电有限公司 | Steering wheel button based on camera |
| CN112034446A (en) * | 2020-08-27 | 2020-12-04 | 南京邮电大学 | Gesture recognition system based on millimeter wave radar |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113822795A (en) | 2021-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210287019A1 (en) | Method and apparatus for detecting obstacle, electronic device and storage medium | |
| US9811168B2 (en) | Apparatus for performing gesture recognition and control based on ultrasonic positioning | |
| CN101387931B (en) | Infrared touch screen multi-point recognizing method | |
| US8914254B2 (en) | Latency measurement | |
| CN111949111B (en) | Interaction control method and device, electronic equipment and storage medium | |
| US9122345B2 (en) | Method of determining touch gesture and touch control system | |
| CN108595029B (en) | Method, device and equipment for determining inclination angle of capacitive pen and storage medium | |
| US20060034486A1 (en) | Passive touch system and method of detecting user input | |
| US20100164869A1 (en) | Virtual keyboard structure of electric device and data inputting method thereof | |
| WO2007098697A1 (en) | Remote control, imaging device, method and system for the same | |
| CN109584375A (en) | A kind of object information display methods and mobile terminal | |
| CN113822795B (en) | Virtual key projection method and device based on millimeter wave radar and electronic equipment | |
| US9772725B2 (en) | Hybrid sensing to reduce latency | |
| US20110279412A1 (en) | Scanning method for determining a touch position of a touch input apparatus | |
| US8354996B2 (en) | Electronic reading apparatus and input method thereof | |
| CN112783406A (en) | Operation execution method and device and electronic equipment | |
| CN110213407B (en) | Electronic device, operation method thereof and computer storage medium | |
| US20130016069A1 (en) | Optical imaging device and imaging processing method for optical imaging device | |
| US9652081B2 (en) | Optical touch system, method of touch detection, and computer program product | |
| US20220237807A1 (en) | Interacting Multi-Model Tracking Algorithm using Rest State Model | |
| US11946996B2 (en) | Ultra-accurate object tracking using radar in multi-object environment | |
| CN105718121A (en) | Optical touch device | |
| US20230168744A1 (en) | Information processing apparatus and information processing method based on input operation by user, and computer program for executing the method | |
| US20200401271A1 (en) | Operation detection device and operation detection method | |
| CN110377186B (en) | Virtual keyboard generation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |