WO2020073245A1 - 手势识别方法、vr视角控制方法以及vr系统 - Google Patents

手势识别方法、vr视角控制方法以及vr系统 Download PDF

Info

Publication number
WO2020073245A1
WO2020073245A1 PCT/CN2018/109698 CN2018109698W WO2020073245A1 WO 2020073245 A1 WO2020073245 A1 WO 2020073245A1 CN 2018109698 W CN2018109698 W CN 2018109698W WO 2020073245 A1 WO2020073245 A1 WO 2020073245A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
point
gesture recognition
recognition method
edge
Prior art date
Application number
PCT/CN2018/109698
Other languages
English (en)
French (fr)
Inventor
郑欣
Original Assignee
深圳市道通智能航空技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市道通智能航空技术有限公司 filed Critical 深圳市道通智能航空技术有限公司
Priority to PCT/CN2018/109698 priority Critical patent/WO2020073245A1/zh
Priority to CN201880098482.5A priority patent/CN113039550B/zh
Publication of WO2020073245A1 publication Critical patent/WO2020073245A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • the invention relates to the field of virtual reality technology, in particular to a gesture recognition method, a VR perspective control method and a VR system.
  • Virtual Reality is a technology that uses related equipment to generate a highly simulated simulation environment and immerses users in the simulated environment through three-dimensional interaction and simulation to obtain an excellent user experience.
  • Typical virtual reality is achieved through VR glasses or similar devices worn on the user's head. These VR glasses require a wrap-around cover to fit on the user's head. Therefore, when wearing VR glasses, it is inconvenient for the user to adjust the VR field of view through traditional operation methods such as a remote control.
  • gesture operation may be a better control operation method that can meet the needs of use.
  • the process of gesture detection and recognition requires a very large amount of calculation, which makes the application of gesture operations very limited. How to effectively simplify the complexity of gesture recognition algorithms is an urgent problem to be solved.
  • embodiments of the present invention provide a gesture recognition method, a VR perspective control method, and a VR system that can reduce the amount of calculation required for gesture detection.
  • the gesture recognition method includes: acquiring depth information; acquiring spatial point cloud information according to the depth information; determining a target area in the spatial point cloud information, the target area refers to an area containing hand point cloud information; generating and A planar image corresponding to the target area; extracting edge points of the hand in the planar image; determining the number of fingers in the hand according to the edge points of the hand; determining the number of fingers according to the number of fingers Hand gestures.
  • the acquiring the depth information includes: acquiring the depth information through a depth sensor.
  • the determining the target area in the spatial point cloud information includes extracting point cloud information within a preset distance as the target area.
  • the method further includes: filtering noise in the target area.
  • the filtering of noise in the target area includes filtering the noise in the target area through a maximum connected domain algorithm.
  • the generating the plane image corresponding to the target area includes: mapping point cloud information in the target area to a two-dimensional space to generate the plane image corresponding to the target area.
  • the extracting the edge points of the hand in the planar image includes: extracting the edge points of the hand in the planar image by using the Moore Neighborhood method.
  • the determining the number of fingers in the hand according to the edge point of the hand includes: finding a convex hull (Convec Hull) according to the edge point of the hand Point; determine that the convex hull point is the fingertip of the finger; according to the number of the fingertip, determine the number of the fingers in the hand.
  • Convec Hull convex hull
  • the finding the convex hull point according to the edge point of the hand includes: using a Graham's scan method to find the convex hull point.
  • the determining that the convex hull point is the fingertip of the finger includes: selecting, from the edge points of the hand, respectively located on both sides of the convex hull point and The first edge point and the second edge point adjacent to the convex hull point to calculate the straight line connecting the convex hull point and the first edge point and the convex hull point and the second edge point The angle between the straight lines; wherein the first edge point and the second edge point are on the same finger as the convex hull point, and between the first edge point and the convex hull point, and A predetermined number of edge points spaced between the second edge point and the convex hull point; determining whether the included angle is within the first preset range; if so, determining that the convex hull point is the finger's fingertip.
  • the judging whether the included angle is within the first preset range includes: calculating the included angle:
  • ⁇ is the included angle
  • P i is the convex hull point
  • P l is the first edge point
  • Pr is the second edge point
  • the preset value is between 20 ° and 60 °.
  • 10-50 edge points are spaced between the first edge point and the convex hull point and between the second edge point and the convex hull point.
  • the method further includes: determining whether a hand exists in the planar image.
  • the judging whether there is a hand in the planar image includes: calculating a distance between each of the candidate points and each of the edge points among the candidate points, the candidate points being the edge points Points within the enclosed range; determine the candidate point corresponding to the maximum distance as the palm of the palm of the hand; calculate the angle formed by the connection line between any two adjacent fingertips and the palm of the hand; determine the Whether the included angle is within the second preset range; if so, it is determined that there is a hand in the planar image.
  • the judging whether there is a hand in the planar image includes: calculating a distance between each of the candidate points and each of the edge points among the candidate points, the candidate points being the edge points The points within the enclosing range; determine the candidate point corresponding to the maximum distance to be the palm of the palm of the hand; calculate the sum of the angle formed by the connection between any two adjacent fingertips and the palm of the hand; judge Whether the sum of the included angles exceeds 180 °; if so, it is determined that there is a hand in the planar image.
  • the method further includes: determining whether there is a hand in the planar image.
  • the judging whether there is a hand in the planar image includes: calculating a distance between each of the candidate points and each of the edge points among the candidate points, the candidate points being the edge points A point within the enclosing range; determine the maximum distance to be the radius of the largest inscribed circle of the palm of the hand; determine whether the radius is within the third preset range; if so, determine that the hand exists in the planar image .
  • the method further includes: determining whether there is a hand in the planar image.
  • the judging whether the hand exists in the plane image includes: judging whether the hand exists in the plane image according to the number of the fingers.
  • the embodiments of the present invention also provide the following technical solution: A VR perspective control method.
  • the VR perspective control method includes:
  • adjusting the VR viewing angle according to the fingers of the user's palm includes: recognizing the current user's hand gesture based on the number of the user's hand fingers; adjusting the shooting device through the user's hand gesture The position and orientation of the camera; follow the changes in the position and orientation of the shooting device to change the VR perspective.
  • the method further includes: applying the gesture recognition method described above to determine the palm position of the user's hand; and adjusting the VR perspective according to the palm position of the user's hand.
  • the embodiments of the present invention also provide the following technical solution: A VR system.
  • the VR system includes: a mobile vehicle, a shooting device, a depth sensor, a controller, and a VR display device; both the shooting device and the depth sensor are provided on the mobile vehicle;
  • the VR display device is in communication connection with the shooting device, and is used to generate a corresponding VR scene according to the video image information collected by the shooting device;
  • the controller is used to recognize the user's gesture based on the depth information acquired by the depth sensor using the gesture recognition method as described above, and adjust the VR viewing angle of the VR display device according to the gesture.
  • the controller is specifically configured to: recognize the gesture of the user's hand according to the number of fingers, and control the gesture of the shooting device and the movement of the mobile vehicle according to the gesture.
  • the controller is also used to acquire the palm position of the user using the gesture recognition method as described above, and adjust the VR viewing angle of the VR display device according to the palm position.
  • the controller is specifically configured to determine the relative position between the palm position and the mobile vehicle according to the palm position to control the posture of the shooting device and the movement of the mobile vehicle .
  • the photographing device is mounted on the front of the mobile vehicle, and the depth sensor is mounted on the rear of the mobile vehicle.
  • the mobile vehicle is a drone.
  • the shooting device is mounted on the mobile vehicle through a gimbal.
  • the gesture recognition method can detect and recognize gestures by generating a point cloud and a plane image of the user's hand to extract the edge points of the user's hand, without relying on machine learning, etc.
  • the sample learning algorithm has a simple implementation structure, which can effectively reduce the amount of gesture recognition and ensure a high gesture recognition rate. It can be well applied to platforms with low power consumption, low cost or more sensitive to delay.
  • FIG. 1 is a schematic diagram of an application environment of a VR system provided by an embodiment of the present invention.
  • FIG. 2 is a structural block diagram of a controller provided by an embodiment of the present invention.
  • FIG. 3 is a flowchart of a gesture recognition method provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of fingertip recognition of a finger provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of matching the number of gestures and gesture actions provided by an embodiment of the present invention.
  • FIG. 6 is a flowchart of a gesture recognition method according to another embodiment of the present invention.
  • FIG. 7 is a flowchart of a method for determining whether a hand is present in a planar image provided by an embodiment of the present invention.
  • FIG. 8 is a flowchart of a method for determining whether a hand exists in a planar image according to another embodiment of the present invention.
  • FIG. 1 is an application environment of a VR system provided by an embodiment of the present invention.
  • the application environment includes a mobile vehicle, a controller 20 provided on the mobile vehicle, a VR display device 30, a user 40 and a wireless network 50.
  • the mobile vehicle may be any type of power-driven loading platform, including but not limited to a four-axis unmanned aerial vehicle, a fixed-wing aircraft, and a helicopter model.
  • the mobile vehicle can have the corresponding volume or power according to the needs of the actual situation, thereby providing the load capacity, speed and cruising range that can meet the needs of use.
  • One or more functional modules can also be added to the mobile vehicle, so that the mobile vehicle can realize the corresponding function.
  • the drone 10 will be used as an example.
  • the drone 10 described in this embodiment may include a fuselage, an arm connected to the fuselage, and a power device provided on the arm.
  • the arm may be fixedly connected to the fuselage, integrally formed, or foldable relative to the fuselage.
  • the power unit includes a motor and a propeller connected to the motor. The rotation of the motor shaft of the motor drives the propeller to rotate to provide the pulling force required for the flight of the drone.
  • the drone 10 may also have at least one shooting device 11 for collecting image information.
  • the shooting device 11 may be a high-definition camera, a motion camera, or other types of image acquisition devices.
  • the drone 10 may be equipped with a shooting device 11 through a gimbal or a similar jitter cancellation device.
  • the gimbal allows the shooting device 11 to rotate relative to the drone 10 about at least one axis.
  • the drone 10 may also be provided with a depth sensor 12 for collecting depth information.
  • the depth sensor 12 may specifically be a binocular camera, a TOF camera, a structured light camera, or a laser radar.
  • the photographing device 11 is carried on the front of the drone 10
  • the depth sensor bridge 12 is carried on the rear of the drone.
  • the controller 20 is a control core provided in the mobile vehicle 10, and is used to perform one or more logical judgment steps to realize control of the mobile vehicle 10.
  • the controller 20 may include multiple functional units, such as a flight control unit for controlling the flying attitude of the drone, a target recognition unit for identifying targets, a tracking unit for tracking specific targets, and a navigation for navigating the aircraft Units (for example, GPS (Global Positioning System), Beidou), and data processing units for processing environmental information acquired by related airborne equipment (such as the photographing equipment 11).
  • a flight control unit for controlling the flying attitude of the drone
  • a target recognition unit for identifying targets
  • a tracking unit for tracking specific targets
  • a navigation for navigating the aircraft Units (for example, GPS (Global Positioning System), Beidou)
  • data processing units for processing environmental information acquired by related airborne equipment (such as the photographing equipment 11).
  • the controller 20 may include: a processor 21, a memory 22 and a communication module 25.
  • the processor 21, the memory 22, and the communication module 25 establish a communication connection between any two through a bus.
  • the processor 21 is any type of single-threaded or multi-threaded processor with one or more processing cores, which is used as the core of logical processing and operations, and is used to acquire data, perform logical operation functions, and deliver operation processing results.
  • the memory 22 is a non-volatile computer-readable storage medium, such as at least one magnetic disk storage device, a flash memory device, a distributed storage device remotely disposed relative to the processor 21, or other non-volatile solid-state storage devices.
  • the memory 22 has a program storage area for storing non-volatile software programs, non-volatile computer executable programs, and modules for the processor 21 to call to enable the processor 21 to perform one or more method steps.
  • the memory 22 may also have a data storage area for storing the operation processing result delivered by the processor 21.
  • the communication module 25 is a functional module used by the UAV 10 to establish a communication connection and provide a physical channel, such as a WiFi module and a Bluetooth module to obtain other radio frequency transmission modules.
  • the wireless network 50 may be a wireless communication network for establishing a data transmission channel between two nodes, such as a Bluetooth network, a WiFi network, or a wireless cellular network located in different signal frequency bands.
  • the drone 10 can be added to the wireless network 50 through the communication module 25, and the communication connection with the VR display device 30 can be achieved through the wireless network 50.
  • the VR display device 30 is a device located on the user side and providing a virtual display environment for the user.
  • the VR display device 30 can be of any type, and can realize a combination of one or more devices of VR technology. For example, traditional wrap-around VR glasses, head-mounted VR devices, and augmented reality (AR) devices incorporating VR technology.
  • AR augmented reality
  • the VR display device 30 establishes a communication connection with the shooting device 11 in the drone 10, and can receive video or image information captured by the shooting device 11 and generate a corresponding VR display image accordingly, which is provided to the user for immersion Virtual reality experience.
  • the user 40 is a user wearing the VR display device 30. It uses a VR display device 30 to implement services such as drone 10 flight simulation.
  • the user can control the heading angle of the drone 10 (or the rotation angle of the gimbal) and the position of the drone (such as controlling the forward or backward movement of the drone) to change the viewing angle or display interface of the VR display device.
  • the user 40 is wearing a VR display device. Therefore, it is difficult to control the UAV by controlling the remote controller.
  • the adjustment method of turning the head or moving the body to control the drone is very easy to cause user fatigue and dizziness.
  • a gesture operation control method may be used.
  • the user 40 is wearing the VR display device 30, and when the drone 10 is hovering near the user 40, the user 40 can extend his hand and issue control instructions through different gestures.
  • the UAV 10 uses the depth sensor to collect the relevant depth image at the position facing the user.
  • the controller 20 mounted on the drone 10 can analyze and detect the user's gestures, parse the corresponding control instructions to adjust the operating state of the drone 10 (including the mobile drone 10 Position, changing the orientation of the shooting device 11, the focal length, or changing the angle of rotation of the gimbal) in response to control commands issued by the user.
  • the above operation method of changing the attitude or movement of the drone through gestures to change the VR perspective is more intuitive and convenient. There is no need to move or turn head while operating, which makes user operation more comfortable and user experience better.
  • the drone 10 is used as an example for description.
  • Those skilled in the art can also replace the UAV 10 with any type of mobile vehicle, such as a remote control car, etc., to carry the above-mentioned functional modules, provide a data source for the VR display device 30, and realize an immersive experience of virtual reality.
  • FIG. 3 is a gesture recognition method provided by an embodiment of the present invention. As shown in FIG. 3, the gesture recognition method includes:
  • the depth sensor 12 shown in FIG. 1 may be used to collect and obtain relevant depth information as basic data for gesture recognition.
  • the depth information refers to three-dimensional information that can reflect the subject.
  • the received depth information may also be pre-processed to filter noise in the depth information.
  • Spatial point cloud information is another way to represent the three-dimensional information of the subject, which can be converted from depth information.
  • the depth information collected by the depth sensor is reduced to three-dimensional spatial point cloud information.
  • the target area is an area that contains hand point cloud information within a certain depth range.
  • the user 40 during gesture operation, the user 40 usually performs gesture operation by swinging out a gesture by extending the palm of the hand. Therefore, on the premise that there is no obstruction and no other foreign objects, the palm will fall within a certain distance from the rest of the body. Combining experience and experiments, you can select point cloud information within a specific distance area as the target area that contains hand point cloud information.
  • point cloud information within a preset distance may be extracted as the target area.
  • the preset distance is determined by the specific distance entered by the hand. It shows that the hand is closer to the depth sensor that collects depth information than other parts of the body.
  • the preset distance is an empirical parameter, which can be set by a person skilled in the art according to the detection accuracy of the sensor, for example, about 10 cm.
  • the above-mentioned method of spatial point cloud can effectively and quickly select the target area containing the hand point cloud, which is convenient for subsequent operations.
  • the maximum connected domain algorithm may also be used to filter noise in the target area to improve the accuracy of gesture recognition.
  • the plane image is the projection result of the target area on the plane, and reflects the plane image of the hand.
  • the point cloud information in the target area may be specifically mapped to a two-dimensional space to generate a corresponding plane image. Converting 3D information into a flat image is a simple and fast mapping process that can be performed quickly.
  • the edge point of the hand in the plane image refers to the place where the regional attribute changes suddenly. It is usually formed by the junction between two areas. That is, the plane image is divided into several areas with different attributes (such as hands and background) by the recognized edges. Specifically, any existing edge detection or extraction algorithm may be used to complete step 350.
  • a series of continuous hand edge points can be extracted from the image using the Moore Neighborhood method.
  • the shape formed by the outline of the hand can be used to determine or judge the specific number of fingers.
  • the number of the fingers can be calculated and determined based on the edge feature of the planar image.
  • the number of the fingers may be calculated and determined by the following method:
  • convex hull detection algorithms such as incremental algorithm, wrap method (Jarvis step method), monotone chain, divide and conquer method, fast package method (AKI-Toussaint heuristic), Ge Liheng scanning algorithm (Graham Algorithms such as scan) find convex hull points in the edge points.
  • Convec (Convec Hull) is a geometric concept in graphics, which usually refers to a set of points that surround a convex polygon that can contain exactly all the target points.
  • the convex hull point can reflect the degree of tortuosity of the curved part of the tortuous line segment.
  • the convex hull point is the fingertip of the finger.
  • the fingertips corresponding to the fingers are all convex hull points (shown by the white squares in FIG. 4), and can be judged as the fingertips of the hand based on the fact that the degree of curvature is sufficient and the edge has a tip shape that meets the requirements.
  • the determination that the convex hull point is the fingertip of the finger can be determined by the following manner:
  • n edge points of the n edge points adjacent to it to the left and right are the first detection points
  • the i + n edge points are the second detection points.
  • the i-th edge point is the convex hull point
  • n is a positive integer, which represents an edge point spaced between the convex hull point and the first edge point and the second edge point, respectively.
  • n is a constant indicating the number of edge points spaced between the first edge point and the convex hull point and between the second edge point and the convex hull point. It can be determined by the size of the palm and the resolution of the depth sensor. In some embodiments, n can optionally be set to 10-50, but should not exceed the shortest finger length.
  • edge points in the edge point set are all continuous, it is convenient to determine the first and last n edge points as the first in the edge point set according to the position of the convex hull point The detection point and the second detection point.
  • the second detection point and the convex hull point calculate the vector as well as The angle between.
  • P i is the convex hull point
  • Pr is the first detection point
  • P l is the second detection point.
  • the first preset range is an empirical value used to measure the length and sharpness of the image area. Considering the accuracy of the commonly used depth sensor, the first preset range can be set to 20-60 °, which is used to determine whether the image area meets the shape requirements of the fingertip.
  • the included angle ⁇ can be calculated by the following formula (3):
  • the number of the fingers in the hand is determined. After the number of fingertips is known, each fingertip corresponds to one finger, and the number of fingers can be determined according to the number of fingertips.
  • the number of fingers obtained by corresponding shooting is also different.
  • the correspondence diagram shown in FIG. 5 Based on the number of fingers, different gestures can be quickly and easily identified to determine specific control instructions.
  • the relative movement of the palm may also be determined or recognized based on the position change of the recognized palm between different image frames.
  • the control instruction specifically corresponding to each gesture can be configured by a technician according to actual needs and stored as a software computer program in the memory.
  • personalized settings can also be made according to the user's personal habits.
  • the following provides a specific configuration example to detail the configuration process of gestures.
  • Movement instructions for controlling the drone to move in the front, back, left, and right directions that is, the pitch and roll angles of the drone
  • PTZ rotation instruction to adjust the rotation angle of the optical axis of the drone's gimbal (that is, the roll angle of the drone);
  • the method may further include the step of determining whether a hand exists in the planar image.
  • the judging whether the hand exists in the plane image may include the following steps:
  • the candidate point refers to a point within the range enclosed by the edge point in the planar image.
  • the distance between the candidate point and the edge point is the shortest distance between the two points.
  • the candidate point with the largest distance can be determined as the center of the largest inscribed circle of the palm of the hand by comparison.
  • the position of the largest inscribed circle is the position of the palm, and the position of the center of the largest inscribed circle can be regarded as the palm.
  • centroid method cannot distinguish the arm, the position of the centroid is easily affected by the arm, and the error is very large.
  • the above method of using the largest inscribed circle can well distinguish the palm and arm parts (the area of the arm is small, the area of the inscribed circle is small), and it is robust to the case where the arm also appears in the planar image .
  • the maximum inscribed circle can be calculated by the following method:
  • C is the edge point set
  • c i is the i-th edge point in the edge point set
  • p i is the i-th candidate point
  • D i is the minimum distance between the candidate point p i and all edge points
  • H is the set of all candidate points
  • D n is the radius of the largest inscribed circle.
  • the angle formed by the connecting line between the fingertip and the palm can be regarded as the angle between two adjacent fingers.
  • step 740 Determine whether the included angle is within a second preset range. If yes, go to step 750. If not, go to step 760.
  • the second preset range is also an empirical value, which can be set by a technician according to actual sensor accuracy and the like.
  • the angle between the fingertip and the palm line can also use other different judgment criteria to determine whether there is a hand.
  • the sum of the angle formed by any two adjacent lines between the fingertip and the palm is calculated. And, it is judged whether the sum of the included angles exceeds 180 °. If so, it is determined that there is a hand in the planar image. If not, it is determined that there is no hand in the plane image.
  • the limit range of the above included angle is related to the number of fingers of the user.
  • the upper limit of the sum of the included angles can be further adjusted and set to adapt to different situations.
  • the maximum inscribed circle can also be used to determine whether there is a hand in the planar image. As shown in FIG. 8, the determining whether the hand exists in the plane image may specifically include:
  • the candidate point refers to a point located in the inner region of the edge in the planar image.
  • the distance between the candidate point and the edge is the shortest distance between the two points.
  • the candidate point with the largest distance can be determined as the center of the largest inscribed circle by comparison.
  • the position of the largest inscribed circle is the position of the palm.
  • step 830 Determine whether the radius is within a third preset range. If yes, go to step 840, if no, go to step 850.
  • the maximum inscribed circle of the palm of the hand detected by the user 40 will fluctuate within a certain area.
  • the palms have a specific size and are not prone to significant changes. Therefore, the radius corresponding to the normal detection result usually fluctuates only within a certain range, and a result beyond this range has a great probability of being a recognition error.
  • the technician can set the corresponding third preset range according to the actual situation and / or the experimental result, which is used as a criterion for judging whether there is a hand in the planar image.
  • the third preset range is an empirical value, which can be adjusted and set according to actual conditions.
  • the number of fingers of the normal user 40 should not exceed 5 at most, or in the case of two-hand control, the number of fingers of the normal user should not exceed 10.
  • FIG. 6 is a method flowchart of the gesture recognition method with the capability of misjudgment correction. As shown in FIG. 6, this gesture recognition method combines the judgment methods of palm size, angle between fingers, and number of fingers. It can include the following steps:
  • the depth map is a disparity map of a binocular camera, and includes a baseline length and disparity data to represent depth information of the image.
  • the mutual conversion between the depth map and the spatial point cloud information is a very common conversion method.
  • the input depth map is a binocular camera parallax map
  • the three-dimensional coordinates of each point (xi, yi, zi) can be expressed as follows:
  • baseline is the length of the baseline
  • disparity is the parallax data
  • px, py, cx, cy, fx, fy are the calibration parameters.
  • the target area refers to a partial image area including the palm.
  • the preset edge extraction algorithm may be any existing type of edge extraction algorithm.
  • the edge points are a series of continuous edge points obtained by extraction.
  • step 617 Determine whether the radius of the largest inscribed circle is within the third preset range. If yes, go to step 620. If not, go to step 619.
  • the third preset range refers to a possible radius fluctuation range of the palm detected by the user 40 in a normal state.
  • the palms are of a specific size and are not prone to significant changes. Therefore, the radius corresponding to the normal detection result usually fluctuates only within a certain range, and a result beyond this range has a great probability of being a recognition error.
  • the technician can set the third preset range according to actual conditions and / or experimental results.
  • the third preset range when the third preset range is exceeded, it can basically be considered that the detection result is not the palm, and is usually a detection error caused by a situation such as being blocked. At this time, you can take the initiative to report and carry out the corresponding error correction process according to the actual situation. For example, it prompts the client that the gesture is invalid, and causes the client to re-gesture to acquire a new depth map.
  • the convex hull point can be found using Graham's scan method.
  • step 625 Determine whether the number of fingers of a single hand is greater than 5. If yes, go to step 620, if no, go to step 626.
  • the maximum number of fingers of a single hand of the normal user 40 should not exceed five. Therefore, when a large number of fingers are detected, it can basically indicate that a detection or recognition error has occurred, and the current detection result should be corrected and discarded.
  • the “sequential calculation” refers to calculating the angle formed by the connection between two adjacent finger tips and the palm according to the order of the detected fingers.
  • the angle between the first finger's fingertip and the second finger's fingertip and the palm of the hand is 1, then the second finger's fingertip and the third finger The angle between the line between your fingertip and the palm 2.
  • step 627 Determine whether the sum of the included angles is greater than 180 °. If yes, go to step 620. If not, it is determined that there is a hand in the target area (step 628).
  • the sum of the included angles is the sum of all calculated included angles. For example, add angle 1 and angle 2 to obtain the sum of two angles. Understandably, the greater the number of fingers, the greater the number of included angles. In most cases, when five fingers are detected, the four included angles need to be added.
  • the sum of these included angles should not exceed the upper limit of 180 ° (the user 40 cannot make the gesture). Therefore, when the sum of the included angles is greater than 180 °, it can also be confirmed that a recognition error has occurred.
  • the judgment criteria (the number of fingers and the sum of included angles) of the error correction judgment part of the finger are the most extreme cases. Those skilled in the art can understand that the above judgment criteria can be adjusted, combined or split according to the needs of different scenarios to obtain the same technical effect.
  • the upper limit of the sum of included angles can be dynamically changed according to the number of fingers, or the sum of included angles can be determined before calculating whether the number of fingers can meet the judgment standard.
  • the controller 20 shown in FIG. 1 may use software, hardware, or a combination of software and hardware to execute the gesture recognition method disclosed in the above method embodiment according to the received depth information, to realize gesture detection for the user 40 and to parse the corresponding control Instruction to realize the control of the drone 10 and the VR display device 30.
  • the controller 20 may control the drone 10 to hover near the user 40 so that the user 40 is within the detection range of the depth sensor.
  • the controller 20 applies the gesture recognition method as described above to determine the user's hand from the depth map. Then, according to the gesture obtained by the user's hand recognition, the VR viewing angle of the VR display device is adjusted to realize the gesture operation for the VR viewing angle or the VR scene.
  • the above-mentioned way of using gestures to control the angle of view of VR can not only avoid the inconvenience of misoperation due to the inability of the VR to observe the remote control, but also reduce the fatigue and dizziness caused by the traditional turning of the head to control the angle of view of VR.
  • the controller 20 can accurately control the movement of the drone 10 and the adjustment of the VR perspective according to the relative position between the palm and the drone.
  • the gesture recognition method provided by the embodiment of the present invention uses the depth map to extract the hand, avoiding the relatively cumbersome process of using the planar image to extract the hand, while greatly reducing the amount of calculation.
  • the traditional machine learning recognition method can be replaced by the geometric analysis method, which can have a higher operation frame rate than the machine learning method on the same device, ensuring the application on the device with low power consumption and low computing power.
  • the hand area is extracted based on the depth information and the arm is easily extracted at the same time.
  • the maximum inscribed circle detection method is used creatively to distinguish between the arm and the hand.
  • the palm is accurately determined in the case of the arm. Recognition of fingers is accomplished by the detection of the angle between the adjacent points of the convex hull point, which has the characteristics of high efficiency and strong robustness.
  • the gesture recognition method provided by the embodiment of the present invention, it can be widely used in robot palm tracking, drone palm landing, gesture recognition, and somatosensory operation, providing more control options the way.
  • the computer software can be stored in a computer-readable storage medium.
  • the storage medium may be a magnetic disk, an optical disk, a read-only storage memory or a random storage memory, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种手势识别方法、VR视角控制方法及VR系统。该手势识别方法包括：获取深度信息（310）；根据所述深度信息，获取空间点云信息（320）；确定所述空间点云信息中的目标区域，所述目标区域指包含手部点云信息的区域（330）；生成与所述目标区域对应的平面图像（340）；提取所述平面图像中手部的边缘点（350）；根据所述手部的边缘点，确定所述手部中手指的数量（360）；根据所述手指的数量，确定所述手部的手势（370）。该方法通过几何形态分析来实现手势的检测和识别，不需要依靠机器学习等样本学习算法，可以有效的降低手势识别的运算量的同时保证较高的手势识别率，满足低功耗、低时延以及低算力平台的应用要求。

Description

手势识别方法、VR视角控制方法以及VR系统 【技术领域】
本发明涉及虚拟现实技术领域，尤其涉及一种手势识别方法、VR视角控制方法及VR系统。
【背景技术】
虚拟现实(Virtual Reality VR)是一种利用相关设备，产生模拟度极高的模拟环境，通过三维交互、仿真等方式使用户沉浸在模拟环境中，以获得极佳的使用体验的技术。
典型的虚拟现实是通过佩戴在用户头部的VR眼镜等类似设备来实现的。这些VR眼镜需要包覆式的罩套在用户的头部。因此，当佩戴VR眼镜时，用户不方便通过遥控器等传统的操作方式实现VR视野调整。
现有一些通过位置传感器，采集用户头部位置变化来调整VR视野的操作方式。但是，由于佩戴在用户头部的VR眼镜重量和体积较大，通过转头调整视野的操作非常容易令用户感到疲劳并加重用户使用VR眼镜时的眩晕感。
随着技术发展，手势操作可能是一种能够满足使用需要的，较好的控制操作方式。但是，对于手势的检测和识别过程需要耗费非常大的运算量，使得手势操作的应用受到了很大的局限，如何有效的简化手势识别算法的复杂程度是一个迫切需要解决的问题。
【发明内容】
为了解决上述技术问题，本发明实施例提供一种可以降低手势检测所需运算量的手势识别方法、VR视角控制方法及VR系统。
为解决上述技术问题，本发明实施例提供以下技术方案：一种手势识别方法。该手势识别方法包括：获取深度信息；根据所述深度信息，获取空间点云信息；确定所述空间点云信息中的目标区域，所述目标区域指包含手部点云信息的区域；生成与所述目标区域对应的平面图像；提取所述平面图像 中手部的边缘点；根据所述手部的边缘点，确定所述手部中手指的数量；根据所述手指的数量，确定所述手部的手势。
可选地，所述获取所述深度信息包括：通过深度传感器获取所述深度信息。
可选地，所述确定所述空间点云信息中的目标区域，包括抽取预设距离之内的点云信息作为所述目标区域。
可选地，所述方法还包括：过滤所述目标区域中的噪点。
可选地，所述过滤所述目标区域中的噪点，包括通过最大连通域算法，过滤所述目标区域中的噪点。
可选地，所述生成与所述目标区域对应的所述平面图像，包括：将所述目标区域中的点云信息映射到二维空间，以生成所述目标区域对应的所述平面图像。
可选地，所述提取所述平面图像中所述手部的所述边缘点，包括：采用摩尔邻域(Moore Neighborhood)法，提取所述平面图像中所述手部的所述边缘点。
可选地，所述根据所述手部的所述边缘点，确定所述手部中所述手指的数量，包括：根据所述手部的所述边缘点，找出凸包(Convec Hull)点；确定所述凸包点为所述手指的指尖；根据所述指尖的数量，确定所述手部中所述手指的数量。
可选地，所述根据所述手部的所述边缘点，找出所述凸包点，包括：利用葛立恒扫描法(Graham's scan)找出所述凸包点。
可选地，所述确定所述凸包点为所述手指的所述指尖，包括：从所述手部的所述边缘点中，选取分别位于所述凸包点两侧且与所述凸包点相邻的第一边缘点和第二边缘点，以计算所述凸包点与所述第一边缘点连成的直线和所述凸包点与所述第二边缘点连成的直线之间的夹角；其中，所述第一边缘点和所述第二边缘点与所述凸包点位于同一根手指，且所述第一边缘点与所述凸包点之间、以及所述第二边缘点与所述凸包点之间间隔预设数量的边缘点；判断所述夹角是否在第一预设范围内；若是，则确定所述凸包点为所述手指的指尖。
可选地，所述判断所述夹角是否在所述第一预设范围内，包括：计算所述夹角：
Figure PCTCN2018109698-appb-000001
其中，θ为所述夹角，P i为所述凸包点，P l为所述第一边缘点，P r为所述第二边缘点；判断所述夹角是否小于预设值；若是，则确定所述凸包点为所述手指的指尖。
可选地，所述预设值在20°至60°之间取值。
可选地，所述第一边缘点与所述凸包点之间、以及所述第二边缘点与所述凸包点之间间隔10-50个边缘点。
可选地，所述方法还包括：判断所述平面图像中是否存在手部。
可选地，所述判断所述平面图像中是否存在手部，包括：计算候选点中每一个所述候选点与每一个所述边缘点之间的距离，所述候选点为所述边缘点围合范围内的点；确定最大距离对应的候选点为所述手部的手掌的掌心；计算任意两个相邻的所述指尖与所述掌心的连线形成的夹角；判断所述夹角是否在第二预设范围内；若是，则确定所述平面图像中存在手部。
可选地，所述判断所述平面图像中是否存在手部，包括：计算候选点中每一个所述候选点与每一个所述边缘点之间的距离，所述候选点为所述边缘点围合范围内的点；确定最大距离对应的候选点为所述手部的手掌的掌心；计算任意两个相邻的所述指尖与所述掌心的连线形成的夹角之和；判断所述夹角之和是否超过180°；若是，则确定所述平面图像中存在手部。
可选地，该方法还包括：判断所述平面图像中是否存在手部。
可选地，所述判断所述平面图像中是否存在手部，包括：计算候选点中每一个所述候选点与每一个所述边缘点之间的距离，所述候选点为所述边缘点围合范围内的点；确定最大距离为所述手部的手掌的最大内接圆的半径；判断所述半径是否在第三预设范围内；若是，则确定所述平面图像中存在手部。
可选地，该方法还包括：判断所述平面图像中是否存在手部。
可选地，所述判断所述平面图像中是否存在所述手部，包括：根据所述 手指的数量，判断所述平面图像中是否存在所述手部。
为解决上述技术问题，本发明实施例还提供以下技术方案：一种VR视角控制方法。其中，所述VR视角控制方法包括：
应用如上所述的手势识别方法，确定用户手部的手势；根据所述用户手部的手指，调整VR视角。
可选地，所述根据所述用户手掌的手指，调整VR视角，包括：根据所述用户手部的手指数量，识别当前用户手部的手势；通过所述用户手部的手势，调整拍摄设备的位置和朝向；跟随所述拍摄设备的位置和朝向变化来改变VR视角。
可选地，该方法还包括：应用如上述所述的手势识别方法，确定所述哟用户手部的掌心位置；根据所述用户手部的掌心位置，调整VR视角。
为解决上述技术问题，本发明实施例还提供以下技术方案：一种VR系统。其中，所述VR系统包括：移动载具、拍摄设备、深度传感器、控制器以及VR显示设备；所述拍摄设备和所述深度传感器均设置在所述移动载具上；
所述VR显示设备与所述拍摄设备通信连接，用于根据所述拍摄设备采集的视频图像信息生成对应的VR场景；
所述控制器用于使用如上所述的手势识别方法，根据所述深度传感器获取的深度信息识别用户的手势，并且根据所述手势，调整所述VR显示设备的VR视角。
可选地，所述控制器具体用于：根据所述手指的数量识别用户手部的手势，并且根据所述手势，控制所述拍摄设备的姿态和所述移动载具的移动。
可选地，所述控制器还用于使用如上述所述的手势识别方法，获取所述用户的掌心位置，并根据所述掌心位置，调整所述VR显示设备的VR视角。
可选地，所述控制器具体用于根据所述掌心位置，确定所述掌心位置与所述移动载具之间的相对位置，以控制所述拍摄设备的姿态和所述移动载具的移动。
可选地，所述拍摄设备搭载于所述移动载具的前部，所述深度传感器搭载于所述移动载具的后部。
可选地，所述移动载具为无人机。
可选地，所述拍摄设备通过云台搭载于所述移动载具。
与现有技术相比较，本发明实施例提供的手势识别方法，通过生成用户手部空间点云和平面图像来提取用户手部的边缘点来实现手势的检测和识别，不需要依靠机器学习等样本学习算法，算法实现的结构简单，有效的降低手势识别的运算量的同时保证较高的手势识别率，可以很好的应用于低功耗、低成本或者对时延较为敏感的平台。
【附图说明】
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1为本发明实施例提供的VR系统的应用环境示意图；
图2为本发明实施例提供的控制器的结构框图；
图3为本发明实施例提供的手势识别方法的方法流程图；
图4为本发明实施例提供的手指的指尖识别示意图；
图5为本发明实施例提供的手势数量与手势动作的匹配示意图；
图6为本发明另一实施例提供的手势识别方法的方法流程图；
图7为本发明实施例提供的判断平面图像中是否存在手部的方法流程图；
图8为本发明另一实施例提供的判断平面图像中是否存在手部的方法流程图。
【具体实施方式】
为了便于理解本发明，下面结合附图和具体实施例，对本发明进行更详细的说明。需要说明的是，当元件被表述“固定于”另一个元件，它可以直接在另一个元件上、或者其间可以存在一个或多个居中的元件。当一个元件被表述“连接”另一个元件，它可以是直接连接到另一个元件、或者其间可以存在一个或多个居中的元件。本说明书所使用的术语“上”、“下”、“内”、“外”、“底部”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。 此外，术语“第一”、“第二”“第三”等仅用于描述目的，而不能理解为指示或暗示相对重要性。
除非另有定义，本说明书所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本说明书中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是用于限制本发明。本说明书所使用的术语“和/或”包括一个或多个相关的所列项目的任意的和所有的组合。
此外，下面所描述的本发明不同实施例中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。
图1为本发明实施例提供的VR系统的应用环境。如图1所示，所述应用环境包括移动载具、设于所述移动载具的控制器20、VR显示设备30、用户40以及无线网络50。
其中，移动载具可以是以任何类型的由动力驱动的装载平台，包括但不限于四轴无人机、固定翼飞行器以及直升机模型等。该移动载具可以根据实际情况的需要，具备相应的体积或者动力，从而提供能够满足使用需要的载重能力、速度以及续航里程等。移动载具上还可以添加有一种或者多种功能模块，令移动载具能够实现对应的功能。
在本实施例中以无人机10为例进行陈述。本实施例中所述的无人机10可以包括机身、与机身相连的机臂和设于机臂的动力装置。机臂可以与机身固定连接、一体成型或可相对于所述机身折叠。动力装置包括电机和与电机相连的螺旋桨，电机的电机轴转动带动螺旋桨旋转以提供无人机飞行所需的拉升力。
例如，该无人机10还可以至少具备一个用于采集图像信息的拍摄设备11。该拍摄设备11可以是高清摄像机、运动相机或者其它类型的图像采集装置。
具体的，无人机10可以通过云台或者类似的抖动消除装置，搭载拍摄设备11，云台允许拍摄设备11相对于无人机10绕至少一个轴转动。
该无人机10还可以设置有用于采集深度信息的深度传感器12。该深度传感器12具体可以是双目相机、TOF相机、结构光相机或者激光雷达等。
在一些实施例中，所述拍摄设备11搭载于所述无人机10的前部，所述深度传感器搭12载于所述无人机的后部。
控制器20是设置在该移动载具10中的控制核心，用于执行一个或者多个逻辑判断步骤，实现对于移动载具10的控制。控制器20可以包括多个功能性单元，如，用于控制无人机飞行姿态的飞行控制单元、用于识别目标的目标识别单元、用于跟踪特定目标的跟踪单元、用于导航飞行器的导航单元(例如GPS(Global Positioning System)、北斗)、以及用于处理相关机载设备(如，拍摄设备11)所获取的环境信息的数据处理单元等。
图2为本发明实施例提供的控制器20的结构框图。如图2所示，该控制器20可以包括：处理器21、存储器22以及通信模块25。所述处理器21、存储器22以及通信模块25之间通过总线的方式，建立任意两者之间的通信连接。
处理器21为任何类型的单线程或者多线程的，具有一个或者多个处理核心的处理器，作为逻辑处理和运算的核心，用于获取数据、执行逻辑运算功能以及下发运算处理结果。
存储器22为非易失性计算机可读存储介质，例如至少一个磁盘存储器件、闪存器件、相对于处理器21远程设置的分布式存储设备或者其他非易失性固态存储器件。
存储器22具有程序存储区，用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，供处理器21调用以使处理器21执行一个或者多个方法步骤。存储器22还可以具有数据存储区，用以存储处理器21下发输出的运算处理结果。
通信模块25是无人机10用于建立通信连接，提供物理信道的功能模块例如WiFi模块、蓝牙模块获取其它的射频传输模块等。
无线网络50可以是用于建立两个节点之间的数据传输信道的无线通信网络，例如位于不同信号频段的蓝牙网络、WiFi网络或无线蜂窝网络。无人机10可以通过通信模块25加入到无线网络50中，通过无线网络50实现与VR显示设备30之间的通信连接。
VR显示设备30是位于用户侧，为用户提供虚拟显示环境的设备。该VR显示设备30具体可以是任何类型，可以实现VR技术的一个或者多个设备的组合。例如，传统的包覆式VR眼镜，头戴式VR设备以及结合VR技术的增强现实(AR)设备。
VR显示设备30与无人机10中的拍摄设备11建立有通信连接，可以接收来自拍摄设备11拍摄采集的视频或者图像信息，并据此生成相应的VR显示图像，提供给用户以实现沉浸式的虚拟现实体验。
用户40是佩戴有VR显示设备30的用户。其使用VR显示设备30来实现诸如无人机10飞行模拟等的服务。用户可以控制无人机10的航向角(或云台的转角)以及无人机的位置(如控制无人机前进或者后退)来改变VR显示设备的视角或者显示界面。
此时，由于用户40佩戴有VR显示设备。因此，难以通过控制遥控器的方式来对无人机进行控制。而采用转头或者移动身体来控制无人机的调整方式则非常容易造成用户的疲劳和眩晕。
在一些操作场景中，为了克服上述VR显示设备视角调整方式存在的问题，可以使用手势操作控制的方式来实现。用户40佩戴着VR显示设备30，无人机10悬停在用户40附近时，用户40可以伸手，通过不同的手势动作发出控制指令。此时，无人机10通过深度传感器，在朝向用户的位置采集相关的深度图像。
搭载在无人机10上的控制器20基于深度传感器采集获得的深度图像，可以分析检测用户的手势动作，解析对应的控制指令以调整无人机10的运行状态(包括移动无人机10的位置，改变拍摄设备11的朝向，焦距或者改变云台转动的角度)来响应用户发出的控制指令。
上述通过手势改变无人机的姿态或运动来改变VR视角的操作方式，与传统的遥控器控制相比，更为直观便捷。操作的同时无需走动或转头，使用户操作更舒适，用户体验更好。
在图1所示的应用环境中，仅以无人机10为例进行描述。本领域技术人员还可以将无人机10替换为任何类型的移动载具，例如遥控车等，用以承载上述的功能模块，为VR显示设备30提供数据源，实现虚拟现实的沉浸式体验。
本发明实施例提供的手势识别方法可以由控制器20的处理器21执行，以降低对于控制器20的运算量要求，有效的降低了控制器20的硬件成本以及功耗等，使其能够满足无人机10在使用上的限制。图3为本发明实施例提供的手势识别方法。如图3所示，该手势识别方法包括：
310、获取深度信息。
在一些实施例中，应用图1所示的深度传感器12可以采集获得相关的深度信息作为手势识别的基础数据。该深度信息是指能够反映拍摄对象的三维信息。
较佳的是，在存在较大噪声信息的情况下，还可以对接收到的深度信息进行预处理，过滤深度信息中的噪点。
320、根据所述深度信息，获取空间点云信息。
空间点云信息是另一种用于表示拍摄对象的三维信息的方式，其可以由深度信息转换得到。例如，深度传感器采集获得深度信息还原为三维的空间点云信息。
330、确定所述空间点云信息中的目标区域，所述目标区域为包含手部点云信息的区域。
该目标区域是在一定深度范围内，包含了手部点云信息的区域。如图1所示，用户40在手势操作时，惯常是通过伸出手掌的方式摆出手势来进行手势操作的。因此，在无遮挡和没有其他异物的前提下，区别于身体的其它部分，手掌将会落在一个特定距离的区域内。结合经验和实验，便可以选定特定距离区域内的点云信息作为包含了手部点云信息的目标区域。
具体的，可以抽取预设距离之内的点云信息作为所述目标区域。如上所述的，该预设距离由手部落入的特定距离所决定的。其表明了手部与身体其它部分相比，更接近于采集深度信息的深度传感器。
该预设距离是一个经验参数，其可以由本领域技术人员根据传感器的检测精度而设置，例如设置为10cm左右。
上述通过空间点云的方式能够有效、快速的选定包含手部点云的目标区域，便于进行后续的操作。
在其他可能的实施例中，还可以通过最大连通域算法过滤目标区域中的噪点，以提高手势识别的精确度。
340、生成与所述目标区域对应的平面图像。
该平面图像是目标区域在平面上的投影结果，反映了手部的平面图像。
在一些实施例中，具体可以将所述目标区域内的点云信息映射到一个二维空间以生成对应的平面图像。将三维信息转换为平面图像是一个简单并且 快速的映射过程，可以被快速的执行。
转换为平面图像以后，可以对其采用现有多种成熟或者常规的图像处理算法，从平面图像中提取一个或者多个特征来完成手势识别的任务。
350、提取所述平面图像中手部的边缘点。
该平面图像中手部的边缘点是指区域属性发生突变之处。其通常是由两个区域之间的交界形成。亦即，通过识别出的边缘将平面图像分为若干个不同属性的区域(如手部和背景)。具体可以采用现有任何类型的边缘检测或者提取算法来完成步骤350。
在一些实施例中，可以采用摩尔邻域(Moore Neighborhood)法在图像中提取一系列连续的手部的边缘点。
360、根据所述手部的边缘点，确定所述手部中手指的数量。
手部的边缘勾勒形成的形状可以用于确定或者判断手指的具体数量。该手指的数量可以基于平面图像的边缘特征计算确定。
在一些实施例中，在边缘通过一系列连续的边缘点组成的边缘点集合表示时，所述手指的数量可以通过如下方法计算确定：
首先，通过现有常用的凸包检测算法，例如增量式算法、包裹法(Jarvis步进法)、单调链、分治法、快包法(AKI-Toussaint启发式)、葛立恒扫描算法(Graham scan)等算法在边缘点中找出凸包点。
“凸包”(Convec Hull)是一个图形学上的几何概念，其通常是指围成能够恰好包含所有目标点的凸多边形的点组成的集合。通过凸包点可以反映边缘这一曲折的线段的弯曲部分的曲折程度。
然后，确定所述凸包点为所述手指的指尖。
如图4所示，在平面图像存在手部的情况下，由于手指是整个手部最外围的部分，呈现出相对长而尖的形状。因此，手指对应的指尖都是凸包点(图4中白色正方形所示)，在弯曲程度足够，边缘呈现满足要求的尖端形状的情况下便可以据此判断为手部的指尖。
具体的，所述确定所述凸包点为所述手指的指尖可以通过如下方式判断确定：
对于每个凸包点，分别找到与其左右相邻n个边缘点的第i-n个边缘点为第一检测点，第i+n个边缘点为第二检测点。其中，第i个边缘点为所述凸包 点，n为正整数，表示凸包点分别与第一边缘点和第二边缘点之间间隔的边缘点。
n是一个常数，表明第一边缘点与所述凸包点之间，第二边缘点与所述凸包点之间间隔的边缘点数量。其具体可以由手掌的大小和深度传感器的分辨率所确定。在一些实施例中，n可以选择设置为10-50，但不应当超过最短的手指长度。
在本实施例中，由于边缘点集合中的边缘点都是连续的，因此，可以方便的在边缘点集合中根据凸包点的位置，确定在前和在后的n个边缘点作为第一检测点和第二检测点。
最后，基于第一检测点、第二检测点以及凸包点，计算向量
Figure PCTCN2018109698-appb-000002
以及
Figure PCTCN2018109698-appb-000003
之间的夹角。其中，P i为所述凸包点，P r为所述第一检测点，P l为所述第二检测点。当所述两个向量之间的夹角小于第一预设范围时，表明此部分的边缘呈现出一个足够尖的部分，可以认为该凸包点P i为手指的指尖，并将相应的区域记录为手指。
该第一预设范围是一个经验数值，用于衡量图像区域长而且尖的程度。考虑到一般使用的深度传感器的精度，可以将第一预设范围设置为20-60°，用于判断图像区域是否满足指尖的形状要求。
具体的，所述夹角θ可以通过如下算式(3)计算获得：
Figure PCTCN2018109698-appb-000004
最后，根据指尖的数量，确定所述手部中所述手指的数量。在已知指尖的数量以后，每个指尖与一个手指对应，可以根据指尖的数量来确定手指的数量。
370、根据所述手指的数量，确定所述手部的手势。
在不同的手势下，对应拍摄获得的手指数量也不相同。例如，如图5所示的对应关系图。基于手指的数量可以快速简单的识别不同手势以确定具体的控制指令。
另外，连续采集多个图像帧的情况下，还可以基于识别到的手掌在不同的图像帧之间的位置变化来确定或者识别手掌的相对运动。
每个手势具体对应的控制指令可以根据实际情况的需要，由技术人员进行配置并作为软件计算机程序，存储在所述存储器中。当然，还可以根据用 户的个人习惯，进行个性化的设置。以下提供一个具体的配置实例以详细的陈述手势的配置过程。
首先，确定无人机可以根据手势执行的控制指令如下：
1、控制无人机在前后左右方向上移动(即无人机的pitch和roll角)的移动指令；
2、调整无人机的航向角(即无人机的yaw角)的翻转指令；
3、调整无人机的云台在左右上下方向上的转角(即云台的pitch角和yaw角)的云台摆动指令；
4、调整无人机的云台的光轴的转角(即无人机的roll角)的云台旋转指令；
5、控制无人机和云台恢复至调整前的初始位置或者暂停调整的重置指令或者暂停指令。
然后，按照如下的对应方式，为不同手指数量的手势配置对应的控制指令：
1、当识别到手指数量为5时，确定为移动指令，根据手掌在上下左右方向上的位移而相应的调整无人机的pitch和roll角；
2、当识别到手指数量为4时，确定为翻转指令，根据手掌的左右摆动而相应的调整无人机的yaw角；
3、当识别到的手指数量为3时，确定为云台摆动指令，根据手掌在上下左右方向上的位移而相应的改变云台的pitch角和yaw角；
4、当识别到的手指数量为2时，确定为云台旋转指令，根据手掌的左右摆动而相应的控制云台的光轴旋转，调整云台的roll角；
5、当识别到的手指数量为1或者0时，确定为重置或者暂停指令，控制云台和无人机恢复至初始位置(即初始化)或者暂停云台和无人机的位置调整。
通过上述手势识别方法，基于深度信息，可以快速准确的确定包含有手部的目标区域，并且结合使用现有成熟稳定的图像处理算法，从目标区域的平面图中提取凸包点和边缘，创造性的通过几何图形分析的方法来确定手指的数量。该确定手指的方法运算步骤和过程简单，避免了机器学习等的数据训练过程，需要消耗的运算量较少，有利于满足硬件设备低功耗的要求。
在一些实施例中，为了进一步的提高手势检测的准确度，避免错误识别等情况，所述方法还可以包括判断所述平面图像中是否存在手部的步骤。
在判断平面图像中存在手部时，才确认手势分析结果正确，对无人机作出对应的控制指令。
具体的，如图7所示，所述判断平面图像中是否存在手部可以包括如下步骤：
710、计算候选点中每一个所述候选点与每一个所述边缘点之间的距离。
该候选点是指所述平面图像中，位于所述边缘点围合范围内的点。所述候选点与所述边缘点之间的距离是两点之间的最短间距。
720、确定最大距离对应的候选点为所述手部的手掌的掌心。
在计算了所有候选点对应的距离以后，通过比较可以确定距离最大的候选点为手部的手掌最大内接圆的圆心。
在平面图像存在手部的情况下，通常手部的手掌占据的面积是最大的。因此，最大内接圆所在的位置即为手掌所在的位置，最大内接圆的圆心位置可以被认为是掌心。
与惯常使用的，根据平面图像中的边缘划出的内部区域的质心来确定手掌掌心的质心法相比：“质心法”无法区分手臂，质心的位置容易受到手臂的影响，误差很大。
而上述使用最大内接圆的方法可以很好的区分手掌和手臂部分(手臂的面积较小，内接圆的面积很小)，对于手臂也出现在平面图像的情况具有较强的鲁棒性。
具体的，在边缘通过一系列连续的边缘点组成的边缘点集合表示时，所述最大内接圆可以通过如下方法计算获得：
上述方法的两个步骤可以通过如下算式(1)和算式(2)表示：
D i＝Min{distance(p i,c i)|c i∈C} (1)
D n＝Max{D i|i∈H} (2)
其中，C为边缘点集合，c i为边缘点集合中的第i个边缘点，p i为第i个候选点，D i为候选点p i与所有边缘点之间的距离的最小值；H为所有候选点的集合，D n为最大内接圆的半径。
730、计算任意两个相邻的所述指尖与所述掌心的连线形成的夹角。
上述指尖与掌心之间的连线形成的夹角可以认为是两个相邻手指之间的夹角。
740、判断所述夹角是否在第二预设范围内。若是，则执行步骤750。若否，则执行步骤760。
750、确定所述平面图像中存在手部。
760、确定所述平面图像中不存在手部。
所述第二预设范围也是一个经验数值，可以由技术人员根据实际的传感器精度等进行设置。
在一些实施例中，指尖与掌心连线的夹角，还可以使用其它不同的判断标准来确定是否存在手部。
例如，计算任意两个相邻的所述指尖与所述掌心的连线形成的夹角之和。并且，判断所述夹角之和是否超过180°。若是，则确定所述平面图像中存在手部。若否，则确定所述平面图像中不存在手部。
当然，应当说明的是，上述夹角之和的极限范围与用户的手指数量是相关的。所述夹角之和的上限还可以进一步的调整和设置，以适应不同的情况。
在另一些实施例中，还可以结合最大内接圆的方式来判断平面图像中是否存在手部。如图8所示，所述判断平面图像中是否存在手部具体可以包括：
810、计算候选点中每一个所述候选点与每一个所述边缘点之间的距离。
该候选点是指所述平面图像中，位于所述边缘内部区域的点。所述候选点与边缘之间的距离是两点之间的最短间距。
820、确定最大距离为所述手部的手掌的最大内接圆的半径。
在计算了所有候选点对应的距离以后，通过比较可以确定距离最大的候选点为最大内接圆的圆心。
在平面图像存在手部的情况下，通常手部的手掌占据的面积是最大的。因此，最大内接圆所在的位置即为手掌所在的位置。
830、判断所述半径是否在第三预设范围内。若是，则执行步骤840，若否，则执行步骤850。
840、确定所述平面图像中存在手部。
850、确定所述平面图像中不存在手部。
在正常状态下，用户40检测到的手部的手掌的最大内接圆会在一定的面 积范围内波动。对于大部分用户而言，手掌都具有特定的大小，不容易发生显著的变动。因此，正常检测结果对应的半径通常只处于某个范围内发生波动，超出这个范围的结果具有极大的概率为识别错误。
因此，技术人员可以根据实际情况和/或实验结果，设置对应的第三预设范围，用以作为判断平面图像中是否存在手部的判断标准。该第三预设范围是一个经验数值，可以根据实际情况调整和设置。
在又一实施例中，还可以根据检测获得的所述的手指数量来判断所述平面图像中是否存在手部。亦即，判断上述步骤中计算获得的手指数量是否满足对应的条件。若是，则确定所述平面图像是否存在手部。若否，则确定所述平面图像不存在手部。
例如，在单手控制的情况下，正常用户40的手指数量最大不应当超过5个，或者在双手控制的情况下，正常用户的手指数量不应当超过10个。
因此，在检测到手指的数量超出正常范围时，可以表明出现了检测或者识别错误，确定平面图像中不存在手部，应当纠正并放弃当前的检测结果。
图6为该具备误判纠正能力的手势识别方法的方法流程图。如图6所示，该手势识别方法结合了手掌大小、手指间夹角和手指数量的判断方式。其可以包括如下步骤：
611、接收来自采集设备的深度图。具体的，该深度图为双目相机视差图，包含基线长度以及视差数据来表示图像的深度信息。
612、将所述深度图还原为空间点云信息。
深度图与空间点云信息之间的相互转换是一个非常常用的转换方式。在输入的深度图为双目相机视差图时，每个点(xi，yi，zi)的三维坐标可以表示如下：
Figure PCTCN2018109698-appb-000005
Figure PCTCN2018109698-appb-000006
Figure PCTCN2018109698-appb-000007
其中，baseline为基线长度，disparity为视差数据，px，py，cx，cy，fx， fy为标定参数。
613、抽取预设距离以内的空间点云信息作为目标区域。所述目标区域是指包含了手掌的局部图像区域。
614、将目标区域中的三维空间点云信息映射到二维空间，生成与所述目标区域对应的平面图像。
615、通过预设的边缘提取算法，提取所述平面图像中的边缘点。所述预设的边缘提取算法可以是现有任何类型的边缘提取算法，例如，摩尔邻域(Moore Neighborhood)法所述边缘点是提取获得的一系列连续的边缘点。
616、根据所述边缘点，计算手掌的最大内接圆的半径。
617、判断所述最大内接圆的半径是否在第三预设范围内。若是，执行步骤620。若否，则执行步骤619。
如图5所示，在假设深度图正确采样的情况下，最大内接圆的位置应当是与手掌重合的。因此，通过最大内接圆便可以相应的计算确定手掌的面积。
该第三预设范围是指正常状态下，用户40检测到的手掌可能的半径波动范围。由于对于大部分用户而言，手掌都是具有特定的大小，不容易发生显著的变动。因此，正常检测结果对应的半径通常只处于某个范围内发生波动，超出这个范围的结果具有极大的概率为识别错误。技术人员可以根据实际情况和/或实验结果，设置该第三预设范围。
619、确定所述最大内接圆的圆心为手掌的掌心。
在手掌面积处于正常状况下，可以确认手掌识别没有发生错误，可以将最大内接圆的圆心作为手掌的掌心。
620、确定所述目标区域中不存在手部。
相反地，当超出了第三预设范围时，基本可以认为检测的结果并不是手掌，通常是由于受遮挡等情况造成的检测错误。此时，可以主动报告，并根据实际情况的需要进行相应的纠错过程。例如，提示客户手势无效，令客户重新作出手势以采集新的深度图。
621、搜索所述边缘中的凸包点。所述凸包点可以利用葛立恒扫描法(Graham's scan)找出。
622、对于每个凸包点，分别找到与其左右相邻n个边缘点的第i-n个边 缘点为第一检测点，第i+n个边缘点为第二检测点。其中，第i个边缘点为所述凸包点，n为正整数。
623、基于第一检测点、第二检测点以及凸包点，计算向量
Figure PCTCN2018109698-appb-000008
和
Figure PCTCN2018109698-appb-000009
之间的夹角。
624、判断所述夹角是否在第一预设范围内。若是，则确定所述凸包点为手指的指尖。若否，则确定所述凸包点不是指尖。
每次确认获得一个手指指尖以后，令手指的数量加1，否则维持手指数量维持不变。
625、判断单个手部的手指数量是否大于5。若是，则执行步骤620，若否，则执行步骤626。
显然的，正常用户40的单个手部的手指数量最大不应当超过5个。因此，在检测到数量较多的手指时，基本可以表明出现了检测或者识别错误，应当纠正并放弃当前的检测结果。
626、根据指尖和掌心，依次计算两个相邻的所述指尖与所述掌心的连线形成的夹角之和。该“依次计算”是指按照检测到的手指的顺序，计算两个相邻指尖与掌心连线形成的夹角。
例如，当检测到3个手指以后，依次第一个手指的指尖与第二个手指的指尖与掌心之间的连线的夹角1，第二个手指的指尖与第三个手指的指尖与掌心之间的连线的夹角2。
627、判断所述夹角之和是否大于180°。若是，则执行步骤620。若否，则确定目标区域存在手部(步骤628)。
该夹角之和是所有计算获得的夹角的和。例如，将夹角1和夹角2相加，获得两个夹角之和。可以理解的，手指数量越多的情况下，相加的夹角的数量越多。最多的情况下，检测到5个手指时，需要将四个夹角相加。
显然地，即使在检测到全部的5个手指的情况下，这些夹角之和也不应当超过180°的上限(用户40无法做出该手势动作)。因此，当所述夹角之和是否大于180°时，同样也可以确认为出现了识别错误。
在本实施例中，关于手指的纠错判断部分的判断标准(手指数量以及夹角之和)均采用了最为极端的情况。本领域技术人员可以理解的是，可以根据不同的场景需要，对上述判断标准进行调整、组合或者拆分，获得同样的 技术效果。
例如，可以根据手指的数量动态的改变夹角之和的上限值或者是先判断夹角之和再计算手指的数量是否能够符合判断标准。
图1所示的控制器20可以使用软件、硬件或者软硬件结合的方式，根据接收到的深度信息，执行上述方法实施例揭露的手势识别方法，实现对于用户40的手势检测，解析对应的控制指令以实现对无人机10和VR显示设备30的控制。
在一些实施例中，所述控制器20可以控制无人机10在用户40附近悬停，使用户40处于深度传感器的探测范围内。控制器20应用如上所述的手势识别方法，从深度图中确定用户的手部。然后，根据所述用户的手部识别获得的手势，调整VR显示设备的VR视角，实现对于VR视角或者VR场景的手势操作。
上述使用手势控制VR视角的方式，既可以避免头戴VR无法观察遥控器而造成误操作的不便，又可以减少传统的转头控制VR视角带来的疲劳和眩晕感。
具体的，控制器20可以根据所述掌心和无人机之间的相对位置来精确的控制无人机10的移动和VR视角的调整。
例如，使无人机移动到目标位置并调整云台的转角以调整拍摄设备的位置和朝向，最终实现VR视角的改变和调整，为用户40提供更为便利操作方式和更好的沉浸式体验效果。
综上所述，本发明实施例提供的手势识别方法利用深度图来抽取手部，避免了利用平面图像抠出手部的相对繁琐的过程，同时大大减少了计算量。并且，以几何分析的方法替代传统的机器学习识别法，可以在相同的设备上拥有比机器学习法更高的运算帧率，保证了在低功耗低运算能力的设备上的应用。
进一步地，考虑到手臂容易与手部出现在同一平面内，基于深度信息抽取手部区域容易同时提取到手臂，创造性的使用了最大内接圆检测法来区分识别手臂和手部，可以在有手臂的情况下精确的确定掌心。对于手指的识别则由凸包点左右相邻点的夹角检测来完成，具有高效、鲁棒性强的特点。
基于本发明实施例提供的手势识别方法提供的手部的精确空间位置，可 以广泛的应用在机器人手掌跟踪、无人机掌上降落、手势识别和体感操作等用途上，提供了更多的控制选择方式。
本领域技术人员应该还可以进一步意识到，结合本文中所公开的实施例描述的示例性的数据传输控制方法的各个步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。
本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。所述的计算机软件可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体或随机存储记忆体等。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；在本发明的思路下，以上实施例或者不同实施例中的技术特征之间也可以进行组合，步骤可以以任意顺序实现，并存在如上所述的本发明的不同方面的许多其它变化，为了简明，它们没有在细节中提供；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (30)

  1. 一种手势识别方法，其特征在于，包括：
    获取深度信息；
    根据所述深度信息，获取空间点云信息；
    确定所述空间点云信息中的目标区域，所述目标区域指包含手部点云信息的区域；
    生成与所述目标区域对应的平面图像；
    提取所述平面图像中手部的边缘点；
    根据所述手部的边缘点，确定所述手部中手指的数量；
    根据所述手指的数量，确定所述手部的手势。
  2. 根据权利要求1所述的手势识别方法，其特征在于，所述获取所述深度信息，包括：
    通过深度传感器获取所述深度信息。
  3. 根据权利要求1或2所述的手势识别方法，其特征在于，所述确定所述空间点云信息中的目标区域，包括：
    抽取预设距离之内的点云信息作为所述目标区域。
  4. 根据权利要求1-3中任一项所述的手势识别方法，其特征在于，所述方法还包括：
    过滤所述目标区域中的噪点。
  5. 根据权利要求4所述的手势识别方法，其特征在于，所述过滤所述目标区域中的噪点，包括：
    通过最大连通域算法，过滤所述目标区域中的噪点。
  6. 根据权利要求1-5中任一项所述的手势识别方法，其特征在于，所述生成与所述目标区域对应的所述平面图像，包括：
    将所述目标区域中的点云信息映射到二维空间，以生成所述目标区域对应的所述平面图像。
  7. 根据权利要求1-6中任一项所述的手势识别方法，其特征在于，所述提取所述平面图像中所述手部的所述边缘点，包括：
    采用摩尔邻域(Moore Neighborhood)法，提取所述平面图像中所述手部的 所述边缘点。
  8. 根据权利要求1-7中任一项所述的手势识别方法，其特征在于，所述根据所述手部的所述边缘点，确定所述手部中所述手指的数量，包括：
    根据所述手部的所述边缘点，找出凸包(Convec Hull)点；
    确定所述凸包点为所述手指的指尖；
    根据所述指尖的数量，确定所述手部中所述手指的数量。
  9. 根据权利要求8所述的手势识别方法，其特征在于，所述根据所述手部的所述边缘点，找出所述凸包点，包括：
    利用葛立恒扫描法(Graham's scan)找出所述凸包点。
  10. 根据权利要求8或9所述的手势识别方法，其特征在于，所述确定所述凸包点为所述手指的所述指尖，包括：
    从所述手部的所述边缘点中，选取分别位于所述凸包点两侧且与所述凸包点相邻的第一边缘点和第二边缘点，以计算所述凸包点与所述第一边缘点连成的直线和所述凸包点与所述第二边缘点连成的直线之间的夹角；其中，所述第一边缘点和所述第二边缘点与所述凸包点位于同一根手指，且所述第一边缘点与所述凸包点之间、以及所述第二边缘点与所述凸包点之间间隔预设数量的边缘点；
    判断所述夹角是否在第一预设范围内；
    若是，则确定所述凸包点为所述手指的指尖。
  11. 根据权利要求10所述的手势识别方法，其特征在于，所述判断所述夹角是否在所述第一预设范围内，包括：
    计算所述夹角：
    Figure PCTCN2018109698-appb-100001
    其中，θ为所述夹角，P i为所述凸包点，P l为所述第一边缘点，P r为所述第二边缘点；
    判断所述夹角是否小于预设值；
    若是，则确定所述凸包点为所述手指的指尖。
  12. 根据权利要求11所述的手势识别方法，其特征在于，所述预设值在 20°至60°之间取值。
  13. 根据权利要求10-12中任一项所述的手势识别方法，其特征在于，所述第一边缘点与所述凸包点之间、以及所述第二边缘点与所述凸包点之间间隔10-50个边缘点。
  14. 根据权利要求7-13中任一项所述的手势识别方法，其特征在于，所述方法还包括：
    判断所述平面图像中是否存在手部。
  15. 根据权利要求14所述的手势识别方法，其特征在于，所述判断所述平面图像中是否存在手部，包括：
    计算候选点中每一个所述候选点与每一个所述边缘点之间的距离，所述候选点为所述边缘点围合范围内的点；
    确定最大距离对应的候选点为所述手部的手掌的掌心；
    计算任意两个相邻的所述指尖与所述掌心的连线形成的夹角；
    判断所述夹角是否在第二预设范围内；
    若是，则确定所述平面图像中存在手部。
  16. 根据权利要求14所述的手势识别方法，其特征在于，所述判断所述平面图像中是否存在手部，包括：
    计算候选点中每一个所述候选点与每一个所述边缘点之间的距离，所述候选点为所述边缘点围合范围内的点；
    确定最大距离对应的候选点为所述手部的手掌的掌心；
    计算任意两个相邻的所述指尖与所述掌心的连线形成的夹角之和；
    判断所述夹角之和是否超过180°；
    若是，则确定所述平面图像中存在手部。
  17. 根据权利要求1-13中任一项所述的手势识别方法，其特征在于，该方法还包括：
    判断所述平面图像中是否存在手部。
  18. 根据权利要求17所述的手势识别方法，其特征在于，所述判断所述平面图像中是否存在手部，包括：
    计算候选点中每一个所述候选点与每一个所述边缘点之间的距离，所述 候选点为所述边缘点围合范围内的点；
    确定最大距离为所述手部的手掌的最大内接圆的半径；
    判断所述半径是否在第三预设范围内；
    若是，则确定所述平面图像中存在手部。
  19. 根据权利要求1-13中任一项所述的手势识别方法，其特征在于，该方法还包括：
    判断所述平面图像中是否存在手部。
  20. 根据权利要求19所述的手势识别方法，其特征在于，所述判断所述平面图像中是否存在所述手部，包括：
    根据所述手指的数量，判断所述平面图像中是否存在所述手部。
  21. 一种VR视角控制方法，其特征在于，所述方法包括：
    应用如权利要求1-20中任一项所述的手势识别方法，确定用户手部的手势；
    根据所述用户手部的手势，调整VR视角。
  22. 根据权利要求21所述VR视角控制方法，其特征在于，所述根据所述用户手部的手势，调整所述VR视角，包括：
    根据所述用户手部的手指数量，识别当前用户手部的手势；
    根据所述用户手部的手势，调整拍摄设备的位置和朝向；
    根据所述拍摄设备的位置和朝向变化来调整所述VR视角。
  23. 根据权利要求21或22所述的VR视角控制方法，其特征在于，该方法还包括：
    应用如权利要求1-20中任一项所述的手势识别方法，确定所述用户手部的掌心位置；
    根据所述用户手部的掌心位置，调整VR视角。
  24. 一种VR系统，其特征在于，包括：
    移动载具；
    拍摄设备，搭载于所述移动载具；
    深度传感器，设置在所述移动载具上；
    控制器，设置于所述移动载具内；以及
    VR显示设备，所述VR显示设备与所述拍摄设备通信连接，用于根据所述拍摄设备采集的视频图像信息生成对应的VR场景；
    所述控制器用于使用如权利要求1-20任一项所述的手势识别方法，根据所述深度传感器获取的深度信息识别用户的手势，并且根据所述手势，调整所述VR显示设备的VR视角。
  25. 根据权利要求24所述的VR系统，其特征在于，所述控制器具体用于：根据所述手指的数量识别用户手部的手势，并且根据所述手势，控制所述拍摄设备的姿态和所述移动载具的移动。
  26. 根据权利要求24或25所述的VR系统，其特征在于，所述控制器还用于使用如权利要求1-20中任一项所述的手势识别方法，获取所述用户的掌心位置，并根据所述掌心位置，调整所述VR显示设备的VR视角。
  27. 根据权利要求26所述的VR系统，其特征在于，所述控制器具体用于根据所述掌心位置，确定所述掌心位置与所述移动载具之间的相对位置，以控制所述拍摄设备的姿态和所述移动载具的移动。
  28. 根据权利要求24-27中任一项所述的VR系统，其特征在于，所述拍摄设备搭载于所述移动载具的前部，所述深度传感器搭载于所述移动载具的后部。
  29. 根据权利要求24-28所述的VR系统，其特征在于，所述移动载具为无人机。
  30. 根据权利要求24-29所述的VR系统，其特征在于，所述拍摄设备通过云台搭载于所述移动载具。
PCT/CN2018/109698 2018-10-10 2018-10-10 手势识别方法、vr视角控制方法以及vr系统 WO2020073245A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/109698 WO2020073245A1 (zh) 2018-10-10 2018-10-10 手势识别方法、vr视角控制方法以及vr系统
CN201880098482.5A CN113039550B (zh) 2018-10-10 手势识别方法、vr视角控制方法以及vr系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/109698 WO2020073245A1 (zh) 2018-10-10 2018-10-10 手势识别方法、vr视角控制方法以及vr系统

Publications (1)

Publication Number Publication Date
WO2020073245A1 true WO2020073245A1 (zh) 2020-04-16

Family

ID=70164991

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/109698 WO2020073245A1 (zh) 2018-10-10 2018-10-10 手势识别方法、vr视角控制方法以及vr系统

Country Status (1)

Country Link
WO (1) WO2020073245A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982557A (zh) * 2012-11-06 2013-03-20 桂林电子科技大学 基于深度相机的空间手势姿态指令处理方法
CN104317391A (zh) * 2014-09-24 2015-01-28 华中科技大学 一种基于立体视觉的三维手掌姿态识别交互方法和系统
CN104914993A (zh) * 2015-05-15 2015-09-16 北京航空航天大学 一种手势控制民机客舱座椅调整的体验式设计方法
CN108230383A (zh) * 2017-03-29 2018-06-29 北京市商汤科技开发有限公司 手部三维数据确定方法、装置及电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982557A (zh) * 2012-11-06 2013-03-20 桂林电子科技大学 基于深度相机的空间手势姿态指令处理方法
CN104317391A (zh) * 2014-09-24 2015-01-28 华中科技大学 一种基于立体视觉的三维手掌姿态识别交互方法和系统
CN104914993A (zh) * 2015-05-15 2015-09-16 北京航空航天大学 一种手势控制民机客舱座椅调整的体验式设计方法
CN108230383A (zh) * 2017-03-29 2018-06-29 北京市商汤科技开发有限公司 手部三维数据确定方法、装置及电子设备

Also Published As

Publication number Publication date
CN113039550A (zh) 2021-06-25

Similar Documents

Publication Publication Date Title
US11914370B2 (en) System and method for providing easy-to-use release and auto-positioning for drone applications
US11340606B2 (en) System and method for controller-free user drone interaction
US11120261B2 (en) Imaging control method and device
US10551834B2 (en) Method and electronic device for controlling unmanned aerial vehicle
US20210227122A1 (en) System and method for providing autonomous photography and videography
US20200346750A1 (en) Method for generating flight path, control device, and unmanned aerial vehicle
US11715224B2 (en) Three-dimensional object reconstruction method and apparatus
CN111344644B (zh) 用于基于运动的自动图像捕获的技术
WO2018095278A1 (zh) 飞行器的信息获取方法、装置及设备
WO2021035646A1 (zh) 可穿戴设备及其控制方法、识别手势的方法和控制系统
WO2019242553A1 (zh) 控制拍摄装置的拍摄角度的方法、控制装置及可穿戴设备
KR20170100641A (ko) 실세계 객체들의 가상 표현들
EP3413165B1 (en) Wearable system gesture control method and wearable system
WO2021078003A1 (zh) 无人载具的避障方法、避障装置及无人载具
WO2021027886A1 (zh) 无人机飞行控制方法及无人机
WO2018195905A1 (zh) 一种无人机手掌降落的控制方法、控制设备及无人机
WO2019061466A1 (zh) 一种飞行控制方法、遥控装置、遥控系统
JPWO2018146922A1 (ja) 情報処理装置、情報処理方法、及びプログラム
WO2022082440A1 (zh) 确定目标跟随策略的方法、装置、系统、设备及存储介质
WO2020073245A1 (zh) 手势识别方法、vr视角控制方法以及vr系统
WO2021016875A1 (zh) 飞行器的降落方法、无人飞行器及计算机可读存储介质
CN113039550B (zh) 手势识别方法、vr视角控制方法以及vr系统
US11960660B2 (en) Terminal device, virtual object manipulation method, and virtual object manipulation program
CN113221729B (zh) 一种基于手势人机交互的无人机集群控制方法及系统
US20200167005A1 (en) Recognition device and recognition method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18936845

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18936845

Country of ref document: EP

Kind code of ref document: A1