WO2020124995A1 - 手掌法向量确定方法、装置、设备及存储介质 - Google Patents

手掌法向量确定方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2020124995A1
WO2020124995A1 PCT/CN2019/095095 CN2019095095W WO2020124995A1 WO 2020124995 A1 WO2020124995 A1 WO 2020124995A1 CN 2019095095 W CN2019095095 W CN 2019095095W WO 2020124995 A1 WO2020124995 A1 WO 2020124995A1
Authority
WO
WIPO (PCT)
Prior art keywords
palm
image
correction
normal vector
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/095095
Other languages
English (en)
French (fr)
Inventor
卢艺帆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Publication of WO2020124995A1 publication Critical patent/WO2020124995A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12—Fingerprints or palmprints

Definitions

  • Embodiments of the present disclosure relate to the field of image processing technology, for example, to a palm normal vector determination method, device, device, and storage medium.
  • Human-computer interaction is the main communication channel between people and devices. Its technology is constantly developing from feasible to convenient and comfortable. More and more research is placed on hand-based interaction technology. For other parts of the human body, the hands are free and flexible, and they are responsible for a lot of interactive work in the daily life of the user. There are countless operations completed through the hands.
  • the device can produce different responses by detecting the gesture of the human hand.
  • the detection of the palm normal vector is a part of the human hand gesture detection.
  • a corresponding interactive response can be generated based on the palm normal vector.
  • the palm normal vector is a palm normal vector obtained by performing plane fitting based on palm image data.
  • it is necessary to perform plane fitting based on the data of each palm image it is necessary to perform plane fitting based on the data of each palm image, and the data processing method is more complicated, resulting in the process of determining the palm normal vector is more complicated, And the accuracy of determining the palm normal vector needs to be improved.
  • Embodiments of the present disclosure provide a palm normal vector determination method, device, equipment, and storage medium, which can simplify the palm normal vector determination process and can improve the accuracy of palm normal vector determination.
  • An embodiment of the present disclosure provides a palm normal vector determination method, including:
  • the true normal vector of the palm in the palm image is determined.
  • An embodiment of the present disclosure also provides a palm normal vector determination device, including:
  • the palm image determination module is set to determine the palm image of the user
  • a palm correction image determination module configured to determine a correction relationship from the palm image to the palm correction image, and correct the palm image based on the correction relationship to obtain the palm correction image;
  • the target normal vector determination module is set to determine the normal vector of the palm in the palm correction image as the target normal vector
  • the real normal vector determination module is set to determine the real normal vector of the palm in the palm image based on the correction relationship and the target normal vector.
  • An embodiment of the present disclosure also provides a device, including:
  • One or more processors are One or more processors;
  • Storage device configured to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement a palm normal vector determination method provided by an embodiment of the present disclosure.
  • An embodiment of the present disclosure provides a computer-readable storage medium that stores a computer program that implements the palm normal vector determination method provided by the embodiment of the present disclosure when the program is executed by a processor.
  • FIG. 1 is a flowchart of a palm normal vector determination method provided by an embodiment of the present disclosure
  • FIG. 2 is a flowchart of another palm normal vector determination method provided by an embodiment of the present disclosure.
  • FIG. 3 is a flowchart of another palm normal vector determination method provided by an embodiment of the present disclosure.
  • FIG. 4 is a structural block diagram of a palm normal vector determination device provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic structural diagram of a device provided by an embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a palm normal vector determination method provided by an embodiment of the present disclosure.
  • the method may be executed by a palm normal vector determination device, the device is executed by software and/or hardware, and the device may be configured at Terminal.
  • the method can be applied in a scenario where palm normal vectors are determined.
  • the method can be applied to a scenario of human-computer interaction.
  • the method can also be applied to a scenario where a user interacts with a video application interface.
  • the method can also be applied to scenes that require image rendering based on the palm normal vector.
  • the technical solutions provided by the embodiments of the present disclosure include:
  • S110 Determine the palm image of the user.
  • the acquiring the palm image of the user includes: acquiring the user image, the user image including the palm of the user and the background other than the palm; inputting the user image to In the first deep learning model, position data of the palm detection frame is obtained; based on the position data of the palm detection frame, the palm image of the user is intercepted from the user image.
  • the user image may be a user image captured in real time, and the user image includes the user's palm and the background other than the palm. Input the user image into the first deep learning model to obtain the position data of the palm detection frame.
  • the first deep learning model may be a trained neural network model, or other trained models.
  • the first deep learning model can be pre-trained.
  • the training process can be: user image training samples can be selected, the user image training samples are used as the input of the first deep learning model, and the position data of the palm detection frame in the user image training samples is The output of the first deep learning model trains the first deep learning model to obtain the trained first deep learning model.
  • the user images taken in real time are input to the first deep learning model, and the first deep learning model can output the position data of the palm detection frame.
  • the palm detection frame is used to locate the palm in the user image. Through the position data of the palm detection frame, the user's palm image can be intercepted from the user image.
  • the method for determining the palm image of the user is not limited to the method described above, and other methods may also be used to determine the palm image of the user.
  • S120 Determine a correction relationship from the palm image to the palm correction image, and correct the palm image based on the correction relationship to obtain the palm correction image.
  • the correction relationship from the palm image to the palm correction image can be understood as the conversion relationship from the palm image to the palm correction image.
  • the correction relationship may be a correction matrix.
  • determining the correction relationship between the palm image and the palm correction image may be: determining two palm joint points in the palm image, and converting the positions of the two palm joint points to the corresponding palm joint points in the palm correction image Position, get the correction relationship; apply the correction relationship to the palm image to get the palm correction image.
  • S130 Determine the normal vector of the palm in the palm correction image and use it as the target normal vector.
  • the determining the normal vector of the corrected palm image includes: inputting the palm corrected image into a third deep learning model to obtain the palm of the palm corrected image Normal vector.
  • the third deep learning model may be a trained neural network model, or it may be another trained model.
  • the training process of the third deep learning model may be: selecting the palm image training samples, using the palm image training samples as the input of the third deep learning model, and using the palm normal vector in the palm image as the output of the third deep learning model.
  • Three neural network models are trained to obtain the trained third deep learning model.
  • the third deep learning model may output the normal vector of the palm in the palm correction image, and use the output normal vector as the target normal vector.
  • the palm normal vector may be a vector perpendicular to the plane of the palm. In one embodiment, the palm normal vector may be a unit vector that is perpendicular to the plane of the palm.
  • the third deep learning model can determine the normal vector of the palm in the palm correction image, so that the true normal vector of the palm in the palm image can be determined based on the normal vector of the palm in the palm correction image.
  • the third deep learning model is a model that uses the palm standard image for training, where the angle of the palm in the multiple palm standard images is the same, and the third deep learning model is performed using the palm standard image with the same palm angle During training, a deep learning model with good effect can be obtained, so that the output of the third deep learning model is more accurate.
  • the same angle of the palm in the palm standard image refers to the joint of the joint between the middle finger of the hand and the palm in the palm standard image, and the joint of the joint between the palm and the wrist is parallel to the vertical of the screen coordinate system Direction axis.
  • the palm normal vector is determined directly from the palm image data. Since the palm angle in the acquired palm image is very likely, it is not the same as the palm angle in the palm standard image. Therefore, when the acquired palm image is input to the third deep learning model, the normal vector of the palm output by the third deep learning model is not accurate.
  • the palm image can be corrected, and the palm correction image can be input into the third deep learning model to accurately obtain the palm normal vector in the palm correction image, and the palm normal vector in the palm correction image can be used to accurately obtain the palm image Normal vector of palm.
  • the palm in the palm-corrected image meets the conditions specified by the palm in the palm standard image.
  • the conditions specified by the palm in the palm standard image may be the joints of the joint of the middle finger of the hand and the palm in the palm standard image, and the joints of the joints of the palm and the wrist are parallel to the screen coordinate system. Vertical coordinate axis.
  • the palm angles in the acquired palm images are not the same, and palms based on different palm angles are required
  • the image determines the normal vector of the palm.
  • Each palm image uses a different algorithm, and the processing is more complicated.
  • the calculation of the palm normal vector will be relatively simple, which simplifies the determination of the palm normal vector Process, and can also improve the accuracy of palm normal vector determination.
  • S140 Determine the true normal vector of the palm in the palm image based on the correction relationship and the target normal vector.
  • the true normal vector of the palm in the palm image can be determined.
  • the process of determining the true normal vector of the palm in the palm image can be: based on the correction relationship from the palm image to the palm correction image, the inverse transformation relationship from the palm correction image to the palm image can be determined, and the palm correction image is based on the palm correction image The inverse transform relationship to the palm image and the normal vector of the palm in the palm correction image can determine the true normal vector of the palm in the palm image.
  • An embodiment of the present disclosure provides a method for determining a palm normal vector, which is corrected by the palm image, and the true normal vector of the palm in the palm image is determined by the palm normal vector in the palm correction image, which can simplify the process of determining the palm normal vector.
  • the accuracy of determining the palm normal vector can be improved.
  • the determining the correction relationship from the palm image to the palm correction image, and correcting the palm image based on the correction relationship to obtain the palm correction image includes: determining the palm image to all A correction matrix of the palm correction image, and correcting the palm image based on the correction matrix to obtain a palm correction image; accordingly, based on the correction relationship and the target normal vector, the palm of the palm image is determined
  • the true normal vector includes: determining the inverse matrix of the correction matrix, and multiplying the inverse matrix and the target normal vector to obtain the true normal vector of the palm in the palm image.
  • the technical solutions provided by the embodiments of the present disclosure include:
  • S210 Determine the palm image of the user.
  • S220 Determine a correction matrix from the palm image to the palm correction image, and correct the palm image based on the correction matrix to obtain a palm correction image.
  • determining a correction matrix from the palm image to the palm correction image, and correcting the palm image based on the correction matrix to obtain the palm correction image may include: Inputting the palm image into the second deep learning model to obtain the position coordinates of multiple palm joint points and the identification of the palm joint points corresponding to each of the position coordinates; based on the identification of the palm joint points from multiple palms Select the position coordinates of the first target joint point and the position coordinates of the second target joint point from the position coordinates of the key node; transform the position coordinates of the first target joint point to the position coordinates of the first preset joint point, and Transforming the position coordinates of the second target joint point to the position coordinates of the second preset joint point to obtain a correction matrix; performing coordinate transformation on the palm image based on the correction matrix to obtain a palm correction image.
  • the second deep learning model may be a trained neural network model or other trained models.
  • the training process of the second deep learning training model can be: selecting the palm image training samples, using the palm image training samples as the input of the second deep learning model, using the palm image training samples for the position coordinates of the palm joint points and the corresponding coordinates of each position coordinate
  • the mark of the palm joint point is the output of the second deep learning model, and the second deep learning network model is trained to obtain the trained second deep learning model.
  • the position coordinates of multiple palm joint points in the palm image and the identification of the palm joint point corresponding to each position coordinate may be output.
  • the mark of the palm joint point can be a number.
  • the position coordinates of the two joint points can be selected from the output position coordinate of the palm joint point, and the selected two position coordinates are respectively determined as the position of the first target joint point based on the identification of the palm joint point corresponding to the selected two position coordinates Coordinates and position coordinates of the second target joint point.
  • first target joint point and the second target joint point may be any two joint points in the palm joint point, or the first target joint point may be the joint point where the middle finger is connected to the palm, and the second target joint point The point may be a joint point where the palm is connected to the wrist.
  • the identifiers of the first target joint point and the first preset joint point may be the same, that is, the first target joint point and the first preset joint point may be the same position of the palm.
  • the identifiers of the second target joint point and the second preset joint point may be the same, that is, the second target joint point and the second preset joint point may be the same position of the palm.
  • the position coordinates of the first preset joint point and the position coordinates of the second preset joint point may be respectively: the position coordinates of the first target joint point and the position coordinates of the second target joint point are transformed into a standard palm image Corresponding position coordinates in.
  • the first target joint point and the second target joint point are joint point 1 and joint point 9, respectively, then the position coordinates of the first preset joint point are the position coordinates of joint point 1 in the standard palm image.
  • the second preset joint point is the position coordinate of joint point 9 in the standard palm image.
  • the position coordinates of the first target joint point are transformed into the position coordinates of the first preset joint point
  • the position coordinates of the second target joint point are transformed into the position coordinates of the second preset joint point to obtain the correction Matrix, and coordinate transformation based on the correction matrix to the palm image to obtain the palm correction image.
  • the position coordinates of the first target joint point and the position coordinates of the second target joint point are (x1, y1) and (x2, y2), respectively.
  • the position coordinates of the first preset joint point and the position coordinates of the second preset joint point are (x1', y1') and (x2', y2'), respectively (x1, y1) and (x2, y2) Transform to (x1', y1') and (x2', y2').
  • [x1y1]R [x1'y1']
  • [x2y2]R [x2'y2'].
  • R is the correction matrix
  • the correction matrix can be obtained through calculation, and the correction matrix is used to coordinate transform the position coordinates of each pixel in the palm image to obtain the palm correction image. Therefore, by selecting the positions of the two palm joint points and performing coordinate transformation to obtain a correction matrix, the coordinate changes of the palm image based on the correction matrix to obtain the palm correction image can reduce the amount of data processing and increase the processing speed.
  • S230 Determine the normal vector of the palm in the palm correction image and use it as the target normal vector.
  • S240 Determine the inverse matrix of the correction matrix, and multiply the inverse matrix and the target normal vector to obtain the true normal vector of the palm in the palm image.
  • FIG. 3 is a flowchart of another palm normal vector determination method provided by an embodiment of the present disclosure. As shown in FIG. 3, the technical solution provided by the embodiment of the present disclosure includes:
  • S310 Acquire a user image, where the user image includes the user's palm and the background outside the palm.
  • S320 Input the user image into the first deep learning model to obtain the position data of the palm detection frame.
  • S330 Intercept the palm image of the user from the user image based on the position data of the palm detection frame.
  • the position data of the palm detection frame is obtained through the first deep learning model, and the palm image of the user is intercepted based on the position data, and the palm image can be quickly obtained.
  • S340 Input the palm image into the second deep learning model to obtain the position coordinates of the multiple palm joint points and the identification of the palm joint points corresponding to each position coordinate.
  • S350 Select the position coordinates of the first target joint point and the position coordinates of the second target joint point from a plurality of position coordinates of the palm joint point based on the identification of the palm joint point.
  • S360 Transform the position coordinates of the first target joint point to the position coordinates of the first preset joint point, and transform the position coordinates of the second target joint point to the position coordinates of the second preset joint point to obtain Correction matrix.
  • S370 Perform coordinate transformation on the palm image based on the correction matrix to obtain a palm correction image.
  • S380 Determine the normal vector of the palm in the palm correction image and use it as the target normal vector.
  • S390 Determine an inverse matrix of the correction matrix, and multiply the inverse matrix and the target normal vector to obtain the true normal vector of the palm in the palm image.
  • the third deep learning model is used to determine the normal vector of the palm in the palm correction image, and the true normal vector of the palm in the palm image is determined based on the normal vector of the palm in the palm correction image, because
  • the palm angle in the palm correction image input into the third deep learning model is the same, and the calculation of the palm normal vector is relatively simple.
  • the correction matrix from the palm image to the palm correction image is also easy to obtain. Therefore, the process of determining the true normal vector of the palm is simplified .
  • the third deep learning model can be trained using the palm image training samples with the same palm angle to obtain a model with better training effect.
  • the accuracy of the output palm normal vector is improved, so that the palm correction image is input into the third depth model At this time, the palm normal vector of the palm correction image can be accurately obtained, and then the palm normal vector based on the palm correction image can accurately obtain the true normal vector of the palm image.
  • the palm normal vector determination method may further include: performing image rendering in a direction in which the true normal vector of the palm is located in the palm image.
  • multiple animated objects can be rendered in the direction of the true normal vector of the palm.
  • the terminal acquires the user image, intercepts the user palm image from the user image, and determines the palm normal vector method in this application
  • the terminal can render multiple circles etc. along the direction of the true normal vector of the palm, that is, multiple circles are arranged along the direction of the true normal vector of the palm. Users can watch the circle array launched by themselves, which can improve the fun and improve the user experience.
  • FIG. 4 is a structural block diagram of a palm normal vector determination device provided by an embodiment of the present disclosure.
  • the technical solution provided by the embodiment of the present disclosure includes: a palm image determination module 410, a palm correction image determination module 420, The target normal vector determination module 430 and the real normal vector determination module 440.
  • the palm image determination module 410 is set to determine the user's palm image; the palm correction image determination module 420 is set to determine the correction relationship from the palm image to the palm correction image, and correct the palm image based on the correction relationship, The palm correction image is obtained; the target normal vector determination module 430 is set to determine the palm normal vector in the palm correction image as the target normal vector; the true normal vector determination module 440 is set to be based on the correction relationship and Said target normal vector to determine the true normal vector of the palm in the palm image.
  • the palm correction image determination module 420 is a correction matrix configured to determine the palm image to the palm correction image, and corrects the palm image based on the correction matrix to obtain a palm correction image; true
  • the normal vector determining module 440 is configured to determine the inverse matrix of the correction matrix, and multiply the inverse matrix and the target normal vector to obtain the true normal vector of the palm in the palm image.
  • the palm correction image determination module 420 is configured to input the palm image into the second deep learning model to obtain the position coordinates of multiple palm joint points and the identification of the palm joint points corresponding to each position coordinate Selecting the position coordinates of the first target joint point and the position coordinates of the second target joint point from the position coordinates of the plurality of palm joint points based on the identification of the palm joint point; the position of the first target joint point Transforming the coordinates to the position coordinates of the first preset joint point, and transforming the position coordinates of the second target joint point to the position coordinates of the second preset joint point to obtain a correction matrix; based on the correction matrix, the palm The image undergoes coordinate transformation to obtain a palm-corrected image.
  • the palm image determination module 410 is configured to acquire a user image, wherein the user image includes the user's palm and a background other than the palm; the user image is input to the first deep learning model , The position data of the palm detection frame is obtained; based on the position data of the palm detection frame, the palm image of the user is intercepted from the user image.
  • the target normal vector determination module 430 is configured to determine the normal vector of the palm in the palm correction image by inputting the palm correction image into a third deep learning model to obtain the palm correction Image palm normal vector.
  • the device further includes a rendering module configured to render the image along the direction of the palm's true normal vector in the palm image.
  • the above-mentioned device can execute the method provided by any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 5 shows a schematic structural diagram of an electronic device (eg, terminal device) 500 suitable for implementing the embodiments of the present disclosure.
  • the terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant (PDA), tablet computers (Portable Android Device, PAD), portable multimedia players (Portable Media Player, PMP), mobile terminals such as in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital televisions (Television, TV), desktop computers, and so on.
  • PDA Personal Digital Assistant
  • PMP portable multimedia players
  • mobile terminals such as in-vehicle terminals (such as in-vehicle navigation terminals)
  • fixed terminals such as digital televisions (Television, TV), desktop computers, and so on.
  • the electronic device shown in FIG. 5 is just an example, and should not bring any limitation to the functions and use scope of the embodiments of the present disclosure.
  • the electronic device 500 may include a processing device (such as a central processing unit, a graphics processor, etc.) 501, and the processing device may be based on a program stored in a read-only memory (Read-only Memory, ROM) 502 or from a storage
  • the device 508 loads the program in the random access memory (Random Access Memory, RAM) 503 to perform one or more appropriate actions and processes.
  • RAM Random Access Memory
  • the processing device 501, ROM 502, and RAM 503 are connected to each other via a bus 504.
  • An input/output (Input/Output, I/O) interface 505 is also connected to the bus 504.
  • the following devices can be connected to the I/O interface 505: including input devices 506 such as touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, Liquid Crystal Display (LCD), speakers , An output device 507 such as a vibrator; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509.
  • the communication device 509 may allow the electronic device 500 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 5 shows an electronic device 500 having various devices, it is not required to implement or have all the devices shown. More or fewer devices may be implemented or provided instead.
  • an embodiment of the present disclosure includes a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication device 509, or from the storage device 508, or from the ROM 502.
  • the processing device 501 When the computer program is executed by the processing device 501, the above-described functions defined in the method of the embodiments of the present disclosure are executed.
  • the computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above.
  • Examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer diskettes, hard drives, RAM, ROM, erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, and the computer-readable signal medium carries computer-readable program code.
  • This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: electric wires, optical cables, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • the computer-readable medium may be included in the electronic device; or it may exist alone without being assembled into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device causes the electronic device to: determine the palm image of the user; determine the correction relationship from the palm image to the palm correction image And correct the palm image based on the correction relationship to obtain the palm correction image; determine the normal vector of the palm in the palm correction image as the target normal vector; based on the correction relationship and the target method Vector to determine the true normal vector of the palm in the palm image.
  • the computer program code for performing the operations of the present disclosure can be written in one or more programming languages or a combination thereof.
  • the programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN) or a wide area network (Wide Area Network, WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing a prescribed logical function.
  • the functions marked in the blocks may also occur in an order different from that marked in the drawings. For example, two blocks represented in succession may actually be executed in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.
  • modules and units described in the embodiments of the present disclosure may be implemented in software or hardware. Among them, the name of the module or unit does not constitute a limitation on the module or unit itself under certain circumstances.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

一种手掌法向量确定方法、装置、设备及存储介质，其中，该方法包括：确定用户的手掌图像（S110）；确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像（S120）；确定所述手掌矫正图像中手掌的法向量，并作为目标法向量（S130）；基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量（S140）。

Description

手掌法向量确定方法、装置、设备及存储介质
本申请要求在2018年12月20日提交中国专利局、申请号为201811565432.1的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开实施例涉及图像处理技术领域，例如涉及一种手掌法向量确定方法、装置、设备及存储介质。
背景技术
人机交互作为人与设备之间的主要交流通道，其技术也在不断的由操作可行到操作便利、操作舒适的方向发展，越来越多的研究放在基于手的交互技术方面，相比于其他人体部位，手部自由灵活，在用户的日常生活中担负着大量的交互工作，通过手部完成的操作数不胜数。
在人机交互过程中，通过对人手姿态的检测，设备可以产生不同的响应。其中，手掌法向量的检测是人手姿态检测的一部分，通过检测手掌法向量，可以基于手掌法向量产生对应的交互响应。相关技术中，手掌法向量是通过根据手掌图像数据进行平面拟合得到手掌的法向量。但是相关技术中，在确定不同的手掌图像中手掌的法向量的情况下，需要基于每张手掌图像的数据进行平面拟合，数据处理方法比较复杂，从而导致手掌法向量的确定过程比较复杂、并且手掌法向量的确定精确度有待于提高。
发明内容
本公开实施例提供了一种手掌法向量确定方法、装置、设备及存储介质，可以简化手掌法向量的确定过程，可以提高手掌法向量的确定精确度。
本公开实施例提供了一种手掌法向量确定方法，包括：
确定用户的手掌图像；
确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像；
确定所述手掌矫正图像中手掌的法向量，并作为目标法向量；
基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量。
本公开实施例还提供了一种手掌的法向量确定装置，包括：
手掌图像确定模块，设置为确定用户的手掌图像；
手掌矫正图像确定模块，设置为确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像；
目标法向量确定模块，设置为确定所述手掌矫正图像中手掌的法向量，并作为目标法向量；
真实法向量确定模块，设置为基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量。
本公开实施例还提供了一种设备，包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序，
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本公开实施例提供的一种手掌法向量确定方法。
本公开实施例提供了一种计算机可读存储介质，存储有计算机程序，该程序被处理器执行时实现本公开实施例提供的手掌法向量确定方法。
附图说明
图1是本公开实施例提供的一种手掌法向量确定方法流程图；
图2是本公开实施例提供的另一种手掌法向量确定方法流程图；
图3是本公开实施例提供的另一种手掌法向量确定方法流程图；
图4是本公开实施例提供的一种手掌法向量确定装置结构框图；
图5是本公开实施例提供的一种设备结构示意图。
具体实施方式
下面结合附图和实施例对本公开进行说明。此处所描述的具体实施例仅仅用于解释本公开，而非对本公开的限定。另外，为了便于描述，附图中仅示出了与本公开相关的部分而非全部结构。
图1是本公开实施例提供的一种手掌法向量确定方法流程图，所述方法可以由手掌法向量确定装置来执行，所述装置由软件和/或硬件来执行，所述装置可以配置在终端中。所述方法可以应用在确定手掌法向量的场景中。一实施例中，所述方法可以应用到人机交互的场景中，一实施例中，所述方法也可以应 用在用户与视频应用界面交互的场景中。一实施例中，所述方法还可以应用在基于手掌法向量需要进行图像渲染的场景中。
如图1所示，本公开实施例提供的技术方案包括：
S110：确定用户的手掌图像。
在本公开实施例的一个实施方式中，所述获取用户的手掌图像，包括：获取用户图像，所述用户图像中包含用户的手掌和所述手掌之外的背景；将所述用户图像输入到第一深度学习模型中，得到手掌检测框的位置数据；基于所述手掌检测框的位置数据从所述用户图像中截取所述用户的手掌图像。
一实施例中，用户图像可以是实时拍摄的用户的图像，用户图像中包含用户的手掌和手掌之外的背景。将用户图像输入到第一深度学习模型中，得到手掌检测框的位置数据。一实施例中，第一深度学习模型可以是已训练的神经网络模型，可以是已训练的其他模型等。第一深度学习模型可以预先进行训练，训练的过程可以是：可以选取用户图像训练样本，以用户图像训练样本为第一深度学习模型的输入，以用户图像训练样本中手掌检测框的位置数据为第一深度学习模型的输出，对第一深度学习模型进行训练，得到已训练的第一深度学习模型。第一深度学习模型训练后，将实时拍摄的用户图像输入到第一深度学习模型，第一深度学习模型可以输出手掌检测框的位置数据。一实施例中，手掌检测框，用于定位用户图像中的手掌。通过手掌检测框的位置数据，可以从用户图像中截取用户的手掌图像。
对于用户的手掌图像的确定方法并不局限于上述的方法，还可以采用其他方法确定用户的手掌图像。
S120：确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像。
在本公开实施例中，手掌图像到手掌矫正图像的矫正关系可以理解是手掌图像变换到手掌矫正图像的变换关系。一实施例中，矫正关系可以是矫正矩阵。
一实施例中，确定手掌图像到手掌矫正图像的矫正关系，可以是：确定手掌图像中的两个手掌关节点，将该两个手掌关节点的位置变换到手掌矫正图像中对应手掌关节点的位置，得到矫正关系；并将矫正关系作用到手掌图像中，得到手掌矫正图像。
S130：确定所述手掌矫正图像中手掌的法向量，并作为目标法向量。
在本公开实施例中的一个实施方式中，所述确定所述矫正的手掌图像的法向量，包括：将所述手掌矫正图像输入到第三深度学习模型中，得到所述手掌矫正图像的手掌法向量。一实施例中，第三深度学习模型可以是已训练的神经 网络模型，或者也可以是已训练的其他模型。第三深度学习模型的训练过程可以是：选取手掌图像训练样本，以手掌图像训练样本为第三深度学习模型的输入，以手掌图像中手掌的法向量为第三深度学习模型的输出，对第三神经网络模型进行训练，得到已训练的第三深度学习模型。
一实施例中，在将手掌矫正图像输入到第三深度学习模型的情况下，第三深度学习模型可以输出手掌矫正图像中手掌的法向量，并将输出的法向量作为目标法向量。一实施例中，手掌法向量可以是垂直于手掌所在平面的向量。一实施例中，手掌法向量可以是垂直于手掌所在平面的单位向量。
由此，通过第三深度学习模型可以确定手掌矫正图像中手掌的法向量，从而可以基于手掌矫正图像中手掌的法向量确定手掌图像中手掌的真实法向量。一实施例中，第三深度学习模型是采用手掌标准图像进行训练的模型，其中，多张手掌标准图像中手掌的角度是相同的，采用手掌角度相同的手掌标准图像对第三深度学习模型进行训练，可以得到效果较好的深度学习模型，以使第三深度学习模型的输出更加准确。一实施例中，手掌标准图像中手掌的角度相同是指手掌标准图像中手部的中指与手掌连接处的关节点，以及手掌与手腕连接处的关节点的连线平行于屏幕坐标系的垂直方向坐标轴。在不对获取到的手掌图像进行矫正的情况下，直接根据手掌图像的数据确定手掌法向量，由于获取到的手掌图像中手掌的角度很大可能，与手掌标准图像中手掌的角度并不相同，因此，在将获取到的手掌图像输入到第三深度学习模型的情况下，第三深度学习模型输出的手掌的法向量并不准确。所以，可以将手掌图像进行矫正，将手掌矫正图像输入到第三深度学习模型中，可以准确得到手掌矫正图像中手掌的法向量，并采用手掌矫正图像中手掌的法向量可以准确得到手掌图像中手掌的法向量。其中，手掌矫正图像中的手掌符合手掌标准图像中对手掌规定的条件。一实施例中，手掌标准图像中对手掌规定的条件可以是手掌标准图像中手部的中指与手掌连接处的关节点，以及手掌与手腕连接处的关节点的连线平行于屏幕坐标系的垂直方向坐标轴。
并且在不对手掌图像进行矫正，并将获取到的多张手掌图像输入到第三深度学习模型的情况下，获取到的多张手掌图像中手掌的角度并不相同，需要基于不同手掌角度的手掌图像确定手掌的法向量，每张手掌图像采用不同的算法，处理比较复杂。在将每一张手掌图像进行矫正，使得输入到第三深度学习模型的多张手掌矫正图像中手掌的角度相同的情况下，手掌法向量的计算会相对简单，从而简化了手掌法向量确定的过程，并且也可以提高手掌法向量的确定精确度。
S140：基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的 真实法向量。
在本公开实施例中，基于手掌图像到手掌矫正图像的矫正关系、以及手掌矫正图像中手掌的法向量，可以确定手掌图像中手掌的真实法向量。基于矫正关系和目标法向量，确定手掌图像中手掌的真实法向量的过程可以是：基于手掌图像到手掌矫正图像的矫正关系，可以确定手掌矫正图像到手掌图像的逆变换关系，基于手掌矫正图像到手掌图像的逆变换关系，以及手掌矫正图像中手掌的法向量，可以确定手掌图像中手掌的真实法向量。
本公开实施例提供的一种手掌法向量的确定方法，通过对手掌图像进行矫正，并通过手掌矫正图像中手掌法向量确定手掌图像中手掌的真实法向量，可以简化手掌法向量的确定过程，可以提高手掌法向量的确定精确度。
图2是本公开实施例提供的另一种手掌法向量确定方法流程图。一实施例中，所述确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像，包括：确定所述手掌图像到所述手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到手掌矫正图像；相应的，基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量，包括：确定所述矫正矩阵的逆矩阵，并将所述逆矩阵与所述目标法向量相乘，得到所述手掌图像中手掌的真实法向量。
如图2所示，本公开实施例提供的技术方案包括：
S210：确定用户的手掌图像。
S220：确定所述手掌图像到所述手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到手掌矫正图像。
在本公开实施例的一个实施方式中，确定所述手掌图像到所述手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到手掌矫正图像，可以包括：将所述手掌图像输入到第二深度学习模型中，得到多个手掌关节点的位置坐标以及每个所述位置坐标对应的手掌关节点的标识；基于所述手掌关节点的标识从多个所述手掌关节点的位置坐标中选取第一目标关节点的位置坐标和第二目标关节点的位置坐标；将所述第一目标关节点的位置坐标变换到第一预设关节点的位置坐标，并将所述第二目标关节点的位置坐标变换到第二预设关节点的位置坐标，得到矫正矩阵；基于所述矫正矩阵对所述手掌图像进行坐标变换，得到手掌矫正图像。
一实施例中，第二深度学习模型可以是已训练的神经网络模型，也可以是已训练的其他模型。第二深度学习训练模型的训练过程可以是：选取手掌图像 训练样本，以手掌图像训练样本为第二深度学习模型的输入，以手掌图像训练样本手掌关节点的位置坐标以及每个位置坐标对应的手掌关节点的标识为第二深度学习模型的输出，对第二深度学习网络模型进行训练，得到已训练的第二深度学习模型。
一实施例中，在将手掌图像输入到第二深度学习模型的情况下，可以输出手掌图像中多个手掌关节点的位置坐标以及每个位置坐标对应的手掌关节点的标识，一实施例中，手掌关节点的标识可以是编号。从输出的手掌关节点的位置坐标可以选取两个关节点的位置坐标，并基于选取的两个位置坐标对应的手掌关节点的标识确定选取的两个位置坐标分别是第一目标关节点的位置坐标和第二目标关节点的位置坐标。一实施例中，第一目标关节点和第二目标关节点可以是手掌关节点中的任意两个关节点，或者第一目标关节点可以是中指与手掌连接处的关节点，第二目标关节点可以是手掌与手腕连接处的关节点。
一实施例中，第一目标关节点和第一预设关节点的标识可以是相同的，即第一目标关节点和第一预设关节点可以是手掌的同一个位置。第二目标关节点和第二预设关节点的标识可以是相同的，即第二目标关节点和第二预设关节点可以是手掌的同一个位置。一实施例中，第一预设关节点的位置坐标和第二预设关节点的位置坐标可以分别是：第一目标关节点的位置坐标和第二目标关节点的位置坐标变换到标准手掌图像中分别对应的位置坐标。例如，第一目标关节点和第二目标关节点分别是1号关节点和9号关节点，则第一预设关节点的位置坐标是1号关节点在标准手掌图像中的位置坐标，第二预设关节点是9号关节点在标准手掌图像中的位置坐标。
一实施例中，将第一目标关节点的位置坐标变换到第一预设关节点的位置坐标，并将第二目标关节点的位置坐标变换到第二预设关节点的位置坐标，得到矫正矩阵，并基于矫正矩阵对手掌图像进行坐标变换，得到手掌矫正图像。例如，第一目标关节点的位置坐标和第二目标关节点的位置坐标分别是(x1，y1)和(x2，y2)。第一预设关节点的位置坐标和第二预设关节点的位置坐标分别是(x1’，y1’)和(x2’，y2’)，将(x1，y1)和(x2，y2)分别变换到(x1’，y1’)和(x2’，y2’)。则[x1y1]R＝[x1’y1’]，[x2y2]R＝[x2’y2’]。其中R为矫正矩阵，经过计算可以得到矫正矩阵，并将矫正矩阵对手掌图像中每个像素点的位置坐标进行坐标变换，得到手掌矫正图像。由此，通过选取两个手掌关节点的位置进行坐标变换得到矫正矩阵，基于矫正矩阵对手掌图像进行坐标变化，得到手掌矫正图像，可以减少数据处理量，提高处理速度。
S230：确定所述手掌矫正图像中手掌的法向量，并作为目标法向量。
S240：确定所述矫正矩阵的逆矩阵，并将所述逆矩阵与所述目标法向量相 乘，得到所述手掌图像中手掌的真实法向量。
在本公开实施例中，手掌图像到手掌矫正图像基于矫正矩阵进行坐标变换，所以手掌图像中手掌的法向量到手掌矫正图像中的手掌法向量也是基于矫正矩阵进行坐标变换。故矫正矩阵的逆矩阵与手掌矫正图像中手掌法向量相乘，可以得到手掌图像中手掌的真实法向量。例如，a1R＝a1’，其中，a1是手掌图像的真实法向量，a1’是手掌矫正图像中手掌的法向量，则a1＝a1’R -1。
图3是本公开实施例提供的另一种手掌法向量确定方法流程图，如图3所示，本公开实施例提供的技术方案包括：
S310：获取用户图像，所述用户图像中包含用户的手掌和所述手掌之外的背景。
S320：将所述用户图像输入到第一深度学习模型中，得到手掌检测框的位置数据。
S330：基于所述手掌检测框的位置数据从所述用户图像中截取所述用户的手掌图像。
由此，通过第一深度学习模型得到手掌检测框的位置数据，并基于该位置数据截取用户的手掌图像，可以快速得到手掌图像。
S340：将所述手掌图像输入到第二深度学习模型中，得到多个手掌关节点的位置坐标以及每个位置坐标对应的手掌关节点的标识。
S350：基于所述手掌关节点的标识从多个所述手掌关节点的位置坐标中选取第一目标关节点的位置坐标和第二目标关节点的位置坐标。
S360：将所述第一目标关节点的位置坐标变换到第一预设关节点的位置坐标，并将所述第二目标关节点的位置坐标变换到第二预设关节点的位置坐标，得到矫正矩阵。
S370：基于所述矫正矩阵对所述手掌图像进行坐标变换，得到手掌矫正图像。
S380：确定所述手掌矫正图像中手掌的法向量，并作为目标法向量。
S390：确定所述矫正矩阵的逆矩阵，并将所述逆矩阵与所述目标法向量相乘，得到所述手掌图像中手掌的真实法向量。
由此，通过将手掌图像基于矫正矩阵进行矫正，通过第三深度学习模型确定手掌矫正图像中手掌的法向量，并基于手掌矫正图像中手掌的法向量确定手掌图像中手掌的真实法向量，由于输入到第三深度学习模型中的手掌矫正图像 中手掌的角度相同，手掌法向量计算相对简单，手掌图像到手掌矫正图像的矫正矩阵也很容易得到，因此，简化了手掌真实法向量的确定过程。同时可以采用手掌角度相同的手掌图像训练样本对第三深度学习模型进行训练，得到训练效果较好的模型，输出的手掌法向量的精确度提高，从而将手掌矫正图像输入到第三深度模型中时，可以准确得到手掌矫正图像的手掌法向量，进而基于手掌矫正图像的手掌法向量可以准确得到手掌图像的真实法向量。
在上述实施例的基础上，本公开实施例提供的手掌法向量确定方法还可以包括：沿所述手掌图像中手掌的真实法向量所在的方向进行图像渲染。一实施例中，可以沿手掌真实法向量所在的方向渲染多个动画对象。例如，在用户与视频应用界面进行交互的情况下，用户可以采用手掌与视频应用界面进行交互，终端获取到用户图像，从用户图像中截取用户手掌图像，通过本申请中的手掌法向量确定方法确定手掌图像中手掌的真实法向量，终端可以沿手掌的真实法向量的方向渲染多个圆圈等，即多个圆圈沿手掌的真实法向量方向排列。用户可以观看到自己发射的圆圈阵，可以提高趣味性，可以提高用户体验。
图4是本公开实施例提供的一种手掌的法向量确定装置的结构框图，如图4所示，本公开实施例提供的技术方案包括：手掌图像确定模块410、手掌矫正图像确定模块420、目标法向量确定模块430和真实法向量确定模块440。
手掌图像确定模块410，设置为确定用户的手掌图像；手掌矫正图像确定模块420，设置为确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像；目标法向量确定模块430，设置为确定所述手掌矫正图像中手掌的法向量，并作为目标法向量；真实法向量确定模块440，设置为基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量。
一实施例中，手掌矫正图像确定模块420，是设置为确定所述手掌图像到所述手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到手掌矫正图像；真实法向量确定模块440，是设置为确定所述矫正矩阵的逆矩阵，并将所述逆矩阵与所述目标法向量相乘，得到所述手掌图像中手掌的真实法向量。
一实施例中，手掌矫正图像确定模块420，是设置为将所述手掌图像输入到第二深度学习模型中，得到多个手掌关节点的位置坐标以及每个位置坐标对应的手掌关节点的标识；基于所述手掌关节点的标识从多个所述手掌关节点的位置坐标中选取第一目标关节点的位置坐标和第二目标关节点的位置坐标；将所述第一目标关节点的位置坐标变换到第一预设关节点的位置坐标，并将所述第 二目标关节点的位置坐标变换到第二预设关节点的位置坐标，得到矫正矩阵；基于所述矫正矩阵对所述手掌图像进行坐标变换，得到手掌矫正图像。
一实施例中，手掌图像确定模块410，是设置为获取用户图像，其中，所述用户图像中包含用户的手掌和所述手掌之外的背景；将所述用户图像输入到第一深度学习模型中，得到手掌检测框的位置数据；基于所述手掌检测框的位置数据从所述用户图像中截取所述用户的手掌图像。
一实施例中，目标法向量确定模块430，是设置为通过如下方式确定所述手掌矫正图像中手掌的法向量：将所述手掌矫正图像输入到第三深度学习模型中，得到所述手掌矫正图像的手掌法向量。
一实施例中，所述装置还包括：渲染模块，设置为沿所述手掌图像中手掌的真实法向量所在的方向进行图像渲染。
上述装置可执行本公开任意实施例所提供的方法，具备执行方法相应的功能模块和有益效果。
下面参考图5，图5示出了适于用来实现本公开实施例的电子设备(例如终端设备)500的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字电视机(Television，TV)、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图5所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，处理装置可以根据存储在只读存储器(Read-only Memory，ROM)502中的程序或者从存储装置508加载到随机访问存储器(Random Access Memory，RAM)503中的程序而执行一种或多种适当的动作和处理。在RAM 503中，还存储有电子设备500操作所需的一种或多种程序和数据。处理装置501、ROM 502以及RAM 503通过总线504彼此相连。输入/输出(Input/Output，I/O)接口505也连接至总线504。
以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(Liquid Crystal Display，LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子 设备500与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有多种装置的电子设备500，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，该计算机程序产品包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从网络上被下载和安装，或者从存储装置508被安装，或者从ROM 502被安装。在该计算机程序被处理装置501执行时，执行本公开实施例的方法中限定的上述功能。
一实施例中，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、RAM、ROM、可擦式可编程只读存储器(Electrically Programmable Read-Only-Memory，EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，计算机可读信号介质中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，RF)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：确定用户的手掌图像；确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫 正，得到所述手掌矫正图像；确定所述手掌矫正图像中手掌的法向量，并作为目标法向量；基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开一种或多种实施例的方法和计算机程序产品的可能实现的体系架构、功能和操作。流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块、单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块、单元的名称在某种情况下并不构成对该模块或单元本身的限定。

Claims (10)

  1. 一种手掌法向量确定方法，包括：
    确定用户的手掌图像；
    确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像；
    确定所述手掌矫正图像中手掌的法向量，并作为目标法向量；
    基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量。
  2. 根据权利要求1所述的方法，其中，所述确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像，包括：
    确定所述手掌图像到手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到所述手掌矫正图像；
    所述基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量，包括：
    确定所述矫正矩阵的逆矩阵，并将所述逆矩阵与所述目标法向量相乘，得到所述手掌图像中手掌的真实法向量。
  3. 根据权利要求2所述的方法，其中，所述确定所述手掌图像到手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到所述手掌矫正图像，包括：
    将所述手掌图像输入到第一深度学习模型中，得到多个手掌关节点的位置坐标以及每个位置坐标对应的手掌关节点的标识；
    基于所述手掌关节点的标识从多个所述手掌关节点的位置坐标中选取第一目标关节点的位置坐标和第二目标关节点的位置坐标；
    将所述第一目标关节点的位置坐标变换到第一预设关节点的位置坐标，并将所述第二目标关节点的位置坐标变换到第二预设关节点的位置坐标，得到矫正矩阵；
    基于所述矫正矩阵对所述手掌图像进行坐标变换，得到所述手掌矫正图像。
  4. 根据权利要求1、2或3所述的方法，其中，所述确定用户的手掌图像，包括：
    获取用户图像，其中，所述用户图像中包含用户的手掌和所述手掌之外的背景；
    将所述用户图像输入到第二深度学习模型中，得到手掌检测框的位置数据；
    基于所述手掌检测框的位置数据从所述用户图像中截取所述用户的手掌图像。
  5. 根据权利要求1-4任一项所述的方法，其中，所述确定所述手掌矫正图像中手掌的法向量，包括：
    将所述手掌矫正图像输入到第三深度学习模型中，得到所述手掌矫正图像的手掌法向量。
  6. 根据权利要求1-5任一项所述的方法，还包括：
    沿所述手掌图像中手掌的真实法向量所在的方向进行图像渲染。
  7. 一种手掌的法向量确定装置，包括：
    手掌图像确定模块，设置为确定用户的手掌图像；
    手掌矫正图像确定模块，设置为确定所述手掌图像到手掌矫正图像的矫正关系，并基于所述矫正关系对所述手掌图像进行矫正，得到所述手掌矫正图像；
    目标法向量确定模块，设置为确定所述手掌矫正图像中手掌的法向量，并作为目标法向量；
    真实法向量确定模块，设置为基于所述矫正关系和所述目标法向量，确定所述手掌图像中手掌的真实法向量。
  8. 根据权利要求7所述的装置，其中，
    所述手掌矫正图像确定模块，是设置为确定所述手掌图像到手掌矫正图像的矫正矩阵，并基于所述矫正矩阵对所述手掌图像进行矫正，得到所述手掌矫正图像；
    所述真实法向量确定模块，是设置为确定所述矫正矩阵的逆矩阵，并将所述逆矩阵与所述目标法向量相乘，得到所述手掌图像中手掌的真实法向量。
  9. 一种设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序，
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-6任一项所述的一种手掌法向量确定方法。
  10. 一种计算机可读存储介质，存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1-6任一项所述的手掌法向量确定方法。
PCT/CN2019/095095 2018-12-20 2019-07-08 手掌法向量确定方法、装置、设备及存储介质 Ceased WO2020124995A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811565432.1A CN109683710B (zh) 2018-12-20 2018-12-20 一种手掌法向量确定方法、装置、设备及存储介质
CN201811565432.1 2018-12-20

Publications (1)

Publication Number Publication Date
WO2020124995A1 true WO2020124995A1 (zh) 2020-06-25

Family

ID=66188138

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/095095 Ceased WO2020124995A1 (zh) 2018-12-20 2019-07-08 手掌法向量确定方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN109683710B (zh)
WO (1) WO2020124995A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109683710B (zh) * 2018-12-20 2019-11-08 北京字节跳动网络技术有限公司 一种手掌法向量确定方法、装置、设备及存储介质
CN110349096B (zh) * 2019-06-14 2024-08-02 平安科技（深圳）有限公司 手掌图像的校正方法、装置、设备及存储介质
CN111414837A (zh) * 2020-03-16 2020-07-14 苏州交驰人工智能研究院有限公司 手势识别方法、装置、计算机设备及存储介质
CN113420700B (zh) * 2021-07-02 2022-10-25 支付宝(杭州)信息技术有限公司 手掌生物特征采集装置及方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101866482A (zh) * 2010-06-21 2010-10-20 清华大学 基于摄像设备自标定技术的全景图拼接方法和装置
CN103488972A (zh) * 2013-09-09 2014-01-01 西安交通大学 基于深度信息的指尖检测方法
EP3147820A1 (en) * 2015-09-25 2017-03-29 Ricoh Company, Ltd. Object tracking method, device, and system as well as relevant program and non-transitory computer-readable medium
CN108282651A (zh) * 2017-12-18 2018-07-13 北京小鸟看看科技有限公司 相机参数的矫正方法、装置及虚拟现实设备
CN109683710A (zh) * 2018-12-20 2019-04-26 北京字节跳动网络技术有限公司 一种手掌法向量确定方法、装置、设备及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103558850B (zh) * 2013-07-26 2017-10-24 无锡信捷电气股份有限公司 一种激光视觉引导的焊接机器人全自动运动自标定方法
CN103745474B (zh) * 2014-01-21 2017-01-18 南京理工大学 基于惯性传感器和摄像机的图像配准方法
KR101593950B1 (ko) * 2014-05-28 2016-02-15 숭실대학교산학협력단 손동작 기반의 인터페이스 장치 및 이를 이용한 포인팅 방법
CN106383586A (zh) * 2016-10-21 2017-02-08 东南大学 一种孤独症谱系障碍儿童训练系统
CN107341473B (zh) * 2017-07-04 2018-07-06 深圳市利众信息科技有限公司 手掌特征识别方法、手掌特征识别设备、及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101866482A (zh) * 2010-06-21 2010-10-20 清华大学 基于摄像设备自标定技术的全景图拼接方法和装置
CN103488972A (zh) * 2013-09-09 2014-01-01 西安交通大学 基于深度信息的指尖检测方法
EP3147820A1 (en) * 2015-09-25 2017-03-29 Ricoh Company, Ltd. Object tracking method, device, and system as well as relevant program and non-transitory computer-readable medium
CN108282651A (zh) * 2017-12-18 2018-07-13 北京小鸟看看科技有限公司 相机参数的矫正方法、装置及虚拟现实设备
CN109683710A (zh) * 2018-12-20 2019-04-26 北京字节跳动网络技术有限公司 一种手掌法向量确定方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN109683710B (zh) 2019-11-08
CN109683710A (zh) 2019-04-26

Similar Documents

Publication Publication Date Title
US11625841B2 (en) Localization and tracking method and platform, head-mounted display system, and computer-readable storage medium
US11893702B2 (en) Virtual object processing method and apparatus, and storage medium and electronic device
CN109712234B (zh) 三维人体模型的生成方法、装置、设备和存储介质
WO2020124995A1 (zh) 手掌法向量确定方法、装置、设备及存储介质
US12591979B2 (en) Image generation method and device
CN111368668B (zh) 三维手部识别方法、装置、电子设备及存储介质
CN114742856B (zh) 一种视频处理方法、装置、设备及介质
WO2022007565A1 (zh) 增强现实的图像处理方法、装置、电子设备及存储介质
CN116596748A (zh) 图像风格化处理方法、装置、设备、存储介质和程序产品
WO2020155915A1 (zh) 用于播放音频的方法和装置
WO2020211422A1 (zh) 视频处理方法、装置及设备
CN110069191B (zh) 基于终端的图像拖拽变形实现方法和装置
CN115937010B (zh) 一种图像处理方法、装置、设备及介质
CN110032978A (zh) 用于处理视频的方法和装置
CN110717467A (zh) 头部姿势的估计方法、装置、设备及存储介质
US20250209714A1 (en) Method and apparatus for generating special effect video, and device and storage medium
CN115619918A (zh) 图像渲染方法、装置、设备及存储介质
CN116185201A (zh) 扩展现实交互方法、装置、电子设备和存储介质
US11622071B2 (en) Follow-up shooting method and device, medium and electronic device
WO2023025181A1 (zh) 图像识别方法、装置和电子设备
WO2023051362A1 (zh) 图像区域处理方法及设备
WO2025021169A1 (zh) 图像处理方法、设备、存储介质及程序产品
CN115880719B (zh) 手势深度信息生成方法、装置、设备和计算机可读介质
CN110070479A (zh) 图像变形拖拽点定位方法和装置
CN116311486A (zh) 一种视线估计方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19901366

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01/10/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19901366

Country of ref document: EP

Kind code of ref document: A1