CN109172066A - Intelligent artificial limb hand and its system and method based on voice control and visual identity - Google Patents
Intelligent artificial limb hand and its system and method based on voice control and visual identity Download PDFInfo
- Publication number
- CN109172066A CN109172066A CN201810943869.8A CN201810943869A CN109172066A CN 109172066 A CN109172066 A CN 109172066A CN 201810943869 A CN201810943869 A CN 201810943869A CN 109172066 A CN109172066 A CN 109172066A
- Authority
- CN
- China
- Prior art keywords
- artificial limb
- intelligent artificial
- gesture motion
- hand
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
- A61F2/50—Prostheses not implantable in the body
- A61F2/68—Operating or control means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
- A61F2/50—Prostheses not implantable in the body
- A61F2/68—Operating or control means
- A61F2002/6809—Operating or control means acoustic
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
- A61F2/50—Prostheses not implantable in the body
- A61F2/68—Operating or control means
- A61F2002/6881—Operating or control means optical
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
- A61F2/50—Prostheses not implantable in the body
- A61F2/68—Operating or control means
- A61F2/70—Operating or control means electrical
- A61F2002/704—Operating or control means electrical computer-controlled, e.g. robotic control
Landscapes
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Transplantation (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Vascular Medicine (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Prostheses (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of intelligent artificial limb hand and its system and method based on voice control and visual identity, belong to prosthetic hand technical field, intelligent artificial limb hand getting Three Degree Of Freedom prosthetic hand ontology, it further include depth camera head module, voice interaction module, LED touch screen, core processing module and wireless charging module, wherein, depth camera head module is for obtaining image data；Voice interaction module is used to carry out interactive voice with user；LED touch screen inputs operation instruction and shows data for user；Core processing module realizes overall task schedule and data processing；And disclose Internet of Things intelligent artificial limb hand system and gesture motion implementation method.Certainly existing mainstream prosthetic hand only relies on myoelectric control, traction control to solution of the present invention, control efficiency is low, electromyography signal interference is big, the problem and voice control function of operation training complexity are single, is unable to complete complicated gesture operation, intelligent low technical problem.
Description
Technical field
The invention belongs to prosthetic hand technical fields, more particularly, to a kind of intelligence based on voice control and visual identity
It can prosthetic hand and its system and method.
Background technique
Intelligent artificial limb hand can drive the opening and closing of artificial hand and the movement of wrist, elbow joint by direct current micromotor, to complete vacation
The compensation of limb can provide convenience for the daily life of upper deficiency of skeletal limb disabled person patient, improve the quality of living.In rehabilitation project
There is huge value in the research in field.Existing prosthetic hand product be divided into single-degree-of-freedom, the Myoelectric Controlled Forearm Prosthesis of two-freedom and
The Muscular-Electric Artificial Upper Arm of two-freedom, Three Degree Of Freedom.First two freedom degree is low, and many scenes are simultaneously not suitable for.
Three Degree Of Freedom myoelectric limb hand is as disclosed in patent CN201610520941.7, although prosthetic hand has enough
High freedom degree solves the problems, such as in production, but wherein only refers to the joint of prosthetic hand by motor control, and how user transports
Electromyography signal control artificial limb is generated with deformed limb muscular movement and carries out complicated movement, is not referred in the patent.In fact root
It is fed back according to user experience, high-freedom degree prosthetic hand is difficult to control, and interfering with each other for electromyography signal often generates malfunction, and fills
The manipulation training after exercise for power and dress limb before limb is more complicated, and the recognition efficiency of electromyography signal is low.Therefore for high free
Degree prosthetic hand just needs other recognition methods other than electromyography signal to control.
Patent CN200710144648.6 discloses the multiple freedom degrees hand-prosthesis voice controlling apparatus based on blue tooth wireless communication,
Voice controller is connect with prosthetic hand by bluetooth, on the one hand can only voice extract the existing movement of Keywords matching, generate
Movement have limitation.On the other hand, due to excessively relying on wireless telecommunications control, voice controller not by the side of, periphery
When radio interference is larger, the generation of control failure will lead to.Only in terms of this angle of voice control, for specific hand
Gesture, voice control exist to the inenarrable limitation of movement.
Also, in power supply mode, existing prosthetic hand is all the method using wired charging.On the one hand in the mistake of charging
Cheng Zhong, long period limit the movement of user or the use of artificial limb.On the other hand, it due to the exposure of charging interface, limits
The waterproof performance of artificial limb is promoted.
In terms of forward position, controlling prosthetic hand by brain wave is also a direction.Have more directly, more by brain wave control
The advantages of close to manpower.But current brain wave field can only identify the simple instructions such as control the emotion, be opened and closed, and not be suitable for hand still
Complex operations.And it is at high cost, complicated order discrimination is low, the device is complicated for operation, individual brain wave diversity factor is big.So in brain
Before electric wave control technology is mature, needs more intelligentized prosthetic hand control system and disabled user is helped to better use artificial limb
Hand.
Summary of the invention
Aiming at the above defects or improvement requirements of the prior art, the present invention provides one kind is known based on voice control and vision
Other intelligent artificial limb hand and its system and method, it is intended that offer combined voice control with visual identity function
Intelligent artificial limb hand, and prosthetic hand three-level operation control system and its implementation based on Internet of Things, thus solve existing master
Stream prosthetic hand only relies on myoelectric control, traction control, and control efficiency is low, electromyography signal interference is big, operation training complexity problem,
And existing prosthetic hand speech control system has a single function, is unable to complete complicated gesture operation, intelligent low technical problem.
To achieve the above object, according to one aspect of the present invention, a kind of intelligent artificial limb hand is provided.
Intelligent artificial limb hand, using Three Degree Of Freedom prosthetic hand ontology, including finger component, palm component, residual the upper arm member, myoelectricity
Signal transducer and power supply for power supply, wherein each joint of each finger and connection palm component in finger component with
It is equipped with motor at the wrist of residual the upper arm member, for driving the rotation at each joint of each finger and wrist, which is characterized in that also wrap
Include depth camera head module, voice interaction module, LED touch screen, core processing module, memory module and wireless charging mould
Block, wherein
The depth camera head module is arranged at the palm on the palm component, for obtaining image data, and passes
It send to the core processing module；
The voice interaction module is used to carry out interactive voice with user, and the voice data that will acquire is sent to the core
Processing module, and play the voice document of the specified format obtained from core processing module；
The LED touch screen is arranged at the back of the hand on the palm component or in residual the upper arm member, inputs behaviour for user
Make instruction and display data；
The core processing module is used to described image data carrying out three-dimensional alignment by image algorithm, pieces together out gesture
The model of movement, and then obtain the angle rotated required for each timing node of each motor, and export machine instruction code；It is also used to
The gesture motion in the memory module is called according to the voice data and to export machine instruction code or newly-increased gesture dynamic
Make to the memory module；Be also used to the operational order inputted according to user by LED touch screen, export machine instruction code or
Person exports display data to LED touch screen；The machine instruction code is for controlling each motor action, so that intelligent artificial limb hand is made
Specified gesture motion；
For the memory module for saving data, which includes gesture motion, the service condition of each gesture motion, figure
As data, voice data；The service condition of each gesture motion includes using time, scene, the frequency；
The wireless charging module is for using wireless charging mode for the power source charges；
The most skin-material of intelligent artificial limb hand is conductor material, so being capable of operation of capacitor screen.
Preferably, above-mentioned intelligent artificial limb hand further includes locating module and rhythm of the heart module, and the locating module is for obtaining
It takes intelligent artificial limb hand current geographic position information and is transmitted to the core processing module；The rhythm of the heart module is for monitoring
The heart rate of user, and heart rate data is transmitted to the core processing module；The core processing module is to the geographical location
Information and heart rate data are transmitted to the memory module and are stored after being handled, or by described after receiving instruction
LED touch screen or the voice interaction module feed back to user.
It is another aspect of this invention to provide that providing a kind of Internet of Things intelligent artificial limb based on voice control and visual identity
Hand system.It is characterized in that, by the cloud server end, secondary operation terminal and intelligent artificial limb hand of wirelessly establishing connection
Composition, wherein
The secondary operation terminal includes the client being mounted on smart phone or computer and/or motion capture gloves；
The Cloud Server is used to provide cloud service for the client and intelligent artificial limb hand, and the cloud service includes online
Storage, online technique is supported and cloud computing.
It is another aspect of this invention to provide that providing a kind of gesture motion based on above-mentioned Internet of Things intelligent artificial limb hand system
Implementation method, which comprises the steps of:
Step 1: information collection being carried out by intelligent artificial limb hand, and is uploaded to cloud server end；
Step 2: calling image recognition/Language Processing basic technology of Baidu AI open platform to connect by cloud server end
Mouthful, it is identified, and recognition result is sent to the core processing module of intelligent artificial limb hand；
Step 3: core processing module combination Cloud Server identification information, the at this time status information of intelligent artificial limb hand and ratio
To the service condition of previous each gesture motion, the gesture motion that most probable is used is inferred；The status information of intelligent artificial limb hand
Refer to intelligent artificial limb hand locating at present spatial position, finger form, palm form etc.；The use feelings of previous each gesture motion
Condition packet expands using time, position, scene, frequency, user's heart rate etc.；
Step 4: the gesture motion that will infer that is sent to LED touch screen and selects for user；
Step 5: user-selected number is exported into machine instruction code, control according to core processing module, core processing module is fed back to
The motor action at each joint of each finger and wrist is made, intelligent artificial limb hand is made to make specified gesture motion.
Preferably, in step 1, the mode of information collection are as follows: shoot two by the depth camera head assembly of intelligent artificial limb hand
Tie up scene and item pictures；
Correspondingly, in step 2, recognition result is respective objects and scene.
Preferably, in step 1, the mode of information collection are as follows: voice is collected by the voice interaction module of intelligent artificial limb hand
Information simultaneously carries out voice de-noising, is converted to specified format；
Correspondingly, in step 2, recognition result is command text information and voice document；
Command text information is the apparatus control code or respective objects and scene information for controlling motor action；Language
Sound file is used to play by voice interaction module, realizes voice interactive function.
Preferably, in step 1, the mode of information collection are as follows: acquire gesture motion information using motion capture gloves；
Correspondingly, in step 2, identification information is corresponding gesture motion information；
In step 3, gesture that core processing module is directly used the corresponding gesture motion information as most probable
Movement.
Preferably, in step 1, it is uploaded to cloud server end, is replaced with, client is uploaded to；
Correspondingly, it in step 2, is identified by client.
Preferably, in step 3, recognition result includes object and scene, time, place, frequency of use and is supervised according to heart rate
Survey the Heart Rate States of module reflection.
It is another aspect of this invention to provide that providing a kind of gesture motion based on above-mentioned Internet of Things intelligent artificial limb hand system
Long-range control method, which comprises the steps of:
Step S1: motion capture gloves are connect with client wireless, and client and intelligent artificial limb hand are wirelessly connected；
Step S2: gesture motion is demonstrated by motion capture gloves, and records the spatial positional information of gesture, real-time Transmission is extremely
Client；
Step S3: the spatial positional information of gesture is directly transferred to the core processing module of intelligent artificial limb hand by client,
Either it is converted into after apparatus control code or is transmitted to core processing module；
Step S3: core processing module converts apparatus control code for the spatial positional information of gesture and exports, Huo Zhezhi
Output apparatus control code is connect, the motor action at each joint of each finger of real-time control and wrist makes intelligent artificial limb hand specified
Gesture motion, realize the online long-range control of gesture motion.
It is another aspect of this invention to provide that providing a kind of gesture motion based on above-mentioned Internet of Things intelligent artificial limb hand system
Control method of client terminal, which comprises the steps of:
Step P1: being arranged the 3D model of intelligent artificial limb hand in client, and the two is wirelessly connected；
Step P2: user accurately adjusts 3D model in client input data and determines gesture motion；
Step P3: determining gesture motion is converted to apparatus control code by client, and is transmitted to the core of intelligent artificial limb hand
Heart control module；
Step P4: kernel control module exports apparatus control code, controls each motor action, realizes specified gesture motion.
In general, through the invention it is contemplated above technical scheme is compared with the prior art, can obtain down and show
Beneficial effect:
1, by depth camera head module, the two-dimensional image data of object and scene is obtained, is turned by core processing module
Be changed to apparatus control code, control articulations digitorum manus and wrist motor action, realize view-based access control model identification to prosthetic hand gesture motion
Control；Voice data is acquired by voice interaction module, apparatus control code is converted to by core processing module, realizes and is based on language
The control to prosthetic hand gesture motion of sound identification, and gesture motion can be increased newly with voice to maneuver library；It is real by LED touch screen
Current family interaction, user experience are good；Greatly improve the intelligence of prosthetic hand, it can be achieved that complicated continuous action control,
It overcomes that the recognition efficiencies such as traditional electromyography signal are low, cannot achieve the defect of compound action；Using wireless charging module, compared to biography
The wired charging mode of system, the present invention avoid the constraint of charging cable and socket with with filling；Prosthetic hand power loss can not
It convenient can also charge in the case of the case where movement；Reduce exposed charging interface, improves the waterproof performance of prosthetic hand；
2, it by cloud server end, secondary operation terminal, the three-level operating system of intelligent artificial limb hand building, realizes to artificial limb
The off-line operation of hand, networking operation mixed mode, to adapt to different demand environments；It, can be not for offline local operation
Under networking situation, by calling intelligent artificial limb hand own module to realize prosthetic hand functional requirement in time；The case where connecting network
Under, recognition effect and interactive experience are promoted by cloud computing, AI open platform interface；It is handed over cloud server end and secondary operation end
Change transmission data；Long-range assistance immediately is realized with the operating side for being connected with motion capture gloves by network；
3, a variety of gesture motion implementation methods based on intelligent artificial limb hand system of the present invention are provided, by Cloud Server,
Client, action gesture gloves keep the resources bank of gesture motion infinitely abundant, the control of gesture motion using Internet resources
Mode, implementation are varied, realize data sharing and long-range assistance immediately, greatly provide convenient and service for user.
Detailed description of the invention
Fig. 1 is the structural schematic diagram at intelligent artificial limb hand palm visual angle in the embodiment of the present invention 1；
Fig. 2 is the structural schematic diagram at intelligent artificial limb hand the back of the hand visual angle in the embodiment of the present invention 1；
Fig. 3 is the framework and data flow diagram of intelligent artificial limb hand action control in the embodiment of the present invention 1；
Fig. 4 is the physical structure figure of intelligent artificial limb hand three-level operating system in the embodiment of the present invention 2；
In all the drawings, identical appended drawing reference is used to denote the same element or structure, in which:
1, voice interaction module 2, depth camera head module 3, locating module
4, rhythm of the heart module 5, LED touch screen 6, wireless communication module
7, wireless charging receiving terminal 8, core processing module 9, power supply
10, electromyography signal sensor 11, memory module
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as in addition, technical characteristic involved in the various embodiments of the present invention described below
Not constituting a conflict with each other can be combined with each other.
Embodiment 1:
As depicted in figs. 1 and 2, intelligent artificial limb hand getting uses Three Degree Of Freedom prosthetic hand as intelligent artificial limb hand ontology comprising
Finger component, palm component, residual the upper arm member, power supply and memory module for power supply, wherein each hand in finger component
Refer to each joint and connect and be equipped with motor at palm component and the wrist of residual the upper arm member, for driving each joint of each finger and hand
Rotation at wrist.
On above-mentioned Three Degree Of Freedom prosthetic hand ontology, it is additionally provided with voice interaction module 1, depth camera head module 2, LED
It touch screen 5, wireless communication module 6, wireless charging module 7, core processing module 8, power supply 9, electromyography signal sensor 10 and deposits
Memory modules 11, wherein
Voice interaction module 1 is for realizing voice interactive function, by collecting voice messaging and to the voice document of generation
Preliminary voice de-noising is carried out, voice document is then converted into specified format, the acquisition of voice messaging is completed, after the completion of acquisition
Give voice document to core processing module；The voice document for the specified format for being also used to obtain from core processing module plays
It goes out, to realize the push of voice messaging；
Depth camera head module 2 is arranged at the palm on the palm component, for obtaining image data, and is sent to
Core processing module；The hand motion that depth camera 2 can capture other people carries out record learning by imitation, acts for user gesture
Library increases new gesture, provides method quickly and efficiently；In conjunction with Cloud Server/client/core processing module, can identify
Object and scene obtain corresponding alternative action, are presented on LED touch screen and user is facilitated to select in time.Depth camera mould
Block 2 provides the method by visual information manipulation and control gesture for user, and perfect voice control describes compound action
The deficiency of aspect.According to the difference for the spatial digitizer scanning angle range that depth camera is equipped with, for individual gesture motions
The relative position Multiple-Scan of convertible depth camera and object, to obtain accurate gesture motion.
In the present embodiment, depth camera head module 2 uses TOF (Time of Flight, flight time) technology.TPF swashs
Optical transmitting set issues modulated near infrared light, meets object back reflection；Sensor is poor by calculating light transmitting and reflection interval
Or phase difference, come the distance of scenery of being taken that converts, to generate depth information；Combining camera shooting, by the three-D profile of object
The topographic map mode for representing different distance in different colors shows.
LED touch screen 5 is arranged at the back of the hand on the palm component or in residual the upper arm member, inputs operation for user and refers to
Enable and show data.It can show the daily information such as time, heart rate, electricity and the gesture that the current most probable of user uses；It can set
Set the parameter of intelligent artificial limb hand；Can input control intelligent artificial limb hand operational order etc..
Wireless communication module 6 by using including but not limited to wifi, 3G, 4G, bluetooth, NFC wireless communication technique, with
Cloud server and mobile phone/PC establish connection, realize the functions such as the interactions of data, position be shared.
Wireless charging module including transmitting terminal and is built in the wireless of prosthetic hand for realizing the wireless charging to power supply 9
Charging receiving end 7.Power supply 9 is used to power for intelligent artificial limb hand, including each module and motor need to electric energy.Have compared to traditional
Line charge power mode, the present invention avoid the constraint of charging cable and socket with with filling；It can not be acted in prosthetic hand power loss
It convenient can also charge in the case of situation；Reduce exposed charging interface, improves the waterproof performance of prosthetic hand.
Core processing module 8 is for realizing overall task schedule and parameter transmitting.Include:
(1) image data is subjected to three-dimensional alignment by image algorithm, pieces together out the model of gesture motion, and then obtained every
The angle rotated required for a each timing node of motor, and export machine instruction code；
(2) voice document acquired according to interactive module 1 calls the gesture motion in memory module and exports machine and refers to
Code is enabled, completes corresponding control operation or newly-increased gesture motion to memory module；
(3) operational order that LED touch screen 5 inputs is passed through according to user, exports machine instruction code or output display number
According to LED touch screen 5；Machine instruction code is for controlling each motor action, so that intelligent artificial limb hand makes specified gesture motion；
(4) it will receive or processed data be transmitted to memory；After receiving existing gesture motion instruction, calling is deposited
The gesture machine instruction code control motor action saved in reservoir completes gesture；Call the audio and figure of memory module storage
As video, and play instruction is issued to LED touch screen or voice interaction module；The data that memory module is saved, pass through nothing
Line communication module 6 is uploaded to mobile phone/pc client and Cloud Server.
Electromyography signal sensor 10 is set to the junction of residual the upper arm member Yu user itself arm, can measure patient's deformed limb
When contraction of muscle, in the small electric potential difference that skin surface generates, is amplified through the amplifier in electrode, becomes control signal,
Input core processing module；It is handled by core processing module 8 and is machine instruction code and issues instruction, closed by setting in each finger
The movements such as each finger-joint of motor driven intelligent artificial limb hand, palm, residual arm at section and wrist.Recognition capability it is low but feedback and
When, it is able to achieve simple action operation.
Memory module 11 is for saving data；Record the service condition of each gesture motion, including use the time, scene,
The frequency；Save the audio that voice interaction module 1 records；Save the image information etc. of depth camera head module record；
Depending on the actual installation position of each module and device can also be according to situation.Particularly, it is different from conventional prosthesis wrist-watch skin
The most surface material of the ambroin rubber material used, the prosthetic hand is conductor material, enables a user to directly use artificial limb
Manual operating capacitance plate.
As a preferred embodiment, intelligent artificial limb hand can also include locating module 3 and rhythm of the heart module 4,
Wherein,
Locating module 3 obtains the geographical location information of user by calling the positional interface API of intelligent artificial limb hand；It is geographical
Location information can upload to Cloud Server by user account cryptoguard, pass through after establishing wireless connection with Cloud Server
Positioning has the function that anti-lost and determines that user location ensures safety；Geographical location information can also be transmitted at core
Module is managed, memory module is sent to by core processing module 8 and is stored, or receives phonetic order or is touched from LED
After the operational order for touching screen, voice interaction module or LED touch screen are sent to by core processing module 8, carry out voice broadcast or
Person's data are shown；
Rhythm of the heart module 4 is used to monitor the heart rate of user, and heart rate data is transmitted to core processing module 8；By core
Heart processing module 8 judges the mood of user and health status and feeds back to user；Arrhythmia cordis etc. is occurring in emergency circumstances,
Early warning is issued by voice interaction module 1 or LED touch screen 5.
Core processing module 8 is transmitted to the progress of memory module 11 after handling geographical location information and heart rate data
Storage, or user is fed back to by LED touch screen or voice interaction module after receiving instruction.
As shown in figure 3, be intelligent artificial limb hand provided by the present invention framework and data flowchart, image action signal by
Motion capture gloves obtain；Voice signal is obtained by voice interaction module；Electromyography signal is obtained by electromyography signal sensor；Client
It holds and data command can be transmitted with Cloud Server；Locating module and rhythm of the heart module obtain position and heart rate；Core processing
Device controls motor action to apparatus control code is converted to after data processing；Data can be stored to memory module；It can be with
Server, AI interface exchange data by wireless telecommunications.
Method provided in this embodiment, prosthetic hand gesture function have compared to the prior art through voice control, vision
The characteristics of identifying newly-increased gesture.Low to Network status dependence, user experience is good under off-line operation, but based on offline
It calculates, height is required to the processing capacity of equipment, the consumption of power supply is larger.
Embodiment 2:
As shown in figure 4, the Internet of Things intelligent artificial limb hand system based on voice control and visual identity, by wirelessly
Establish cloud server end, secondary operation terminal and intelligent artificial limb the hand composition of connection, wherein
Intelligent artificial limb hand of the intelligent artificial limb hand getting in embodiment 1.
Secondary operation terminal includes the client for being used to control intelligent artificial limb hand being mounted on smart phone or computer,
And/or motion capture gloves, wherein
Client can be established with Cloud Server and intelligent artificial limb hand and is wirelessly connected；Client is built-in with the 3D mould of prosthetic hand
Type, user can accurately be adjusted with input data determines gesture, is converted to apparatus control code by client；Client can connect dynamic
Make to capture gloves acquisition motion model, the action message for analyzing motion model is converted to apparatus control code；It will by wireless telecommunications
Apparatus control code is transferred to intelligent artificial limb hand, deposits to memory module；Apparatus control code is for controlling each motor action, so that intelligence
It can the specified gesture motion of prosthetic hand completion；
Motion capture gloves, the motion capture technology based on inertial sensor；Its measuring unit (IMU) is by inertia sensing
Device and data processing unit composition.The semaphore that its system acquisition arrives is few, and convenient for completing Attitude Tracking task in real time, resolving is obtained
Posture information range is big, high sensitivity, dynamic property are good, and inertial sensor it is small in size, convenient for wearing, it is cheap.It is excellent
Choosing, there is wireless communication module, can connect applications client and prosthetic hand, posture information is transmitted.There are two types of use for tool
Mode: acquiring and save offline, in case using；It acquires in real time and to be transmitted to prosthetic hand end online help prosthetic hand user real-time
More complex task is completed in operation.
Cloud server end provides on-line storage, online technique support and cloud computing for applications client and intelligent artificial limb hand
Deng service.Applications client and prosthetic hand operating side can upload gesture motion, use habit data to server, be backed up
It is shared；Can also be uploaded with download technology personnel and other users gesture motion to the prosthetic hand shared carry out using；Optionally may be used
To provide paid sharing and downloading to gesture motion, gesture motion shop is formed；The online friendship of arm disabled patient can be constructed
Flow community；Cloud Server provides gesture, picture, voice content matching primitives；Cloud server end can be the case where agreeing to through user
Under the position of user, heart rate are monitored, be used in combination data user health situation is assessed, emergency carry out
Early warning；Periodically push prosthetic hand firmware upgrade；On-line annealing diagnosis investigation service is provided.
Cloud computing refers to: the three-dimensional motion information, pre- according to scene that will be obtained by depth camera, motion capture gloves
Survey judges that most probable is used this portion of techniques of movement to give Cloud Server and calculated, and reduces core processing module calculation amount.
In use, (1) voice control prosthetic hand function: the core processing module of intelligent artificial limb hand is obtained from voice interaction module
Take the voice document of formatting；In the case where connecting network, the emergency help information conversion distress signal of extraction is passed through into nothing
Line communication module is sent to emergency contact and cloud；Winged or Baidu's speech interface is interrogated by calling, and routing information is converted into language
Message breath；The routing information for being converted into voice passes to voice interaction module；Call Baidu AI open flat by Cloud Server
The Language Processing basic technology interface of platform can carry out exchanging talk with user, complete more complicated, specified based on cloud service
Gesture task；
In off-line case, winged or Baidu's speech interface is interrogated by calling and converts text information for voice messaging；From obtaining
Key message is extracted in the text information taken, calls control corresponding module；Such as: by the gesture motion information of extraction, being converted into machine
Device instruction code controls the corresponding motor rotation of wrist and finger etc., and then makes corresponding gesture motion；By the component shape of extraction
State order is converted into the switch of control signal control such as depth camera, LED touch screen；In off-line case, interactive voice
It is realized by the VoP of intelligent artificial limb hand local.
(2) image information controls prosthetic hand function: the core processing module of intelligent artificial limb hand obtains deep from depth camera
It spends image (depth image)；In the case where connecting network, the image of Baidu AI open platform is called to know by Cloud Server
Other interface, identifies respective objects and scene；Core processing module combination Cloud Server comparing and user's use habit,
Infer the gesture motion that most probable is used；The gesture motion of supposition is sent to LED touch screen screen fast to select for user；It connects
By the corresponding gesture motion that user selects, gesture motion information is fed back into core processing module；It is converted by core processing module
For apparatus control code, corresponding motor movement is controlled, prosthetic hand is made to make desired gesture motion；
In off-line case, core processing module pieces together out the complete mould of gesture motion by image algorithm three-dimensional alignment
Type, and then obtain the angle rotated required for each timing node of each joint motor；It is converted into machine instruction code control wrist
And the corresponding motor rotation of finger etc., and then make corresponding gesture motion.
(3) scene intelligence aspect: Cloud Server/client/core processing module passes through study user's use habit, in conjunction with
Different time sections, position scene, user's heart rate etc., pass through intelligent algorithm, thus it is speculated that the gesture motion that most probable is used is sent to
LED touch screen is selected for user.
(4) in terms of wireless telecommunications: passing through the wireless communication module and mobile phone/pc client and Cloud Server of intelligent artificial limb hand
Establish connection；Artificial limb hand position, institute's heart rate measuring data are uploaded in real time in the case where permitting through user；From mobile phone/pc client and cloud
Server downloads gesture motion to memory module；Newly-built gesture motion is uploaded to save to mobile phone/pc client and Cloud Server；
Obtain online support and firmware upgrade from Cloud Server；Connecting moves captures gloves, obtains motion model；Analysis movement mould
Type obtains apparatus control code, control joint motor movement；By the Language Processing base of interactive voice audio and Baidu AI open platform
The connection of plinth technical interface；The audio of user is uploaded to interface；Receive the voice of Language Processing basic technology interface feedback；Control
Voice interaction module plays；The object scene pictorial information that camera obtains is connect by the image recognition of Baidu's AI open platform
Mouth uploads；Obtain the image recognition result of image recognition interface；Image recognition result is uploaded to Cloud Server, application scenarios with
User's use habit compares matching gesture motion option；It receives matching gesture motion option control motor and completes operation.
The intelligent artificial limb hand three-level operating system built through the invention realizes off-line operation, networking to prosthetic hand
Mixed mode is operated, to adapt to different demand environments.For offline local operation, tune can be passed through under not networking situation
Meet basic function demand in time with intelligent artificial limb hand own module；It is open by cloud computing, AI in the case where connecting network
Platform interface promotes recognition effect and interactive experience；Transmission data are exchanged with cloud server end and secondary operation end；By network
Long-range assistance immediately is realized with the operating side for being connected with motion capture gloves.
Based on the Cloud Server platform of cloud technology of sharing, make user can not only by the gesture motion of oneself typing, daily make
It is uploaded with habit data, prevents loss of data, can more be shared and downloaded by technical staff and other users production upload
Gesture motion, formed arm disabled user share gesture motion exchange community.The work service personnel of cloud platform can also be with
Periodically equipment is upgraded online, malfunction elimination and technical support.This platform can also serve different disabled classes
Other intelligence is help the disabled the user of equipment.Building for this exchange community's platform can also promote exchange, improve individuals with disabilities from
I accepts, and excites the vigor in disabled person market.
Method provided in this embodiment forms intelligent artificial limb hand three-level operating system gesture function, passes through Cloud Server
Calculating task is shared, the computing capability requirement to core processing module, low manufacture cost are reduced；Cloud service resource can be all
User shares, and shares so that the affluent resources such as gesture motion are shared.
Embodiment 3:
The gesture motion implementation method of the present embodiment provides a kind of Internet of Things intelligent artificial limb hand system based on embodiment 2,
Include the following steps:
Step 1: two-dimensional scene and item pictures being shot by the depth camera head assembly of intelligent artificial limb hand, and are uploaded to cloud
Server end；
Step 2: calling the image recognition interface of Baidu AI open platform by cloud server end, identified, and will know
The respective objects and scene that do not obtain are sent to the core processing module of intelligent artificial limb hand；
Step 3: core processing module combination Cloud Server identification information and the use for comparing previous each gesture motion
Situation infers the gesture motion that most probable is used；
Step 4: the gesture motion that will infer that is sent to LED touch screen and selects for user；
Step 5: user-selected number is exported into machine instruction code, control according to core processing module, core processing module is fed back to
The motor action at each joint of each finger and wrist is made, intelligent artificial limb hand is made to make specified gesture motion.
As an alternative embodiment, being uploaded to cloud server end in the present embodiment step 1, may be replaced by
It is uploaded to the client being mounted on smart phone or computer；Correspondingly, it in step 2, is identified by client, and will
Identify that obtained respective objects and scene are sent to the core processing module of intelligent artificial limb hand；Step 3-5 is constant.
Embodiment 4:
The gesture motion implementation method of the present embodiment provides a kind of Internet of Things intelligent artificial limb hand system based on embodiment 2,
Include the following steps:
Step 1: voice messaging being collected by the voice interaction module of intelligent artificial limb hand and carries out voice de-noising, is converted to finger
The formula that fixes simultaneously is uploaded to cloud server end；
Step 2: calling the Language Processing basic technology interface of Baidu AI open platform by cloud server end, known
Not, and by command text information and voice document that identification obtains it is sent to the core processing module of intelligent artificial limb hand；
Step 3: core processing module combination Cloud Server recognition result, the at this time status information of intelligent artificial limb hand and ratio
To the service condition of previous each gesture motion, the gesture motion that most probable is used is inferred；
Step 4: the gesture motion that will infer that is sent to LED touch screen and selects for user；
Step 5: user-selected number is exported into machine instruction code, control according to core processing module, core processing module is fed back to
The motor action at each joint of each finger and wrist is made, intelligent artificial limb hand is made to make specified gesture motion.
As an alternative embodiment, being uploaded to cloud server end in the present embodiment step 1, may be replaced by
It is uploaded to the client being mounted on smart phone or computer；Correspondingly, it in step 2, is identified by client, and will
Identify that obtained command text information and voice document are sent to the core processing module of intelligent artificial limb hand；Step 3-5 is constant.
The method that embodiment 3,4 provides is realized the function of intelligently being inferred gesture based on scene, reduces the operation of user
Amount, improves intelligence degree.
Embodiment 5:
The gesture motion implementation method of the present embodiment provides a kind of Internet of Things intelligent artificial limb hand system based on embodiment 2,
Include the following steps:
Step 1: acquiring gesture motion information using motion capture gloves, and be uploaded to cloud server end；
Step 2: being identified by cloud server end, and the corresponding gesture motion information that identification obtains is sent to intelligence
The core processing module of energy prosthetic hand；
Step 3: corresponding gesture motion is sent to LED touch screen and selected for user by core processing module；
Step 4: user-selected number exports machine instruction code, control according to core processing module, core processing module is fed back to
Motor action at each joint of each finger and wrist, makes intelligent artificial limb hand make specified gesture motion.
Optionally, step 5: saving gesture motion information to memory module or real-time Transmission and synchronize gesture motion to cloud
Server-side.
As a preferred embodiment, being uploaded to cloud server end in the present embodiment step 1, may be replaced by
It is uploaded to the client being mounted on smart phone or computer；Correspondingly, it in step 2, is identified by client, and will
Identify that obtained corresponding gesture motion information is sent to the core processing module of intelligent artificial limb hand；Other steps are constant.
As another embodiment, in the present embodiment step 1, it is uploaded to cloud server end, may be replaced by intelligence
Client on prosthetic hand；Correspondingly, it in step 2, is identified by core processing module；Other steps are constant.
Embodiment 6:
The gesture motion remote control method of the present embodiment provides a kind of Internet of Things intelligent artificial limb hand system based on embodiment 2
Method includes the following steps:
Step 1: motion capture gloves are connect with client wireless, and client and intelligent artificial limb hand are wirelessly connected；
Step 2: demonstrating gesture motion by motion capture gloves, and record the spatial positional information of gesture, real-time Transmission is extremely
Client；
Step 3: the spatial positional information of gesture is directly transferred to the core processing module of intelligent artificial limb hand by client, or
Person is converted into after apparatus control code or is transmitted to core processing module；
Step 3: core processing module converts apparatus control code for the spatial positional information of gesture and exports, or directly
Apparatus control code is exported, the motor action at each joint of each finger of real-time control and wrist makes intelligent artificial limb hand specified
Gesture motion realizes the online long-range control of gesture motion.
Method provided in this embodiment realizes the online long-range control of gesture motion, plays efficient in emergency
The effect of timely remote assistance prosthetic hand user.
Embodiment 7:
The gesture motion remote control method of the present embodiment provides a kind of Internet of Things intelligent artificial limb hand system based on embodiment 2
Method includes the following steps:
Step 1: the 3D model of intelligent artificial limb hand is set in client, and client and intelligent artificial limb hand are wirelessly connected；
Step 2: user accurately adjusts 3D model in client input data and determines gesture motion；
Step 3: determining gesture motion is converted to apparatus control code by client, and is transmitted to the core of intelligent artificial limb hand
Control module；
Step 4: kernel control module exports apparatus control code, controls each motor action, realizes specified gesture motion.
Embodiment 3-7 provides the preferred implementation side of several gesture implementation methods based on intelligent artificial limb hand system of the present invention
Formula, but to be not limited only to this several for the gesture implementation method based on intelligent artificial limb hand and its system of the invention.
Internet of Things intelligent artificial limb hand system based on voice control and visual identity of the invention is compared to myoelectric control vacation
Limb hand, has the characteristics that intelligentized, also has compared to single its intelligence degree of voice control prosthetic hand technology and significantly mentions
It rises；While remaining myoelectric control and fixed movement voice control, there is the autonomous typing personalization gesture of user, continuously move
Work energy allows users to expand gesture and maneuver library according to itself daily life needs；Pass through motion capture gloves and depth
Camera is spent, so that complicated continuous movement typing is also achieved, such as plays an instrument and (plays the musical instrument), drawing, play magic square etc..
The intelligent artificial limb hand is not only able to satisfy the demand of user's daily life, and it is residual more to bring into the arm such as professional skill and artistic treat
It in the life of disease user, really improves the quality of living, increases the joy of life.
In terms of usage experience: the intelligent artificial limb hand system operation logic in the present invention is clear；Compared to myoelectric limb hand
The Complex learning of the brain wave control technology operation of complicated manipulation training and forward position after exercise for power and dress limb before filling limb
Training, it is convenient to be easy to upper hand.
In terms of recognition efficiency: myoelectric control is low for the recognition efficiency of compound action at present；Brain wave is controlled due to technology
It is immature, individual difference is big, same recognition efficiency is low；And the Internet of Things based on voice control and visual identity that the present invention opens altogether
Disabled intelligent artificial limb hand system has used language and image recognition technology, motion capture technology, has recognition efficiency height, man-machine friendship
Mutually good feature.
In terms of cost: the functional module such as interactive voice mould that the more traditional Three Degree Of Freedom myoelectric limb hand increase of this equipment uses
Block, locating module, rhythm of the heart module, wireless communication module, memory module, power supply and wireless charging module etc., are being realized
Has the characteristics that at low cost, function-stable compared to schemes such as brain wave controls under the premise of intelligentized.
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to
The limitation present invention, any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should all include
Within protection scope of the present invention.
Claims (10)
1. intelligent artificial limb hand is believed using Three Degree Of Freedom prosthetic hand ontology, including finger component, palm component, residual the upper arm member, myoelectricity
Number sensor, power supply and memory module for power supply, wherein each joint of each finger and connection hand in finger component
It is equipped with motor at the wrist of metacarpus part and residual the upper arm member, for driving the rotation at each joint of each finger and wrist, feature
It is, further includes depth camera head module, voice interaction module, LED touch screen, core processing module and wireless charging module,
Wherein,
The depth camera head module is arranged at the palm on the palm component, for obtaining image data, and is sent to
The core processing module；
The voice interaction module is used to carry out interactive voice with user, and the voice data that will acquire is sent to the core processing
Module, and play the voice document of the specified format obtained from core processing module；
The LED touch screen is arranged at the back of the hand on the palm component or in residual the upper arm member, inputs operation for user and refers to
Enable and show data；
The core processing module is used to described image data carrying out three-dimensional alignment by image algorithm, pieces together out gesture motion
Model, and then obtain the angle rotated required for each timing node of each motor, and export machine instruction code；It is also used to basis
The voice data calls the gesture motion in the memory module and exports machine instruction code or newly-increased gesture motion extremely
The memory module；It is also used to the operational order inputted according to user by LED touch screen, exports machine instruction code or defeated
Show data to LED touch screen out；The machine instruction code for controlling each motor action so that intelligent artificial limb hand make it is specified
Gesture motion；
The wireless charging module is for using wireless charging mode for the power source charges；
For the memory module for saving data, which includes gesture motion, the service condition of each gesture motion, picture number
According to, voice data；
The most skin-material of intelligent artificial limb hand is conductor material.
2. intelligent artificial limb hand as described in claim 1, which is characterized in that further include locating module and rhythm of the heart module, institute
Locating module is stated for obtaining intelligent artificial limb hand current geographic position information and being transmitted to the core processing module；The heart rate
Monitoring modular is used to monitor the heart rate of user, and heart rate data is transmitted to the core processing module；The core processing mould
Block is transmitted to the memory module and stores after handling the geographical location information and heart rate data, or is receiving
User is fed back to by the LED touch screen or the voice interaction module after instruction.
3. the Internet of Things intelligent artificial limb hand system based on voice control and visual identity, which is characterized in that by wirelessly
Establish cloud server end, secondary operation terminal and intelligent artificial limb the hand composition of connection, wherein
The secondary operation terminal includes the client being mounted on smart phone or computer and/or motion capture gloves；
The Cloud Server is used to provide cloud service for the client and intelligent artificial limb hand, and the cloud service includes depositing online
Storage, online technique is supported and cloud computing.
4. the gesture motion implementation method based on Internet of Things intelligent artificial limb hand system as claimed in claim 3, which is characterized in that packet
Include following steps:
Step 1: information collection being carried out by intelligent artificial limb hand, and is uploaded to cloud server end；
Step 2: the image recognition or Language Processing basic technology interface of Baidu AI open platform are called by cloud server end, into
Row identifies, and recognition result is sent to the core processing module of intelligent artificial limb hand；
Step 3: core processing module combination Cloud Server identification information, the at this time status information of intelligent artificial limb hand and compare with
The service condition of past each gesture motion infers the gesture motion that most probable is used；
Step 4: the gesture motion that will infer that is sent to LED touch screen and selects for user；
Step 5: user-selected number being exported into machine instruction code according to core processing module, core processing module is fed back to, control is each
Motor action at each joint of finger and wrist, makes intelligent artificial limb hand make specified gesture motion.
5. the gesture motion implementation method of Internet of Things intelligent artificial limb hand system as claimed in claim 4, which is characterized in that step
In 1, the mode of information collection are as follows: two-dimensional scene and item pictures are shot by the depth camera head assembly of intelligent artificial limb hand；
Correspondingly, in step 2, calling interface is the image recognition interface of Baidu AI open platform, and recognition result is respective objects
And scene.
6. the gesture motion implementation method of Internet of Things intelligent artificial limb hand system as claimed in claim 4, which is characterized in that step
In 1, the mode of information collection are as follows: voice messaging is collected by the voice interaction module of intelligent artificial limb hand and carries out voice de-noising,
Be converted to specified format；
Correspondingly, in step 2, calling interface is the Language Processing basic technology interface of Baidu AI open platform, and identification information is
Command text information and voice document.
7. the gesture motion implementation method of Internet of Things intelligent artificial limb hand system as claimed in claim 4, which is characterized in that step
In 1, the mode of information collection are as follows: acquire gesture motion information using motion capture gloves；
Correspondingly, in step 2, recognition result is corresponding gesture motion information；
In step 3, gesture motion that core processing module is directly used the corresponding gesture motion information as most probable.
8. the gesture motion implementation method of the Internet of Things intelligent artificial limb hand system as described in claim 4-6 is any, feature exist
In in step 1, being uploaded to cloud server end, replace with, be uploaded to client；
Correspondingly, it in step 2, is identified by client.
9. based on the gesture motion long-range control method of Internet of Things intelligent artificial limb hand system as claimed in claim 3, feature exists
In including the following steps:
Step S1: motion capture gloves are connect with client wireless, and client and intelligent artificial limb hand are wirelessly connected；
Step S2: demonstrating gesture motion by motion capture gloves, and record the spatial positional information of gesture, real-time Transmission to client
End；
Step S3: the spatial positional information of gesture is directly transferred to the core processing module of intelligent artificial limb hand by client, or
It is converted into after apparatus control code or is transmitted to core processing module；
Step S3: core processing module converts apparatus control code for the spatial positional information of gesture and exports, or directly defeated
Apparatus control code out, the motor action at each joint of each finger of real-time control and wrist, makes intelligent artificial limb hand make specified hand
Gesture movement, realizes the online long-range control of gesture motion.
10. based on the gesture motion control method of client terminal of Internet of Things intelligent artificial limb hand system as claimed in claim 3, feature
It is, includes the following steps:
Step P1: being arranged the 3D model of intelligent artificial limb hand in client, and the two is wirelessly connected；
Step P2: user accurately adjusts 3D model in client input data and determines gesture motion；
Step P3: determining gesture motion is converted to apparatus control code by client, and is transmitted to the core control of intelligent artificial limb hand
Molding block；
Step P4: kernel control module exports apparatus control code, controls each motor action, realizes specified gesture motion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810943869.8A CN109172066B (en) | 2018-08-18 | 2018-08-18 | Intelligent prosthetic hand based on voice control and visual recognition and system and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810943869.8A CN109172066B (en) | 2018-08-18 | 2018-08-18 | Intelligent prosthetic hand based on voice control and visual recognition and system and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109172066A true CN109172066A (en) | 2019-01-11 |
| CN109172066B CN109172066B (en) | 2019-12-20 |
Family
ID=64918396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810943869.8A Active CN109172066B (en) | 2018-08-18 | 2018-08-18 | Intelligent prosthetic hand based on voice control and visual recognition and system and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109172066B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262662A (en) * | 2019-06-20 | 2019-09-20 | 河北识缘信息科技发展有限公司 | A kind of intelligent human-machine interaction method |
| CN110711055A (en) * | 2019-11-07 | 2020-01-21 | 江苏科技大学 | Image sensor intelligence artificial limb leg system based on degree of depth learning |
| CN111399641A (en) * | 2020-03-06 | 2020-07-10 | 苏州通和景润康复科技有限公司 | Upper limb myoelectric artificial limb operating device |
| CN111616847A (en) * | 2020-06-02 | 2020-09-04 | 中国科学技术大学先进技术研究院 | Upper arm prosthesis control system and method based on muscle cooperation and man-machine voice interaction |
| CN112587285A (en) * | 2020-12-10 | 2021-04-02 | 东南大学 | Multi-mode information guide environment perception myoelectricity artificial limb system and environment perception method |
| CN114884729A (en) * | 2022-05-06 | 2022-08-09 | 安徽中电光达通信技术有限公司 | Safe operation control method of Internet of things platform |
| WO2023274119A1 (en) * | 2021-06-30 | 2023-01-05 | 华为技术有限公司 | Touch operation recognition method and apparatus, and related device |
| CN116311385A (en) * | 2023-05-18 | 2023-06-23 | 山东睿芯半导体科技有限公司 | Gesture recognition chip and gesture recognition method thereof |
| WO2024114457A1 (en) * | 2022-11-28 | 2024-06-06 | 中国科学院深圳先进技术研究院 | Multi-mode signal-based upper limb prosthesis control method and system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202128568U (en) * | 2010-12-28 | 2012-02-01 | 上海理工大学 | Human artificial hand controlled by double signal sources |
| JP2013085579A (en) * | 2011-10-13 | 2013-05-13 | Toyoda Gosei Co Ltd | Electrically operated artificial arm |
| CN103349578A (en) * | 2013-06-18 | 2013-10-16 | 沈阳创达技术交易市场有限公司 | Voice artificial hand controller |
| US20140277588A1 (en) * | 2013-03-15 | 2014-09-18 | Eli Robert Patt | System and method for providing a prosthetic device with non-tactile sensory feedback |
| CN105278817A (en) * | 2014-05-30 | 2016-01-27 | 金兆栋 | Devices for controlling voice and lip language, and control method for voice and lip language |
| CN105361983A (en) * | 2015-09-14 | 2016-03-02 | 张英华 | Prosthesis mechanical arm, robot and robot control method |
| CN106125925A (en) * | 2016-06-20 | 2016-11-16 | 华南理工大学 | Method is arrested based on gesture and voice-operated intelligence |
| CN106228982A (en) * | 2016-07-27 | 2016-12-14 | 华南理工大学 | A kind of interactive learning system based on education services robot and exchange method |
| CN106909216A (en) * | 2017-01-05 | 2017-06-30 | 华南理工大学 | A kind of Apery manipulator control method based on Kinect sensor |
-
2018
- 2018-08-18 CN CN201810943869.8A patent/CN109172066B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202128568U (en) * | 2010-12-28 | 2012-02-01 | 上海理工大学 | Human artificial hand controlled by double signal sources |
| JP2013085579A (en) * | 2011-10-13 | 2013-05-13 | Toyoda Gosei Co Ltd | Electrically operated artificial arm |
| US20140277588A1 (en) * | 2013-03-15 | 2014-09-18 | Eli Robert Patt | System and method for providing a prosthetic device with non-tactile sensory feedback |
| CN103349578A (en) * | 2013-06-18 | 2013-10-16 | 沈阳创达技术交易市场有限公司 | Voice artificial hand controller |
| CN105278817A (en) * | 2014-05-30 | 2016-01-27 | 金兆栋 | Devices for controlling voice and lip language, and control method for voice and lip language |
| CN105361983A (en) * | 2015-09-14 | 2016-03-02 | 张英华 | Prosthesis mechanical arm, robot and robot control method |
| CN106125925A (en) * | 2016-06-20 | 2016-11-16 | 华南理工大学 | Method is arrested based on gesture and voice-operated intelligence |
| CN106228982A (en) * | 2016-07-27 | 2016-12-14 | 华南理工大学 | A kind of interactive learning system based on education services robot and exchange method |
| CN106909216A (en) * | 2017-01-05 | 2017-06-30 | 华南理工大学 | A kind of Apery manipulator control method based on Kinect sensor |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110262662A (en) * | 2019-06-20 | 2019-09-20 | 河北识缘信息科技发展有限公司 | A kind of intelligent human-machine interaction method |
| CN110711055A (en) * | 2019-11-07 | 2020-01-21 | 江苏科技大学 | Image sensor intelligence artificial limb leg system based on degree of depth learning |
| CN111399641A (en) * | 2020-03-06 | 2020-07-10 | 苏州通和景润康复科技有限公司 | Upper limb myoelectric artificial limb operating device |
| CN111616847A (en) * | 2020-06-02 | 2020-09-04 | 中国科学技术大学先进技术研究院 | Upper arm prosthesis control system and method based on muscle cooperation and man-machine voice interaction |
| CN111616847B (en) * | 2020-06-02 | 2021-08-03 | 中国科学技术大学先进技术研究院 | Upper arm prosthesis control system and method based on muscle cooperation and man-machine voice interaction |
| CN112587285A (en) * | 2020-12-10 | 2021-04-02 | 东南大学 | Multi-mode information guide environment perception myoelectricity artificial limb system and environment perception method |
| WO2023274119A1 (en) * | 2021-06-30 | 2023-01-05 | 华为技术有限公司 | Touch operation recognition method and apparatus, and related device |
| CN114884729A (en) * | 2022-05-06 | 2022-08-09 | 安徽中电光达通信技术有限公司 | Safe operation control method of Internet of things platform |
| CN114884729B (en) * | 2022-05-06 | 2023-07-14 | 安徽中电光达通信技术有限公司 | Safe operation control method of Internet of things platform |
| WO2024114457A1 (en) * | 2022-11-28 | 2024-06-06 | 中国科学院深圳先进技术研究院 | Multi-mode signal-based upper limb prosthesis control method and system |
| CN116311385A (en) * | 2023-05-18 | 2023-06-23 | 山东睿芯半导体科技有限公司 | Gesture recognition chip and gesture recognition method thereof |
| CN116311385B (en) * | 2023-05-18 | 2023-08-15 | 山东睿芯半导体科技有限公司 | Gesture recognition chip and gesture recognition method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109172066B (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109172066A (en) | Intelligent artificial limb hand and its system and method based on voice control and visual identity | |
| US11262841B2 (en) | Wireless wrist computing and control device and method for 3D imaging, mapping, networking and interfacing | |
| US10061389B2 (en) | Gesture recognition system and gesture recognition method | |
| CN108983636B (en) | Man-machine intelligent symbiotic platform system | |
| US20160128450A1 (en) | Information processing apparatus, information processing method, and computer-readable storage medium | |
| CN104102346A (en) | Household information acquisition and user emotion recognition equipment and working method thereof | |
| CN108427910A (en) | Deep-neural-network AR sign language interpreters learning method, client and server | |
| CN104949707A (en) | Movement monitoring equipment and movement monitoring method based on information push | |
| CN110495819A (en) | Control method, robot, terminal, server and the control system of robot | |
| WO2018108176A1 (en) | Robot video call control method, device and terminal | |
| CN105872508A (en) | Projector based on intelligent cell phone and method for presenting multimedia | |
| CN208013683U (en) | Smart home house keeper's control device with body-sensing function | |
| CN109841217A (en) | A kind of AR interactive system and method based on speech recognition | |
| CN113241150A (en) | Rehabilitation training evaluation method and system in mixed reality environment | |
| CN107930087A (en) | A kind of body-building apparatus based on Internet of Things shares ancillary equipment | |
| CN102895093A (en) | Walker aid robot tracking system and walker aid robot tracking method based on RGB-D (red, green and blue-depth) sensor | |
| CN108379817A (en) | Limb rehabilitation training system and method | |
| CN106272446A (en) | The method and apparatus of robot motion simulation | |
| CN108037825A (en) | The method and system that a kind of virtual idol technical ability is opened and deduced | |
| CN116572260A (en) | Emotion communication accompanying and nursing robot system based on artificial intelligence generated content | |
| Yi et al. | [Retracted] Home Interactive Elderly Care Two‐Way Video Healthcare System Design | |
| CN107538492A (en) | Intelligent control system, method and the intelligence learning method of mobile robot | |
| CN110178159A (en) | Audio/video wearable computer system with integrated form projector | |
| CN111259699A (en) | Human body action recognition and prediction method and device | |
| CN106997449A (en) | Robot and face identification method with face identification functions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |