CN212460576U - Tracking processing system for recognizing and representing gestures - Google Patents
Tracking processing system for recognizing and representing gestures Download PDFInfo
- Publication number
- CN212460576U CN212460576U CN202120026852.3U CN202120026852U CN212460576U CN 212460576 U CN212460576 U CN 212460576U CN 202120026852 U CN202120026852 U CN 202120026852U CN 212460576 U CN212460576 U CN 212460576U
- Authority
- CN
- China
- Prior art keywords
- terminal
- depth camera
- processing system
- gestures
- deaf
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The utility model discloses a track processing system for discernment and expression gesture relates to intelligent teaching system technical field, include: main terminal, vice terminal, degree of depth camera, cloud platform, main control unit. The main controller is connected with the main terminal, the auxiliary terminal and the cloud platform, the cloud platform is a rotatable structure, and the depth camera is installed on the cloud platform. This application adopts the mode of reserving vice terminal ID serial number information in advance for when the gesture sensor of vice terminal acquires action information at every turn, main terminal can both lock the position of vice terminal, and then uses the rotatory vice terminal of this of cloud platform control depth camera alignment, consequently makes the depth camera can carry out sign language to the student of arbitrary assigned position in the classroom and discerns. Therefore, when the deaf-mute student encounters a problem, the student can respond timely, effectively and quickly without leaving the seat to go to the lens when playing sign language, and the learning quality effect of the deaf-mute student is greatly enhanced.
Description
Technical Field
The utility model relates to an intelligence teaching system technical field, in particular to a track processing system for discernment and expression gesture.
Background
Sign language is the main language for deaf-mutes. Due to the low prevalence of sign language, it is difficult for them to interact with normal hearing people.
Because of this, researchers have spent much time trying to perform sign language recognition by machine in recent years. A currently common technique is to use a depth camera (Kinect or Leap Motion) to recognize sign language. The sign language is recognized by a depth camera by emitting infrared light through an infrared light source and receiving the infrared light reflected back by the environment and the user. And after three-dimensional imaging is carried out on the user through detecting the phase difference, human body contour recognition and joint positioning are carried out. The user may define the basic action as sign language words by the action read by the depth camera.
A common use of depth cameras is to mount them above a display screen and the user makes the required movements or sign language in front of the display screen.
At present, when a depth camera is used in a teaching system for teaching deaf-mutes, the camera is fixedly installed, so that the recognition range is limited, and the teaching quality is influenced. The current teaching system does not have the function of sending quick messages through simple actions, which is very important for the deaf-mutes, so the current gesture recognition mode is not suitable for the classroom use of the deaf-mutes.
SUMMERY OF THE UTILITY MODEL
The utility model discloses the purpose is to solve the teaching system among the prior art and is unsuitable to the teaching of deaf-mute, influences the problem of teaching quality.
In order to achieve one of the above purposes, the utility model adopts the following technical scheme: a tracking processing system for recognizing and representing gestures, comprising: a master terminal having: the display unit displays the characters converted by the actions of the deaf-mutes; a secondary terminal having: an ID unit, wherein the main terminal pre-stores the number information of the ID unit; the gesture sensor is used for acquiring and identifying the action information of the deaf-mute; a depth camera connected to the primary terminal, the depth camera facing the secondary terminal; the holder is a rotatable structure, and the depth camera is mounted on the holder; and the main controller is connected with the main terminal, the auxiliary terminal and the holder.
Firstly, it is indicated that the main terminal is a computer for teachers and the auxiliary terminal is a computer for deaf-mutes students.
And secondly, the display unit comprises but is not limited to displaying characters converted by the actions of the deaf-mute, also comprises displaying sign language recognized according to the actions of the deaf-mute, displaying a video picture acquired by the depth camera and displaying a shortcut phrase.
Then, the number information of the ID cell corresponds to the name of the student and the angle of the seat and the platform for adjusting the angular direction of the depth camera and displaying the name of the student who is answering on the main terminal.
Finally, the working principle of the application is as follows: when deaf-mute students on a certain pair of terminals carry out sign language actions to the gesture sensor, the depth camera is triggered to work, the main terminal displays pictures obtained by the depth camera, and the main terminal judges whether the position of the main terminal is in the center of the pictures of the depth camera according to angle data corresponding to the ID number information of the pair of terminals obtained through pre-storage, if not, the main terminal sends a signal to the cradle head through the main controller, so that the cradle head drives the depth camera to rotate, and the depth camera is aligned to the pair of terminals.
The angle between the depth camera and the auxiliary terminal is not less than 10 degrees finally, so that the main terminal can acquire the gesture actions of the deaf-mute student according to the depth camera, and further convert the gesture actions into sign language information or trigger shortcut information according to the gesture actions, wherein if the deaf-mute student makes specific actions to the gesture sensor, the depth camera can be turned on or turned off.
Further, in the embodiment of the utility model provides an in, the cloud platform is by main control unit control direction.
Further, in the embodiment of the present invention, the holder is a single-axis camera holder.
Further, in the embodiment of the present invention, the auxiliary terminal is connected to the main controller through a wireless serial port.
Further, in the embodiment of the present invention, the gesture sensor inputs the corresponding shortcut message and the information of whether to activate the gesture sensor according to the basic action. Triggering a shortcut message or activating a gesture recognition system through a basic action made to a gesture sensor of the secondary terminal.
The addition of the gesture sensor enables the shortcut phrases to be matched with the sign language recognition capability, and the communication efficiency between the deaf-mute students and the teacher is improved.
Further, in the embodiment of the present invention, the gesture sensor is PAJ7620 gesture sensor.
Further, in the embodiment of the present invention, the auxiliary terminal further includes an indicator light and an AT89C52 single chip microcomputer, the AT89C52 single chip microcomputer serves as a processor, and the ID unit is allocated on the AT89C52 single chip microcomputer.
Furthermore, in the embodiment of the present invention, the wireless serial port is an LC12S wireless serial port.
Further, in the embodiment of the present invention, the main terminal has a sign language recognition function for tracking the joint position.
Further, in the embodiment of the present invention, the main terminal further has: and the voice module is used for converting the characters or the actions of the deaf-mute into voice information.
Further, in the embodiment of the present invention, the auxiliary terminal further has: and the light module is used for indicating whether the deaf-mute is doing the action or not. The pilot lamp of light module is the information transfer mode who commonly uses in the deaf-mute student special education, and the addition of pilot lamp can more directly perceived attract teachers and students' attention.
Further, in an embodiment of the present invention, the secondary terminal further includes: and the playing module is used for playing the converted voice information.
Further, in the embodiment of the present invention, the main controller is Atmega 328P.
In addition, the gesture sensor has the ability to recognize simple actions, and students can trigger shortcut messages or activate gesture recognition systems by making basic actions, such as: when the hand-depth camera is swung forward to start, the indicator light of the light module is turned on to show that the student starts speaking, the display unit of the main terminal displays the name of the questioner, and the depth camera automatically turns to the answering person according to the ID number of the auxiliary terminal to perform sign language recognition and display recognized characters.
Waving the hand backward-the depth camera is turned off, the pan-tilt is turned back, and the indicator light is turned off to indicate that the speech generation is finished.
Waving his hand to the left-the master terminal makes a sound "i have a question" and displays text to remind "there is a student asking a question".
Waving the hand to the right-the master terminal makes a sound "like" and displays a text reminder "like".
Waving hands up and down-the master terminal makes a sound "the teacher does not understand.
The sign language identification method comprises the following steps: and tracking the palm position, the elbow position and the orientation of the palm by the depth camera and comparing the positions with a pre-recorded action library.
The utility model provides a can cover full classroom, possess sign language discernment, swift mutual, speech synthesis and throw the sign language translation teaching system of screen function for the special education school that gives lessons for deaf-mute student improves deaf-mute's teaching quality.
The utility model has the advantages that: this application adopts the mode of reserving vice terminal ID serial number information in advance for when the gesture sensor of vice terminal acquires action information at every turn, main terminal can both lock the position of vice terminal, and then uses the rotatory vice terminal of this of cloud platform control depth camera alignment, consequently makes the depth camera can carry out sign language to the student of arbitrary assigned position in the classroom and discerns. Therefore, when the deaf-mute student encounters a problem, the student can respond timely, effectively and quickly without leaving the seat to go to the lens when playing sign language, and the learning quality effect of the deaf-mute student is greatly enhanced.
Drawings
Fig. 1 is a schematic structural diagram of a tracking processing system for recognizing and representing gestures according to an embodiment of the present invention.
Fig. 2 is a schematic view of a usage flow of the depth camera according to an embodiment of the present invention.
Fig. 3 is a schematic diagram illustrating the use effect of the tracking processing system for recognizing and representing gestures according to the embodiment of the present invention.
Detailed Description
In order to make the objects and technical solutions of the present invention clear and fully described, and the advantages thereof more clearly understood, the embodiments of the present invention are described in further detail below with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of some, but not all, embodiments of the present invention and are not to be considered as limiting, and that all other embodiments can be made by one of ordinary skill in the art without any inventive work.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "inner", "outer", "top", "bottom", "side", "vertical", "horizontal", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for convenience of description and simplification of description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "a," "an," "first," "second," "third," "fourth," "fifth," and "sixth" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood in specific cases to those skilled in the art.
For the purposes of simplicity and explanation, the principles of the embodiments are described by referring mainly to examples. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. But it is obvious. To one of ordinary skill in the art, the embodiments may be practiced without limitation to these specific details. In some instances, well-known house beams, support post connection methods, and structures have not been described in detail to avoid unnecessarily obscuring these embodiments. In addition, all embodiments may be used in combination with each other.
Example 1: a tracking processing system for recognizing and representing gestures, as shown in fig. 1-3, comprising: main terminal, vice terminal, degree of depth camera, cloud platform, main control unit.
The main terminal is a computer for teachers, and the auxiliary terminal is a computer for deaf-mutes students.
The main terminal is provided with a display unit which is a display screen, and the display unit displays characters converted by the actions of the deaf-mute.
The display unit comprises but is not limited to displaying characters converted by the actions of the deaf-mute, displaying sign language recognized according to the actions of the deaf-mute, displaying video pictures acquired by the depth camera and displaying shortcut phrases.
The sub-terminal has an ID unit and a gesture sensor. The serial number information of the ID unit is pre-recorded in the main terminal, and the gesture sensing is carried out
The device is used for acquiring and identifying the action information of the deaf-mute.
The ID unit number information corresponds to the name of the student and the angle between the seat and the platform, and is used for adjusting the angle direction of the depth camera and displaying the name of the student answering on the main terminal.
Gesture sensors are the prior art, which have been already described and are widely applied, such as gesture sensor technologies disclosed in U.S. patent application publication No. 2012/0280107 (11/08/2012) and U.S. patent application publication No. 2012/0280904 (11/08/2012), gesture units and gesture sensors disclosed in chinese patent CN201410582403.1, and a method and apparatus for controlling rotation of a camera by gestures and an infrared gesture sensor disclosed in chinese patent CN 201410502913.3.
The depth camera is connected with the main terminal, the camera of the depth camera faces the auxiliary terminal, the depth camera is Kinect, the Kinect has the function of 1080P high-definition video recording and the function of human posture recognition, and the limb action of a person can be recognized.
The main terminal is used for displaying pictures shot by the Kinect and displaying sign language recognition characters, and the main terminal can also control the Kinect to perform a sign language recognition function.
The cloud platform is rotatable structure, and the degree of depth camera is installed on the cloud platform. When a student starts the sign language sensor to speak, the main controller can judge whether the position of the student is in the center of the picture of the depth camera according to angle data corresponding to the ID number of the auxiliary terminal, and if the position of the student is not in the identification angle, the main controller can automatically control the cradle head to rotate the depth camera to a required angle.
The main controller is connected with the main terminal, the auxiliary terminal and the holder.
The working principle of this application does: when deaf-mute students on a certain pair of terminals carry out sign language actions to the gesture sensor, the depth camera is triggered to work, the main terminal displays pictures obtained by the depth camera, and the main terminal judges whether the position of the main terminal is in the center of the pictures of the depth camera according to angle data corresponding to the ID number information of the pair of terminals obtained through pre-storage, if not, the main terminal sends a signal to the cradle head through the main controller, so that the cradle head drives the depth camera to rotate, and the depth camera is aligned to the pair of terminals.
The angle between the depth camera and the auxiliary terminal is not less than 10 degrees finally, so that the main terminal can acquire the gesture actions of the deaf-mute student according to the depth camera, and further convert the gesture actions into sign language information or trigger shortcut information according to the gesture actions, wherein if the deaf-mute student makes specific actions to the gesture sensor, the depth camera can be turned on or turned off.
This application adopts the mode of reserving vice terminal ID serial number information in advance for when the gesture sensor of vice terminal acquires action information at every turn, main terminal can both lock the position of vice terminal, and then uses the rotatory vice terminal of this of cloud platform control depth camera alignment, consequently makes the depth camera can carry out sign language to the student of arbitrary assigned position in the classroom and discerns. Therefore, when the deaf-mute student encounters a problem, the student can respond timely, effectively and quickly without leaving the seat to go to the lens when playing sign language, and the learning quality effect of the deaf-mute student is greatly enhanced.
Preferably, the pan/tilt head is direction controlled by a master controller.
Preferably, the pan/tilt head is a single axis camera pan/tilt head.
Preferably, the secondary terminal is connected with the main controller through a wireless serial port.
More preferably, the wireless serial port is an LC12S wireless serial port.
Preferably, the gesture sensor has a function of recognizing sign language, and is also activated or deactivated according to the motion, and the gesture sensor enters the corresponding shortcut message and information on whether to activate the gesture sensor according to the basic motion. Triggering a shortcut message or activating a gesture sensor by a basic action made to the gesture sensor of the secondary terminal.
The addition of the gesture sensor enables the shortcut phrases to be matched with the sign language recognition capability, and the communication efficiency between the deaf-mute students and the teacher is improved.
Preferably, the gesture sensor is PAJ7620 gesture sensor.
Preferably, the auxiliary terminal further comprises an indicator light and an AT89C52 single chip microcomputer, the AT89C52 single chip microcomputer serves as a processor, and the ID unit is distributed on the AT89C52 single chip microcomputer.
Preferably, the master terminal has a sign language recognition function for tracking the joint position.
Preferably, the main terminal is also provided with a voice module, and the voice module is used for converting the characters or the actions of the deaf-mute into voice information.
Preferably, the auxiliary terminal is also provided with a light module which is used for indicating whether the deaf-mute is doing actions or not. The pilot lamp of light module is the information transfer mode who commonly uses in the deaf-mute student special education, and the addition of pilot lamp can more directly perceived attract teachers and students' attention.
More preferably, the secondary terminal further has a playing module, and the playing module is configured to play the converted voice information.
Preferably, the master controller is Atmega 328P.
In addition, the gesture sensor has the ability to recognize simple actions, and students can trigger shortcut messages or activate gesture recognition systems by making basic actions, such as: when the hand-depth camera is swung forward to start, the indicator light of the light module is turned on to show that the student starts speaking, the display unit of the main terminal displays the name of the questioner, and the depth camera automatically turns to the answering person according to the ID number of the auxiliary terminal to perform sign language recognition and display recognized characters.
Waving the hand backward-the depth camera is turned off, the pan-tilt is turned back, and the indicator light is turned off to indicate that the speech generation is finished.
Waving his hand to the left-the master terminal makes a sound "i have a question" and displays text to remind "there is a student asking a question".
Waving the hand to the right-the master terminal makes a sound "like" and displays a text reminder "like".
Waving hands up and down-the master terminal makes a sound "the teacher does not understand.
The sign language identification method comprises the following steps: and tracking the palm position, the elbow position and the orientation of the palm by the depth camera and comparing the positions with a pre-recorded action library.
Although the invention has been described with respect to illustrative embodiments thereof so that those skilled in the art can understand the invention, it is to be understood that the invention is not limited to the disclosed embodiments, but rather, is intended to cover all modifications and variations within the spirit and scope of the invention as defined and defined by the appended claims.
Claims (9)
1. A tracking processing system for recognizing and representing gestures, comprising:
the main terminal is provided with a display unit, and the display unit displays characters converted by the actions of the deaf-mutes;
the auxiliary terminal is provided with an ID unit and a gesture sensor, the main terminal prestores the serial number information of the ID unit, and the gesture sensor is used for acquiring and identifying the action information of the deaf-mute;
a depth camera connected to the primary terminal, the depth camera facing the secondary terminal;
the holder is a rotatable structure, and the depth camera is mounted on the holder;
and the main controller is connected with the main terminal, the auxiliary terminal and the holder.
2. A tracking processing system for recognition and representation of gestures as claimed in claim 1, wherein said depth camera is a Kinect.
3. A tracking processing system for recognition and representation of gestures as claimed in claim 1 in which the pan/tilt head is a single axis camera pan/tilt head.
4. A tracking processing system for recognition and presentation of gestures as claimed in claim 1 in which said secondary terminal is connected to said primary controller via a wireless serial port.
5. A tracking processing system for recognition and representation of gestures as claimed in claim 1 in which the gesture sensor has the function of recognizing sign language.
6. A tracking processing system for recognition and representation of gestures as claimed in claim 1 in which said gesture sensor is a PAJ7620 gesture sensor.
7. The tracking processing system for recognizing and representing gestures as claimed in claim 1, wherein said secondary terminal further comprises an indicator light and an AT89C52 single chip microcomputer, the AT89C52 single chip microcomputer is used as a processor, and said ID unit is distributed on said AT89C52 single chip microcomputer.
8. A tracking processing system for recognition and representation of gestures as claimed in claim 4 wherein said wireless serial port is an LC12S wireless serial port.
9. A trace processing system for recognition and representation of gestures as claimed in claim 1 wherein said master terminal further has: and the voice module is used for converting the characters or the actions of the deaf-mute into voice information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202120026852.3U CN212460576U (en) | 2021-01-07 | 2021-01-07 | Tracking processing system for recognizing and representing gestures |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202120026852.3U CN212460576U (en) | 2021-01-07 | 2021-01-07 | Tracking processing system for recognizing and representing gestures |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN212460576U true CN212460576U (en) | 2021-02-02 |
Family
ID=74473953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202120026852.3U Active CN212460576U (en) | 2021-01-07 | 2021-01-07 | Tracking processing system for recognizing and representing gestures |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN212460576U (en) |
-
2021
- 2021-01-07 CN CN202120026852.3U patent/CN212460576U/en active Active
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Barr et al. | Anchoring comprehension in linguistic precedents | |
| CN104575137B (en) | Split type scene interaction multi-medium intelligent terminal | |
| KR100814330B1 (en) | Robot system for learning-aids and teacher-assistants | |
| CN109102727A (en) | A kind of virtual reality teaching equipment | |
| CN213241435U (en) | Desktop device for protecting eyesight of students during study | |
| CN109241917A (en) | A kind of classroom behavior detection system based on computer vision | |
| CN110085072A (en) | A kind of implementation method and device of the asymmetric display in multimachine position | |
| CN212460576U (en) | Tracking processing system for recognizing and representing gestures | |
| CN206181175U (en) | Scanning writing content's pen when writing | |
| CN113570916A (en) | Multimedia remote teaching auxiliary method, equipment and system | |
| CN206789126U (en) | A kind of multimedia teaching apparatus | |
| CN211475595U (en) | Intelligent desk lamp | |
| CN104867361A (en) | Intelligent terminal for interactional and situational teaching | |
| CN204480508U (en) | A kind of scene interaction formula multimedia system | |
| CN215687779U (en) | Posture data statistical instrument for young students | |
| CN106781744B (en) | Intelligent mathematical multi media for teaching system | |
| CN210402777U (en) | Virtual reality teaching equipment | |
| CN211146176U (en) | Desk lamp | |
| CN204759783U (en) | Interactive teaching intelligent terminal of sight | |
| CN110276242B (en) | Image processing method, device and storage medium | |
| CN113112771A (en) | Student sitting posture identification method and system | |
| JP3074677B2 (en) | A two-way interactive system having a mechanism for matching eyes between interlocutors via transmission means | |
| CN110767010A (en) | Teaching assistance method and device | |
| CN110119216A (en) | Smart pen and teaching equipment | |
| JP2003333561A (en) | Monitor screen displaying method, terminal, and video conference system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| GR01 | Patent grant | ||
| GR01 | Patent grant |