CN109460748A - A kind of trinocular vision hand language recognition device and multi-information fusion sign Language Recognition Method - Google Patents
A kind of trinocular vision hand language recognition device and multi-information fusion sign Language Recognition Method Download PDFInfo
- Publication number
- CN109460748A CN109460748A CN201811501014.6A CN201811501014A CN109460748A CN 109460748 A CN109460748 A CN 109460748A CN 201811501014 A CN201811501014 A CN 201811501014A CN 109460748 A CN109460748 A CN 109460748A
- Authority
- CN
- China
- Prior art keywords
- photographic device
- sign language
- monocular
- gesture
- manpower
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 13
- 230000004927 fusion Effects 0.000 title claims abstract description 9
- 206010011878 Deafness Diseases 0.000 claims abstract description 18
- 230000008921 facial expression Effects 0.000 claims description 33
- 241000905957 Channa melasoma Species 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 238000012546 transfer Methods 0.000 claims description 8
- 238000012795 verification Methods 0.000 claims description 8
- 238000012790 confirmation Methods 0.000 claims description 6
- 238000009434 installation Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 230000003287 optical effect Effects 0.000 claims description 2
- 230000007547 defect Effects 0.000 abstract description 5
- 238000013519 translation Methods 0.000 abstract description 3
- 210000003813 thumb Anatomy 0.000 description 14
- 238000010586 diagram Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 208000032041 Hearing impaired Diseases 0.000 description 1
- 244000062793 Sorghum vulgare Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 208000030251 communication disease Diseases 0.000 description 1
- 210000003414 extremity Anatomy 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 235000019713 millet Nutrition 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/009—Teaching or communicating with deaf persons
Abstract
The invention discloses a kind of trinocular vision hand language recognition device and multi-information fusion sign Language Recognition Methods, are related to the intelligent translation technical field of sign language；The vision dead zone defect and independent gesture identification scheme for solving the problem of binocular equipment can not accurately obtain the information to be conveyed of deaf-mute's sign language；The present invention increases a monocular photographic device in the vertical plane of binocular equipment, acquires the information in vision dead zone, and the data as binocular equipment vision dead zone are supplied；It is proposed by sign language expresser expression, carrying out sign language exchange when hand human body the one group of sign language gesture in relative position and sign language gesture identification common definition, the mode that combination defines sign language in this way increases the quantity of recognizable gesture, expands the rich of sign language library；Meanwhile multiple combinations element has refined each sign language, improves the specificity of each sign language, avoids similar, close gesture that sign language is caused to judge incorrectly, improves the accuracy of Sign Language Recognition.
Description
Technical field
The present invention relates to the intelligent translation technical fields of sign language, and in particular to a kind of trinocular vision hand language recognition device and more
Information merges sign Language Recognition Method.
Background technique
It is counted according to country data, in China, deaf-mute's number in the five big deformity such as disabled, limb is residual, intelligence is residual is most, about
20570000, that is to say, that in every 100 people, just there is a deaf-mute, wherein about 800,000 people were less than 7 years old.According to another newest money
There is 0.02% deaf-mute's birth in material display, China every year, count hearing impaired newborn in, and China exists there are about 1% people listens
Power obstacle.In life, this group of deaf-mute and the external world there are communication disorders, there are this obstacle they from once being born
Almost determine their future, the circle that they can live, the environment that they exchange, live can only all be confined to deaf-mute
The world in.For example, they go to a hospital to see a doctor in the case where nobody accompanies, since there are obstacles for language expression, and there is no hand
Language translation, there is various limitations when doctor illustrates the state of an illness；When removing the service windows such as bank, it can not be carried out with shop assistant
Exchange.A kind of gesture of the sign language as highly structural is that deaf-mute carries out the daily essential means of exchange.Sign Language Recognition
As an important component in field of human-computer interaction, its research and realization is with important learning value and widely
Application prospect.
Stereoscopic vision is an important branch of computer vision, binocular stereo vision be it is one such, it using at
As equipment passes through calculating image using principle of triangulation from two width or multiple image of different position acquisition testees
Position deviation between corresponding points, to obtain the three-dimensional information of object table millet cake, final reconstruction of objects shape or surface number are three-dimensional
Pattern.But traditional binocular equipment has the limitation of vision dead zone, as shown in Figure 1, when identification plane and binocular camera shooting head plane
When vertical, due to mutually blocking for each position of object, cause blind area part that can not accurately identify, secondly independent gesture identification
Scheme can not accurately obtain the information to be conveyed of deaf-mute's sign language, and problem above reduces the accurate of deaf-mute's Sign Language Recognition
Degree.
Summary of the invention
The purpose of the present invention is to provide a kind of trinocular vision hand language recognition device and multi-information fusion sign Language Recognition Method,
The vision dead zone defect and independent gesture identification scheme for solving binocular equipment can not accurately obtain deaf-mute's sign language and be wanted
The problem of information of reception and registration；For the vision dead zone defect for solving binocular equipment, the present invention increases in the vertical plane of binocular equipment
One monocular photographic device, acquires the information in vision dead zone, and the data as binocular equipment vision dead zone are supplied；For existing
Independent gesture identification scheme can not accurately obtain the information to be conveyed of deaf-mute's sign language, propose by sign language expresser's table
Feelings are carrying out when sign language exchange hand in one group of sign language hand in relative position and sign language gesture identification common definition of human body
Gesture, the mode that combination defines sign language in this way increase the quantity of recognizable gesture, expand the rich of sign language library；Meanwhile it is more
Kind integration factor has refined each sign language, improves the specificity of each sign language, avoids similar, close gesture that sign language is caused to judge
Mistake improves the accuracy of Sign Language Recognition.
The technical solution adopted by the invention is as follows: a kind of trinocular vision hand language recognition device characterized by comprising view
Feel platform, left photographic device, right photographic device, monocular adjusting bracket, monocular photographic device, image processing equipment；
The vision platform is horizontal positioned, is symmetrically installed left photographic device, right photographic device, left photographic device and right camera shooting thereon
Monocular adjusting bracket is set on the perpendicular bisector of device optical center line；The monocular adjusting bracket includes support rod, silk
Thick stick, lead screw mounting base, bearing, wind, lifting nut；The vertical vision platform installation of the support rod, the front setting is along branch
Altitude scale is arranged in the lifting limiting slot in stay length direction, side；The lead screw is installed in parallel in front of support rod, above and below
Both ends are installed in rotation in lead screw mounting base, and bearing is installed in lead screw mounting base, and wind is arranged in lead screw upper end；The liter
Drop nut is mounted on lead screw, the lifting of side setting thereafter detent portion, and lifting detent portion, which can slide up and down to, is stuck in lifting limit
In slot, rotating junction is arranged in lifting nut leading flank, on rotating junction installation can horizontal left-right rotation horizontally rotate part,
Horizontally rotate part front end and is horizontally mounted monocular photographic device；Height pointer is set in the lifting detent portion, and height pointer is directed toward
Altitude scale；Protractor is stretched out in the lifting nut bottom surface, and protractor is parallel with vision platform, and angle is arranged on monocular photographic device
Pointer is spent, angle indicator is directed toward the graduation mark on protractor；The height pointer is with monocular camera head lens center same
In height, monocular camera head lens centre-height is indicated；The angle indicator and monocular camera head lens central axis are flat
Row indicates monocular photographic device level angle；The monocular camera head lens central axis and left photographic device and right camera shooting
Vertical plane where device is vertical；The left photographic device, right photographic device, monocular photographic device establish a virtual three-dimensional jointly
Coordinate system, wherein straight line where monocular camera head lens central axis is X-axis, perpendicular to vision platform and passes through left camera shooting
Device and the straight line at right camera head lens line of centres midpoint are Z axis, perpendicular to XZ plane, the straight line intersected with X-axis, Y-axis
For Y-axis, X-axis, Y-axis, Z axis intersection point are coordinate origin (X0, Y0, Z0)；Established virtual three-dimensional coordinate system no longer becomes after use
Dynamic, i.e., left photographic device, right photographic device, the position of monocular photographic device, angle no longer change, and monocular adjusting bracket is for every
Calibration before secondary use debugs monocular photographic device for initial position, when such gesture identification, as long as reproducing gesture model library
Gesture when middle typing, the gesture that energy accurate judgement deaf-mute shows, avoids the position of monocular photographic device, angles shifts from leading
The identification mistake of cause；In use, shaking clockwise or counterclockwise by wind, realize that lifting nut is raised and lowered；Highly
Pointer is directed toward altitude scale, the height for facilitating observation to adjust, and realizes that monocular camera head lens centre-height is accurately adjusted；Monocular
Angle indicator is set on photographic device, and angle indicator is directed toward the graduation mark on protractor, angle indicator and monocular photographic device mirror
Head centerline axis parallel, indicates monocular photographic device level angle, quasi- by graduation mark of the viewing angle pointer on protractor
Really adjust level angle；
The left photographic device, right photographic device, monocular photographic device connect image processing equipment, wherein left photographic device, the right side
Photographic device forms Binocular Stereo Vision System；
Described image processing equipment include at least image acquisition units, three-dimensional modeling unit, gesture model library, face expression database,
Sign language library, gesture authentication unit, human face expression authentication unit, manpower location verification list are combined in manpower and human body relative position library
Member, combination sign language authentication unit, sign language converting unit, sign language output unit；
Gesture model is stored in the gesture model library, gesture model is left photographic device, right photographic device, monocular photographic device
The gesture model information of acquisition typing in advance includes articulations digitorum manus coordinate and vector data；
Human face expression picture is stored in face expression database；Storage manpower and human body relative position in manpower and human body relative position library
Picture；Storage combination sign language in sign language library is combined, each combination sign language is by a gesture model, human face expression picture, one
Open manpower and human body relative position picture common definition.
Further, described image processing equipment is PC machine.
Further, the sign language converting unit is converted to text for sign language is combined, and is sent to the output of sign language output unit；
Sign language output unit is indicator by character.
Further, the sign language converting unit is converted to voice for sign language is combined, and is sent to the output of sign language output unit；
Sign language output unit is speech player.
Further, the three-dimensional modeling unit uses Opencv computer vision library, is filled according to left photographic device, right camera shooting
The manpower that obtains from bottom to top is set to the depth distance information of left photographic device, right photographic device, monocular photographic device is from side
The manpower of acquisition carries out gesture model foundation to the depth distance information of monocular photographic device.
A kind of multi-information fusion sign Language Recognition Method of trinocular vision hand language recognition device, which is characterized in that including as follows
Step:
(1) initial position is calibrated: being adjusted the height and angle of monocular photographic device by monocular adjusting bracket, is met it and establish
The requirement of virtual three-dimensional coordinate system；
(2) by left photographic device, right photographic device, the preparatory typing gesture model of monocular photographic device, gesture model library is established:
Gesture is made in virtual three-dimensional coordinate system and is kept, left photographic device, right photographic device, monocular photographic device continuous collecting
10-15 minutes；Left photographic device, right photographic device form Binocular Stereo Vision System, obtain manpower from bottom to top to left camera shooting
The depth distance information of device, right photographic device；Depth of the monocular photographic device from side acquisition manpower to monocular photographic device
Range information, the data as Binocular Stereo Vision System blind area are supplied, to extract more complete gesture information, very big journey
Spend or completely eliminate binocular vision blind area；Being programmed using java() language by data inputting mysql database, completes a hand
Potential model typing；The gesture model information of typing includes coordinate and vector data of the articulations digitorum manus in virtual three-dimensional coordinate system；
(3) human face expression picture is stored in face expression database, by manpower and human body relative position picture deposit manpower and human body
In the library of relative position；Storage combination sign language in sign language library is combined, each combination sign language is by a gesture model, a human face expression
Picture, a manpower and human body relative position picture common definition；
(4) hand is protruded into progress sign language expression in virtual three-dimensional coordinate system by deaf-mute's face monocular photographic device；
(5) left photographic device, right photographic device form Binocular Stereo Vision System, obtain manpower from bottom to top and fill to left camera shooting
It sets, the depth distance information of right photographic device；Monocular photographic device from side obtain manpower to monocular photographic device depth away from
From information, the data as Binocular Stereo Vision System blind area are supplied, to extract more complete gesture information, high degree
Or completely eliminate binocular vision blind area；In addition, monocular photographic device also acquires manpower and human body relative position picture and face face
Portion's expression picture；
(6) image acquisition units receive the information that the left photographic device of synchronization, right photographic device, monocular photographic device obtain,
And it is respectively sent to three-dimensional modeling unit, human face expression authentication unit, manpower location verification unit；
Three-dimensional modeling unit establishes gesture model, and the gesture that will be built up according to the depth distance information that three photographic devices obtain
Model is sent to gesture authentication unit；Gesture authentication unit transfers the gesture model stored in gesture model library and the gesture built up
Model carries out articulations digitorum manus coordinate and vector data matching, confirms the highest gesture model of matching degree, and send it to combination hand
Language authentication unit；
Human face expression authentication unit transfers the people of the human face expression picture stored in face expression database and the acquisition of monocular photographic device
Face portion expression picture carries out characteristic matching, confirms the highest human face expression picture of matching degree, and send it to combination sign language
Authentication unit；
Manpower location verification unit transfer the manpower stored in manpower and human body relative position library and human body relative position picture with
The manpower and human body relative position picture of monocular photographic device acquisition carry out characteristic matching, the highest manpower of confirmation matching degree and people
Body relative position picture sends it to combination sign language authentication unit；
Combination sign language authentication unit is transferred the combination sign language stored in combination sign language library and is compared, and confirmation combination sign language is sent
It is converted to sign language converting unit, the combination sign language after conversion is sent to sign language output unit and exported by sign language converting unit.
The beneficial effects of the present invention are: the present invention provides a kind of trinocular vision hand language recognition device and multi-information fusions
Sign Language Recognition Method, the vision dead zone defect and independent gesture identification scheme for solving binocular equipment can not be obtained accurately
The problem of information to be conveyed of deaf-mute's sign language；For the vision dead zone defect for solving binocular equipment, the present invention is in binocular equipment
Vertical plane increase a monocular photographic device, acquire vision dead zone in information, the number as binocular equipment vision dead zone
According to supplying；The information to be conveyed of deaf-mute's sign language can not be accurately obtained for existing independent gesture identification scheme, is proposed
By sign language expresser expression, carrying out sign language exchange when hand it is jointly fixed in the relative position of human body and sign language gesture identification
Adopted one group of sign language gesture, the mode that combination defines sign language in this way increase the quantity of recognizable gesture, expand the rich of sign language library
Fu Xing；Meanwhile multiple combinations element has refined each sign language, improves the specificity of each sign language, avoids similar, close gesture
It causes sign language to judge incorrectly, improves the accuracy of Sign Language Recognition.
Detailed description of the invention
Fig. 1 is the schematic diagram that traditional binocular equipment has vision dead zone limitation.
Fig. 2 is trinocular vision hand language recognition device schematic perspective view of the invention.
Fig. 3 is monocular adjusting bracket structural schematic diagram of the invention.
Fig. 4 is lifting nut structural schematic diagram of the invention.
Fig. 5 is angle indicator structural schematic diagram of the invention.
Fig. 6 is multi-information fusion sign Language Recognition Method flow diagram of the invention.
Fig. 7 is the expression schematic diagram of " good " combination sign language in the embodiment of the present invention.
Fig. 8 is " right hand erects thumb gesture " gesture model information schematic diagram of typing in the embodiment of the present invention.
In figure: vision platform 1, left photographic device 2, right photographic device 3, monocular adjusting bracket 4, monocular photographic device 5, branch
Strut 6, lead screw 7, lead screw mounting base 8, bearing 9, wind 10, lifting nut 11 go up and down limiting slot 6-1, altitude scale 6-2,
Detent portion 11-1, rotating junction 11-2 are gone up and down, horizontally rotates part 11-3, height pointer 11-4, protractor 11-5, angle indicator
5-1。
Specific embodiment
In order to have clearer understanding to technical solution of the present invention, purpose and effect, now in conjunction with the Detailed description of the invention present invention
Specific embodiment.
Embodiment, as shown in fig. 7, defining, " right hand erects thumb gesture ", " face's smile expression ", " gesture is located at left chest
Before " a combination sign language is represented, which indicates the meaning of " good "；It is filled using trinocular vision Sign Language Recognition provided by the invention
It sets and multi-information fusion sign Language Recognition Method carries out judgement identification；
(1) initial position is calibrated: being adjusted the height and angle of monocular photographic device by monocular adjusting bracket, is met it and establish
The requirement of virtual three-dimensional coordinate system: left photographic device, right photographic device, monocular photographic device establish a virtual three-dimensional coordinate jointly
System, wherein straight line where monocular camera head lens central axis is X-axis, perpendicular to vision platform and passes through left photographic device
Straight line with right camera head lens line of centres midpoint is Z axis, is Y perpendicular to XZ plane, the straight line intersected with X-axis, Y-axis
Axis, X-axis, Y-axis, Z axis intersection point are coordinate origin (X0, Y0, Z0)；Established virtual three-dimensional coordinate system no longer changes after use,
I.e. left photographic device, right photographic device, the position of monocular photographic device, angle no longer change, and monocular adjusting bracket is for each
Calibration before use debugs monocular photographic device for initial position, when such gesture identification, as long as reproducing in gesture model library
Gesture when typing, the gesture that energy accurate judgement deaf-mute shows, avoids the position of monocular photographic device, angles shifts from causing
Identification mistake；
(2) by left photographic device, right photographic device, the preparatory typing of monocular photographic device " right hand erects thumb gesture " gesture
Model is stored in gesture model library: making the hand that " right hand erects thumb gesture " is located at left front in virtual three-dimensional coordinate system
Gesture is simultaneously kept, left photographic device, right photographic device, monocular photographic device continuous collecting 10-15 minutes；Left photographic device, the right side are taken the photograph
Picture device forms Binocular Stereo Vision System, from bottom to top the depth distance of acquisition manpower to left photographic device, right photographic device
Information；Thumb is located at binocular vision blind area, depth of the monocular photographic device from side acquisition thumb to monocular photographic device
Range information, the data as Binocular Stereo Vision System blind area are supplied,
To extract more complete gesture information, high degree or binocular vision blind area is completely eliminated；It will using java language
In data inputting mysql database, gesture model typing is completed；The gesture model information of typing includes articulations digitorum manus in virtual three-dimensional
Coordinate and vector data in coordinate system, as shown in figure 8, being recorded in " right hand erects thumb gesture " gesture model information of typing
Two coordinate (X of thumb front end joint1, Y1, Z1), (X2, Y2, Z2), and represent thumb front end joint direction and
The vector data of length；
(3) by " face's smile expression " picture be stored in face expression database in, will " gesture is located at left front " picture be stored in manpower and
In the library of human body relative position；The gesture model of " right hand erects thumb gesture ", " face's smile expression " picture, " gesture is located at a left side
Front " picture common definition is the meaning of " good ", is stored in combination sign language library；
(4) hand is protruded into virtual three-dimensional coordinate system and makes that " right hand erects thumb hand again by deaf-mute's face monocular photographic device
Gesture " is located at the gesture of left front, while doing " face's smile expression "；
(5) left photographic device, right photographic device form Binocular Stereo Vision System, obtain manpower from bottom to top and fill to left camera shooting
It sets, the depth distance information of right photographic device；Monocular photographic device from side obtain manpower to monocular photographic device depth away from
From information, the data as Binocular Stereo Vision System blind area are supplied, in addition, monocular photographic device also acquires, " right hand erects big thumb
Refer to gesture " it is located at the gesture picture and " face's smile expression " picture of left front；
(6) image acquisition units receive the information that the left photographic device of synchronization, right photographic device, monocular photographic device obtain,
And it is respectively sent to three-dimensional modeling unit, human face expression authentication unit, manpower location verification unit；
Three-dimensional modeling unit establishes " right hand erects thumb gesture " gesture according to the depth distance information that three photographic devices obtain
Model, and the gesture model built up is sent to gesture authentication unit；Gesture authentication unit transfers preparatory typing gesture model library
In gesture model carry out articulations digitorum manus coordinate with the gesture model built up and vector data match, the confirmation highest gesture of matching degree
Model, and the highest gesture model of matching degree that will confirm that is sent to combination sign language authentication unit；Three-dimensional modeling unit can be adopted
With Opencv computer vision library, the manpower that is obtained from bottom to top according to left photographic device, right photographic device to left photographic device,
The depth distance information of right photographic device, monocular photographic device is from the manpower that side obtains to the depth distance of monocular photographic device
Information carries out gesture model foundation；Articulations digitorum manus coordinate and vector data fit through computer programming realization；
Human face expression authentication unit transfers the people of the human face expression picture stored in face expression database and the acquisition of monocular photographic device
Face portion expression picture carries out characteristic matching, confirms the highest human face expression picture of matching degree, and send it to combination sign language
Authentication unit；" perceptual hash algorithm " in the prior art can be used to be programmed, realize the search work of similar pictures, it
Effect is to generate " fingerprint " (fingerprint) character string to every image, then compares the fingerprint of different images；As a result
It is closer, just illustrate that image is more similar；This is the prior art, so repeating no more；
Manpower location verification unit transfer the manpower stored in manpower and human body relative position library and human body relative position picture with
The manpower and human body relative position picture of monocular photographic device acquisition carry out characteristic matching, the highest manpower of confirmation matching degree and people
Body relative position picture sends it to combination sign language authentication unit；" perceptual hash calculation in the prior art equally can be used
Method " is programmed, and realizes identification；
Combination sign language authentication unit is transferred the combination sign language stored in combination sign language library and is compared, and confirms while meeting the " right hand
The combination sign language that perpendicular thumb gesture ", " face's smile expression ", " gesture is located at left front " represent, that is, confirm " good " meaning
Sign language is combined, sign language converting unit is sent to and is converted, sign language converting unit will combine sign language and be converted to text or voice hair
It send to sign language output unit and exports, sign language output unit uses indicator by character and speech player；Sign language is combined to compare, confirm
Process is realized by computer programming.
Although the present invention is described in detail referring to the foregoing embodiments, for those skilled in the art,
It still can carry out and modify to technical solution documented by foregoing embodiments, or carry out to part of technical characteristic
Equivalent replacement, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should be included in
Within protection scope of the present invention.
Claims (6)
1. a kind of trinocular vision hand language recognition device characterized by comprising vision platform, left photographic device, right camera shooting dress
It sets, monocular adjusting bracket, monocular photographic device, image processing equipment；
The vision platform is horizontal positioned, is symmetrically installed left photographic device, right photographic device, left photographic device and right camera shooting thereon
Monocular adjusting bracket is set on the perpendicular bisector of device optical center line；The monocular adjusting bracket includes support rod, silk
Thick stick, lead screw mounting base, bearing, wind, lifting nut；The vertical vision platform installation of the support rod, the front setting is along branch
Altitude scale is arranged in the lifting limiting slot in stay length direction, side；The lead screw is installed in parallel in front of support rod, above and below
Both ends are installed in rotation in lead screw mounting base, and bearing is installed in lead screw mounting base, and wind is arranged in lead screw upper end；The liter
Drop nut is mounted on lead screw, the lifting of side setting thereafter detent portion, and lifting detent portion, which can slide up and down to, is stuck in lifting limit
In slot, rotating junction is arranged in lifting nut leading flank, on rotating junction installation can horizontal left-right rotation horizontally rotate part,
Horizontally rotate part front end and is horizontally mounted monocular photographic device；Height pointer is set in the lifting detent portion, and height pointer is directed toward
Altitude scale；Protractor is stretched out in the lifting nut bottom surface, and protractor is parallel with vision platform, and angle is arranged on monocular photographic device
Pointer is spent, angle indicator is directed toward the graduation mark on protractor；The height pointer is with monocular camera head lens center same
In height, monocular camera head lens centre-height is indicated；The angle indicator and monocular camera head lens central axis are flat
Row indicates monocular photographic device level angle；The monocular camera head lens central axis and left photographic device and right camera shooting
Vertical plane where device is vertical；The left photographic device, right photographic device, monocular photographic device establish a virtual three-dimensional jointly
Coordinate system, wherein straight line where monocular camera head lens central axis is X-axis, perpendicular to vision platform and passes through left camera shooting
Device and the straight line at right camera head lens line of centres midpoint are Z axis, perpendicular to XZ plane, the straight line intersected with X-axis, Y-axis
For Y-axis, X-axis, Y-axis, Z axis intersection point are coordinate origin (X0, Y0, Z0)；
The left photographic device, right photographic device, monocular photographic device connect image processing equipment, wherein left photographic device, the right side
Photographic device forms Binocular Stereo Vision System；
Described image processing equipment include at least image acquisition units, three-dimensional modeling unit, gesture model library, face expression database,
Sign language library, gesture authentication unit, human face expression authentication unit, manpower location verification list are combined in manpower and human body relative position library
Member, combination sign language authentication unit, sign language converting unit, sign language output unit；
Gesture model is stored in the gesture model library, gesture model is left photographic device, right photographic device, monocular photographic device
The gesture model information of acquisition typing in advance includes articulations digitorum manus coordinate and vector data；
Human face expression picture is stored in face expression database；Storage manpower and human body relative position in manpower and human body relative position library
Picture；Storage combination sign language in sign language library is combined, each combination sign language is by a gesture model, human face expression picture, one
Open manpower and human body relative position picture common definition.
2. trinocular vision hand language recognition device as described in claim 1, which is characterized in that described image processing equipment is PC
Machine.
3. trinocular vision hand language recognition device as described in claim 1, which is characterized in that the sign language converting unit will combine
Sign language is converted to text, and is sent to the output of sign language output unit；Sign language output unit is indicator by character.
4. trinocular vision hand language recognition device as described in claim 1, which is characterized in that the sign language converting unit will combine
Sign language is converted to voice, and is sent to the output of sign language output unit；Sign language output unit is speech player.
5. trinocular vision hand language recognition device as described in claim 1, which is characterized in that the three-dimensional modeling unit uses
Opencv computer vision library, the manpower obtained from bottom to top according to left photographic device, right photographic device to left photographic device, the right side
The depth distance information of photographic device, monocular photographic device are believed from the manpower that side obtains to the depth distance of monocular photographic device
Breath carries out gesture model foundation.
6. a kind of multi-information fusion sign Language Recognition Method using trinocular vision hand language recognition device described in claim 1,
It is characterized in that, includes the following steps:
(1) initial position is calibrated: being adjusted the height and angle of monocular photographic device by monocular adjusting bracket, is met it and establish
The requirement of virtual three-dimensional coordinate system；
(2) by left photographic device, right photographic device, the preparatory typing gesture model of monocular photographic device, gesture model library is established:
Gesture is made in virtual three-dimensional coordinate system and is kept, left photographic device, right photographic device, monocular photographic device continuous collecting
10-15 minutes；Left photographic device, right photographic device form Binocular Stereo Vision System, obtain manpower from bottom to top to left camera shooting
The depth distance information of device, right photographic device；Depth of the monocular photographic device from side acquisition manpower to monocular photographic device
Range information；Using java language by data inputting mysql database, a gesture model typing is completed；The gesture of typing
Model information includes coordinate and vector data of the articulations digitorum manus in virtual three-dimensional coordinate system；
(3) human face expression picture is stored in face expression database, by manpower and human body relative position picture deposit manpower and human body
In the library of relative position；Storage combination sign language in sign language library is combined, each combination sign language is by a gesture model, a human face expression
Picture, a manpower and human body relative position picture common definition；
(4) hand is protruded into progress sign language expression in virtual three-dimensional coordinate system by deaf-mute's face monocular photographic device；
(5) left photographic device, right photographic device form Binocular Stereo Vision System, obtain manpower from bottom to top and fill to left camera shooting
It sets, the depth distance information of right photographic device；Monocular photographic device from side obtain manpower to monocular photographic device depth away from
From information, the data as Binocular Stereo Vision System blind area are supplied；In addition, monocular photographic device also acquires manpower and human body phase
To position picture and face facial expression picture；
(6) image acquisition units receive the information that the left photographic device of synchronization, right photographic device, monocular photographic device obtain,
And it is respectively sent to three-dimensional modeling unit, human face expression authentication unit, manpower location verification unit；
Three-dimensional modeling unit establishes gesture model, and the gesture that will be built up according to the depth distance information that three photographic devices obtain
Model is sent to gesture authentication unit；Gesture authentication unit is transferred pre-stored gesture model in gesture model library and is built up
Gesture model carries out articulations digitorum manus coordinate and vector data matching, confirms the highest gesture model of matching degree, and send it to group
Close sign language authentication unit；
Human face expression authentication unit transfers the people of the human face expression picture stored in face expression database and the acquisition of monocular photographic device
Face portion expression picture carries out characteristic matching, confirms the highest human face expression picture of matching degree, and send it to combination sign language
Authentication unit；
Manpower location verification unit transfer the manpower stored in manpower and human body relative position library and human body relative position picture with
The manpower and human body relative position picture of monocular photographic device acquisition carry out characteristic matching, the highest manpower of confirmation matching degree and people
Body relative position picture sends it to combination sign language authentication unit；
Combination sign language authentication unit is transferred the combination sign language stored in combination sign language library and is compared, and confirmation combination sign language is sent
It is converted to sign language converting unit, the combination sign language after conversion is sent to sign language output unit and exported by sign language converting unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811501014.6A CN109460748B (en) | 2018-12-10 | 2018-12-10 | Three-dimensional visual sign language recognition device and multi-information fusion sign language recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811501014.6A CN109460748B (en) | 2018-12-10 | 2018-12-10 | Three-dimensional visual sign language recognition device and multi-information fusion sign language recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109460748A true CN109460748A (en) | 2019-03-12 |
| CN109460748B CN109460748B (en) | 2024-03-01 |
Family
ID=65612920
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811501014.6A Active CN109460748B (en) | 2018-12-10 | 2018-12-10 | Three-dimensional visual sign language recognition device and multi-information fusion sign language recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109460748B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112734712A (en) * | 2020-12-31 | 2021-04-30 | 武汉第二船舶设计研究所(中国船舶重工集团公司第七一九研究所) | Imaging detection method and system for health state of ship vibration equipment |
| CN113197403A (en) * | 2021-05-14 | 2021-08-03 | 广东华联云谷科技研究院有限公司 | Method capable of preventing virus infection and smart bracelet |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101527092A (en) * | 2009-04-08 | 2009-09-09 | 西安理工大学 | Computer assisted hand language communication method under special session context |
| KR20160109708A (en) * | 2015-03-12 | 2016-09-21 | 주식회사 디지털스케치 | Sign language translator, system and method |
| CN108960158A (en) * | 2018-07-09 | 2018-12-07 | 珠海格力电器股份有限公司 | A kind of system and method for intelligent sign language translation |
| CN209980267U (en) * | 2018-12-10 | 2020-01-21 | 内蒙古科技大学 | Three-vision sign language recognition device |
-
2018
- 2018-12-10 CN CN201811501014.6A patent/CN109460748B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101527092A (en) * | 2009-04-08 | 2009-09-09 | 西安理工大学 | Computer assisted hand language communication method under special session context |
| KR20160109708A (en) * | 2015-03-12 | 2016-09-21 | 주식회사 디지털스케치 | Sign language translator, system and method |
| CN108960158A (en) * | 2018-07-09 | 2018-12-07 | 珠海格力电器股份有限公司 | A kind of system and method for intelligent sign language translation |
| CN209980267U (en) * | 2018-12-10 | 2020-01-21 | 内蒙古科技大学 | Three-vision sign language recognition device |
Non-Patent Citations (1)
| Title |
|---|
| 孙丽娟;张立材;郭彩龙;: "基于视觉的手势识别技术", 计算机技术与发展, no. 10 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112734712A (en) * | 2020-12-31 | 2021-04-30 | 武汉第二船舶设计研究所(中国船舶重工集团公司第七一九研究所) | Imaging detection method and system for health state of ship vibration equipment |
| CN112734712B (en) * | 2020-12-31 | 2022-07-01 | 武汉第二船舶设计研究所(中国船舶重工集团公司第七一九研究所) | Imaging detection method and system for health state of ship vibration equipment |
| CN113197403A (en) * | 2021-05-14 | 2021-08-03 | 广东华联云谷科技研究院有限公司 | Method capable of preventing virus infection and smart bracelet |
| CN113197403B (en) * | 2021-05-14 | 2023-02-17 | 广州乾睿医疗科技有限公司 | Method capable of preventing virus infection and smart bracelet |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109460748B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang et al. | Gesture recognition based on binocular vision | |
| CN106067190B (en) | A kind of generation of fast face threedimensional model and transform method based on single image | |
| CN107358648B (en) | Real-time full-automatic high quality three-dimensional facial reconstruction method based on individual facial image | |
| CN105094337B (en) | A kind of three-dimensional gaze estimation method based on iris and pupil | |
| CN105320271B (en) | It is calibrated using the head-mounted display of direct Geometric Modeling | |
| CN104978548B (en) | A kind of gaze estimation method and device based on three-dimensional active shape model | |
| CN104794722A (en) | Dressed human body three-dimensional bare body model calculation method through single Kinect | |
| CN105427385A (en) | High-fidelity face three-dimensional reconstruction method based on multilevel deformation model | |
| CN110555408B (en) | Single-camera real-time three-dimensional human body posture detection method based on self-adaptive mapping relation | |
| CN106625673A (en) | Narrow space assembly system and assembly method | |
| CN107041585A (en) | The measuring method of human dimension | |
| CN110074788B (en) | Body data acquisition method and device based on machine learning | |
| CN108305321B (en) | Three-dimensional human hand 3D skeleton model real-time reconstruction method and device based on binocular color imaging system | |
| CN109579695A (en) | A kind of parts measurement method based on isomery stereoscopic vision | |
| CN112232310B (en) | Face recognition system and method for expression capture | |
| CN109685855A (en) | A kind of camera calibration optimization method under road cloud monitor supervision platform | |
| CN109460748A (en) | A kind of trinocular vision hand language recognition device and multi-information fusion sign Language Recognition Method | |
| CN107016697A (en) | A kind of height measurement method and device | |
| CN109947253A (en) | The method for establishing model of eyeball tracking, eyeball tracking method, equipment, medium | |
| CN111811462A (en) | Large-component portable visual ranging system and method in extreme environment | |
| JPH0351407B2 (en) | ||
| Su et al. | Cross-validated locally polynomial modeling for 2-D/3-D gaze tracking with head-worn devices | |
| CN113129451B (en) | Holographic three-dimensional image space quantitative projection method based on binocular vision positioning | |
| Bérard et al. | Practical Person‐Specific Eye Rigging | |
| CN107765840A (en) | A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |