CN103558910B - A kind of intelligent display system of automatic tracking head pose - Google Patents
A kind of intelligent display system of automatic tracking head pose Download PDFInfo
- Publication number
- CN103558910B CN103558910B CN201310487330.3A CN201310487330A CN103558910B CN 103558910 B CN103558910 B CN 103558910B CN 201310487330 A CN201310487330 A CN 201310487330A CN 103558910 B CN103558910 B CN 103558910B
- Authority
- CN
- China
- Prior art keywords
- image
- head
- angle
- coordinate system
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The present invention relates to a kind of intelligent display system of automatic tracking head pose, this system can be caught human body head attitude in real time, and make display follow following head moving, and make display keep optimum position with head all the time, can alleviate eye fatigue and pre-myopia prevention. This system comprises image capture module, vision algorithm processing module and display control module. Image capture module is for carrying out Real-time Collection to human body head image; First vision algorithm module carries out pretreatment to image, make head in image, remain vertical position, then utilize ASM active shape model algorithm to extract the characteristic point of image Zhong Ren face, finally obtain the spatial attitude of head according to principle of triangulation, i.e. luffing angle and side tilt angle. Head pose information reaches after steering wheel control module, forms pwm signal control steering wheel and make display follow head pose in two frees degree of pitching and inclination after single-chip microcomputer resolves.
Description
Technical field
The present invention relates to a kind of intelligent display system, be specifically related to a kind of intelligent display system of automatic tracking head pose, belong to intelligent identification technology field.
Background technology
Display, as the indispensable human-computer interaction terminal of computer, is seen everywhere in our Working Life. Present display attitude (being visible angle) only can manual adjustments, can not change and automatically adjust according to user's head pose. Human eye laterally clearly sight line region be 35 °, longitudinally Clear Line Of Sigh region is 20 °. The change of head pose, makes Clear Line Of Sigh area change; If now display attitude is constant, is difficult to clear vision and has easily caused eye fatigue. Not only bother but frequently carry out manual adjustments, can't ensure that display can just in time be positioned at Clear Line Of Sigh region.
Want to realize display and automatically follow head pose adjustment, must ensure the accuracy that head pose builds. In head pose building process, face characteristic point location is an extremely crucial step. Human face characteristic point is orientated the processing of facial image as and is analyzed the geological information that provides important, and computer is automatically oriented accurate location and the face outline etc. of each organ of face in a width facial image. Positioning feature point is the indispensable pretreatment link in face recognition technology, the whether accurate reliability that is directly connected to follow-up work of location. Identification based on facial image feature is extracted human face conventionally if position, yardstick and the ratio to each other of the organs such as eyes, eyebrow, nose and face are as feature, can use further geometry matching human face, thereby using geometric parameter as the feature of describing face.
According to the information type detecting, man face characteristic point positioning method is broadly divided into following 6 classes: (1) method based on half-tone information at present; (2) method based on priori rules; (3) method based on controlling; (4) method based on small echo; (5) 3D method; (6) method based on geometry.
On the whole, the method accuracy rate based on priori rules is lower, and high to picture quality requirement, the attitude of being expressed one's feelings illumination effect is large, but sometimes also has the advantage that operand is little, is applicable to being combined with additive method carrying out positioning feature point. The common advantage of the method based on statistical model and small echo is to require low to picture quality, the attitude of being expressed one's feelings illumination effect is little, and accuracy rate is high, the training to sample etc., and aspect also makes operand greatly improve, but these class methods are development trend and the direction of characteristic point positioning method. 3D method is at the man face characteristic point positioning method of processing in video sequence, also not overripened, needs into-step research. Although it is high that the method based on geometry also requires picture quality, the attitude of being expressed one's feelings illumination effect is large, and accuracy rate is higher, is the main flow algorithm in this field so far.
Summary of the invention
In view of this, the invention provides a kind of intelligent display system of automatic tracking head pose, can automatically change display attitude according to the variation of head pose by this system, allow it keep optimum position with head all the time, can effectively alleviate eye fatigue and pre-myopia prevention.
The intelligent display system of described automatic tracking head pose comprises image capture module, vision algorithm processing module and display control module; Described image capture module comprises monocular-camera, and display control module comprises display, steering wheel control module and rolls steering wheel.
Described monocular-camera is positioned at the centre position of display back upper place, and monocular shooting gathers display front user's head image according to setpoint frequency, and the image collecting is sent to vision algorithm processing module in real time. Described vision algorithm processing module is carried out distortion correction to the head image number receiving, and obtains the monocular image after proofreading and correct. Described vision algorithm processing module is carried out recognition of face and extracts facial feature points the monocular image after proofreading and correct, and the facial feature points in described monocular image is left eye characteristic point and right eye characteristic point.
In described vision algorithm processing module, set up and have image coordinate system o '-x ' y '; Described image coordinate system is taking the pixel in the monocular-camera imaging plane upper left corner as initial point, and horizontal direction is x axle, and vertical direction is y axle; Thus the coordinate of described left eye characteristic point under image coordinate system o '-x ' y ' be (x 'a,y′a), the coordinate of right eye characteristic point under image coordinate system o '-x ' y ' be (x 'b,y′b); Current head side inclination angle theta is:
The head side inclination angle calculating is sent to steering wheel control module by described vision algorithm processing module. Described steering wheel control module obtains after head side degree of tilt, transmits control signal to rolling steering wheel, rolls steering wheel and drives display to roll respective angles, makes display plane be parallel to user's face.
In described image capture module, also comprise by the left lens camera of same model and the right order binocular camera pair forming of making a video recording, in described display control module, also comprise pitch-control motor; Described left lens camera and right lens camera are positioned at the right and left of monocular-camera below, and three video cameras are " product " font and put.
The optical axis of described left lens camera and right lens camera is parallel, left lens camera and right lens camera be simultaneously according to the frequency collection display front user's who sets head image, and two width images of synchronization collection are sent to vision algorithm processing module in real time. Described vision algorithm processing module is carried out distortion correction to the two width images that receive simultaneously, obtains the binocular image pair after proofreading and correct, and is respectively the right order image after left order image and the correction after correction. Described vision algorithm processing module is to the binocular image after proofreading and correct to carrying out respectively recognition of face and extracting respectively facial feature points, and the facial feature points of described binocular image centering is left eye characteristic point, right eye characteristic point and mouth mid point.
In described vision algorithm processing module, set up the image coordinate system o that has left lens cameral-xlyl, right lens camera image coordinate system or-xryrAnd three dimensions three-dimensional coordinate system o-xyz; Wherein said image coordinate system ol-xlyl, image coordinate system or-xryrMethod for building up identical with the method for building up of image coordinate system o '-x ' y '; Described three dimensions three-dimensional coordinate system o-xyz is taking the projection centre of left lens camera as initial point, and xyz three axes becomes right-handed coordinate system, and z axle is the right primary optical axis direction of binocular camera.
Choose the width in left order image or the right order image after correction, carry out the calculating of the angle of pitch taking its three characteristic points as benchmark: to choose left order image after correction as example, under three dimensions three-dimensional coordinate system o-xyz, in the left order image after described correction, the coordinate of left eye characteristic point isThe coordinate of right eye characteristic point isThe coordinate of mouth mid point isWherein said xalFor left eye characteristic point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xarFor left eye characteristic point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa, xblFor right eye characteristic point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xbrFor right eye characteristic point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa, xclFor mouth mid point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xcrFor mouth mid point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa, T is the distance between left lens camera and right lens camera projection centre, f is the focal length of left lens camera and right lens camera.
If the mid point of left eye characteristic point and right eye characteristic point line is D point in the left order image after proofreading and correct, can obtain the coordinate (x that D is ordereddl,ydl,zdl); When the fore head angle of pitchFor:
The head angle of pitch calculating is sent to steering wheel control module by described vision algorithm processing module; Described steering wheel control module obtains after the head angle of pitch, transmits control signal to pitch-control motor, and pitch-control motor drives display pitching respective angles, makes display plane be parallel to user's face.
As the actual angle of heel θ obtaining through the processing of vision algorithm processing module after the i time detectioniBetween [5 ° ,+5 °] time, actual angle of heel is sent to steering wheel control module, and using current state as original state, make θ0=0, then enter following working cycles, until again there is the situation of actual angle of heel between [5 ° ,+5 °];
(a) if the angle of heel θ that vision algorithm processing module calculatesjIn [15 ° ,-5 °) ∪ (+5 ° ,+15 °] scope time, make current actual angle of heel θs=θj+θ0, and by actual angle of heel θsSend to steering wheel control module, then enter step (b);
If vision algorithm processing module cannot complete recognition of face, send wait instruction to steering wheel control module, roll steering wheel and be failure to actuate, display keeps current attitude constant, then enters step (c);
If the angle of heel θ that vision algorithm processing module calculatesjBetween [5 ° ,+5 °], make current actual angle of heel θs=θj+θ0, and by actual angle of heel θsSend to steering wheel control module, then enter step (c);
(b) make initial angle of heel θ0=θj+θ0, vision algorithm processing module is by the head image inclination-θ receiving next timej, then carry out the extraction of recognition of face, characteristic point and the calculating of angle of heel;
(c) vision algorithm processing module is directly carried out the extraction of recognition of face, characteristic point and the calculating of angle of heel to the head image receiving.
The angle of pitch that described monocular-camera and binocular camera are right and highly all adjustable.
Beneficial effect
(1) this system can catch the inclination attitude of human body head in real time, and utilizes steering wheel control module to realize display to follow its position of adjustment of head pose, makes display keep best line-of-sight position with head all the time, thus vision protection etc.
(2) increase binocular camera to rear, make this system can catch in real time the pitch attitude of human body head, and realize automatically following of display, thereby further improve its serviceability.
(3) head image that the utilization of vision algorithm processing module is obtained carries out pretreatment, can obtain larger recognition of face scope; Make the detection of side tilt angle be expanded to [40 ° ,+40 °]; And can reduce because the subconsciousness of head swings the display mistake causing and swing.
(4) described camera height and the angle of pitch is adjustable, can meet the user of different height and sitting posture, and obtains larger visual angle.
Brief description of the drawings
Fig. 1 is the control flow block diagram of system described in embodiment 2;
Fig. 2 is the scheme of installation of system described in embodiment 2;
Fig. 3 is described facial feature points distribution map;
Fig. 4 is head normal attitude and inclination attitude, pitch attitude contrast schematic diagram;
Fig. 5 is the schematic diagram of the three dimensions three-dimensional coordinate system of foundation.
Wherein: 1-monocular-camera, the left lens camera of 2-, the right lens camera of 3-, 4-support, 5-roll steering wheel, 6-pitch-control motor, 7-left eye characteristic point, 8-right eye characteristic point, 9-mouth mid point
Detailed description of the invention
Below in conjunction with drawings and Examples, the present invention is described in further detail.
For ease of describing, in this definition: below the head axes described in two embodiment be the mid point of human face's left eye and right eye line and the line of mouth mid point (being facial line of symmetry). Under normal sitting position, (upper body is in vertical state) head axes is perpendicular to horizontal plane, and the direction of head axes changes with the change of head pose. Described best line-of-sight position refers to that display plane is parallel to human face, and lateral edges place, a display left side (right side) rectilinear direction is parallel to the position of human body head axis all the time.
Embodiment 1:
In the time using computer, what user was prone to most is the inclination attitude of head, thereby causes facial off-target sight line position. Described inclination attitude refers to the attitude that makes head axes depart from vertical position when head swings, as shown in the middle figure of Fig. 4. The change that the system that the present embodiment provides can make display follow human body head inclination attitude realizes follows automatically, makes display keep best line-of-sight position with head all the time.
This system comprises image capture module, vision algorithm processing module and display control module. Wherein image capture module is that resolution ratio is 640 × 480 monocular-camera 1; Vision algorithm processing module is made up of main frame and peripheral circuit thereof; Display control module comprises display, rolls steering wheel 5, steering wheel control module and peripheral circuit. The annexation of its each several part is: monocular-camera 1 is arranged on display back upper place by camera mount 4, the angle of pitch of described monocular-camera 1 is adjustable, be convenient to carry out the image acquisition of different visual angles, by changing the luffing angle of video camera, make user's its head under normal sitting position of different height and sitting posture be positioned at all the time the mid portion (ensureing that video camera can collect complete head image) of monocular-camera collection image; Described camera mount height is adjustable simultaneously, to coordinate the adjustable characteristic of video camera luffing angle to expand its scope of application. Main frame is connected with monocular-camera by USB connecting line, is connected with steering wheel control module by serial communication module, and the control of steering wheel control module rolls steering wheel running. Described display is arranged on display bracket, is fixed with inclination steering wheel on display bracket simultaneously, and display is connected with inclination steering wheel by connecting axle, rolls the adjustment of attitude for realizing display.
The operation principle of this system is: described image capture module gathers human body head image by monocular-camera, and by its inner demarcating module to the image obtaining to correcting, then by correct after image send to vision algorithm processing module; Vision algorithm processing module is extracted the characteristic point in its image receiving based on active shape model (ASM) algorithm and binocular stereo vision principle, and calculates the angle that head rolls, and then the angle value of calculating is sent to display control module; The angle value that display control module rolls according to the head receiving, rolls steering wheel by control and changes display attitude, makes display keep best line-of-sight position with user's head all the time.
Below in conjunction with the monocular pattern in Fig. 1, the workflow of this system is described in detail:
Use first before this system, should first demarcate described monocular-camera, obtain monocular-camera calibrating parameters. The main task of camera calibration is to determine the inner parameter that characterizes video camera built-in attribute and the distortion parameter that characterizes tangential and radial distortion, utilizes calibrating parameters can remove pattern distortion.
(101) before using, adjust height and the angle of pitch of monocular-camera, make it adapt to user's height, ensure that user is in the time of normal sitting position, monocular-camera can complete collection its head image, and make the head image that the gathers centre position in image that monocular-camera gathers as far as possible. Then start monocular-camera, monocular-camera is with the frequency collection user head image of 15 frame/seconds, and gathered image is sent to vision algorithm processing module in real time.
(102) utilization of vision algorithm processing module is demarcated inner parameter and distortion parameter carry out distortion correction to the head image receiving, and obtain the head image after proofreading and correct.
(103) extraction of face feature point:
Because face feature point position does not change with expression shape change, can be used to characterize head pose. Face feature point needs to carry out recognition of face before extracting, and in the present embodiment, vision algorithm processing module adopts in the head image of now comparatively ripe ViolaJones object detector after correction carries out recognition of face. Described ViolaJones object detector can identify the face shown in the left figure of Fig. 4 (face of normal attitude) in image, and selects the position of face with rectangle frame. Face feature point extracts and used ASM algorithm, the face feature point obtaining in the present embodiment is left eye characteristic point 7(A point) and right eye characteristic point 8(B point), as shown in Figure 3.
(104) determine head side degree of tilt:
For carrying out the calculating of side tilt angle, in described vision algorithm processing module, set up and have image coordinate system o '-x ' y '. Image coordinate system described in the present embodiment is taking the pixel in the monocular-camera imaging plane upper left corner as initial point, taking horizontal direction as x axle (to the right for just), taking vertical direction as y axle (downwards as just). thus the coordinate of described left eye characteristic point 7 under image coordinate system o '-x ' y ' as (x 'a,y′a), the coordinate of right eye characteristic point 8 under image coordinate system o '-x ' y ' be (x 'b,y′b); Current head side degree of tilt is left eye characteristic point 7 and the line of right eye characteristic point 8 and the angle theta of y axle:
(105) vision algorithm processing module sends to steering wheel control module by serial communication by the head side degree of tilt calculating; Steering wheel control module obtains after head side degree of tilt, transmits control signal to rolling steering wheel, rolls steering wheel and drives display to roll respective angles, makes display and head keep best line-of-sight position. In the present embodiment, when the side tilt angle calculating on the occasion of time, display rolls left; When the side tilt angle calculating on the occasion of time, display rolls to the right.
But adopt above-mentioned flow process for the exceed ± face of 15 ° of inclination, recognition success rate is very low. If face None-identified, cannot obtain the position of face in image, thereby just cannot use ASM to obtain face feature point, therefore, introduce in the present embodiment a kind of method of expanding face detection range. This method is corrected for the face that rolls attitude, utilizes acquired head to roll attitude information, and side tilt angle θ is rotated compensation to image.
In the present embodiment, establishing the face angle of heel that does not add compensation is [5 ° ,+5 °]. If the actual angle of heel θ obtaining through the processing of vision algorithm processing module in the time detecting for the i timeiBetween [5 ° ,+5 °] time, actual angle of heel is sent to steering wheel control module, using current state as original state, make θ0=0, then enter following working cycles, until again there is the situation of actual angle of heel between [5 ° ,+5 °].
(a) if the angle of heel θ that vision algorithm processing module calculatesjIn [15 ° ,-5 °) ∪ (+5 ° ,+15 °] scope time, make current actual angle of heel θs=θj+θ0, and by actual angle of heel θsSend to steering wheel control module, then enter step (b);
If vision algorithm processing module cannot complete recognition of face, send wait instruction to steering wheel control module, roll steering wheel and be failure to actuate, display keeps current attitude constant, then enters step (c);
If the angle of heel θ that vision algorithm processing module calculatesjBetween [5 ° ,+5 °], make current actual angle of heel θs=θj+θ0, and by actual angle of heel θsSend to steering wheel control module, then enter step (c);
(b) make initial angle of heel θ0=θj+θ0, vision algorithm processing module is by the head image inclination-θ receiving next timej, then carry out the extraction of recognition of face, characteristic point and the calculating of angle of heel.
(c) vision algorithm processing module is directly carried out the extraction of recognition of face, characteristic point and the calculating of angle of heel to the head image receiving.
Due to the situation that always can find actual angle of heel in [5 ° ,+5 °] scope, therefore, system works is in the circulation of said process. Test shows, adopts said method can be expanded to [40 ° ,+40 °] the detection of side tilt angle, and can avoid head to swing fast the display causing swings fast, generally speaking, such swing is subconscious, belongs to the design and want the interference of filtering.
Embodiment 2:
In reality, head pose is the coupling of inclination, pitching and sidespin, for further improving the serviceability of this system, has increased the adjustment (sidespin attitude is not conventional attitude in the time using computer, ignores at this) of pitch attitude in the present embodiment; Consider to roll attitude and pitch attitude simultaneously, and the attitude in these two frees degree is carried out to independent analysis. Described pitch attitude refers to that head axes and computer plane angulation are not 0 attitude, as shown in the right figure in Fig. 4.
Based on this, in the system described in embodiment 1, increased pitch-control motor and the binocular camera that formed by left lens camera 2 and right lens camera 3 to and pitch-control motor. As shown in Figure 2, the change that this system can make display follow human body head inclination and pitch attitude realizes follows the intelligent display system of the automatic tracking head pose forming thus automatically, makes display keep best line-of-sight position with head all the time.
In the present embodiment, binocular camera is to selecting identical model, and resolution ratio is 640 × 480. Three video cameras are " product " font to be put, and is installed on camera mount, and wherein binocular camera is to being positioned at the right and left of monocular-camera below. The angle of pitch of described three video cameras and highly all adjustable. Pitch-control motor is arranged on display bracket, and display is connected with pitch-control motor by another root connecting axle, for realizing the adjustment of display pitch attitude. Main frame by USB connecting line and monocular-camera and binocular camera to being connected; Simultaneous computer main frame is connected with steering wheel control module by serial communication module, and the control of steering wheel control module rolls steering wheel and pitch-control motor running.
In the present embodiment, according to the difference of IMAQ mode, there are following two kinds of mode of operations: monocular work pattern and binocular mode of operation. Wherein, use monocular-camera to obtain head and roll attitude and realize display lag function under monocular mode of operation, specific implementation is identical with embodiment 1. Under binocular mode of operation, use binocular camera to obtaining head pitch attitude and rolling attitude and realize display lag function.
Below the binocular mode of operation of this system is described in detail.
Use first before this system, first to described binocular camera shooting to demarcating, obtain its calibrating parameters. The right calibrating parameters of described binocular camera shooting refers to the intrinsic parameters of the camera, external parameter and the distortion parameter that utilize camera marking method to obtain. Then utilize calibrating parameters can remove pattern distortion.
(201) adjust height and the angle of pitch of left lens camera and right lens camera, ensure that two camera optical axis are parallel; And make it make it adapt to the height using, ensure that the front user of display is in the time of normal sitting position, left lens camera and right lens camera can both its head images of complete collection. Then start binocular camera pair, binocular camera is to the frequency collection user head image with 15 frame/seconds simultaneously, and two width images of synchronization collection are sent to vision algorithm processing module in real time.
(202) inner parameter, distortion parameter and the external parameter that the utilization of vision algorithm processing module is demarcated carries out distortion correction to the two width images that receive simultaneously, obtains left order image and right order image after proofreading and correct.
(203) extraction of face feature point:
In the present embodiment, the extraction of face feature point adopts face recognition technology in the same manner as in Example 1 and human face characteristic point location technology, difference is, needs to obtain respectively 3 characteristic points in left order image and the right order image after correction: left eye characteristic point A, right eye characteristic point B and mouth mid point C in the present embodiment.
(204) determine head side degree of tilt luffing angle:
The acquisition methods of side tilt angle is identical with embodiment 1, and the left order image after selection is proofreaied and correct and any one in right order image are benchmark.
For calculating luffing angle, need in vision algorithm processing module, set up three dimensions three-dimensional coordinate system o-xyz, as shown in Figure 5. The method for building up of described three dimensions three-dimensional coordinate system o-xyz is: the image coordinate system o that sets up first respectively left lens cameral-xlyl, right lens camera image coordinate system or-xryr, method for building up is as the image coordinate system o '-x ' y ' in embodiment 1.
Described three dimensions three-dimensional coordinate system o-xyz is taking the projection centre of left lens camera as initial point, and the right primary optical axis direction of binocular camera is z axle, and xyz three axes becomes right-handed coordinate system. In the present embodiment, carry out the calculating of the angle of pitch taking three characteristic points in the left order image after proofreading and correct as benchmark. To choose left order image after correction as example, in the left order image under three dimensions three-dimensional coordinate system o-xyz after described correction, the coordinate of left eye characteristic point isThe coordinate of right eye characteristic point isThe coordinate of mouth mid point isWherein said xalFor left eye characteristic point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xarFor left eye characteristic point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa, xblFor right eye characteristic point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xbrFor right eye characteristic point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa, xclFor mouth mid point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xcrFor mouth mid point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa, T is the distance between left lens camera and right lens camera projection centre, f is the focal length of left lens camera and right lens camera.
The acquisition methods of current luffing angle is: the mid point of establishing A point and B point line is D point, can obtain the coordinate (x that D is orderedd,yd,zd); The head angle of pitchBe line and the angle of xOy plane, the i.e. angle of current head axes direction and xOy plane that D point and C are ordered:
(205) vision algorithm processing module reaches display control module by serial communication by head side degree of tilt and head luffing angle. Display control module obtains after head side degree of tilt and luffing angle, transmits control signal respectively to rolling steering wheel and pitch-control motor, rolls steering wheel and pitch-control motor and drives display rotation respective angles, makes display and head keep best line-of-sight position.
Need not compensate adjustment to the angle of pitch, angle of heel adjustment is with monocular pattern, and just two width images rotate simultaneously.
The above is only the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, do not departing under the prerequisite of the technology of the present invention principle; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (3)
1. an intelligent display system of automatically following the tracks of head pose, is characterized in that, comprises image capture module, visionAlgorithm processing module and display control module; Described image capture module comprises the left order shooting of monocular-camera, same modelThe binocular camera pair that machine and the shooting of right order form, display control module comprises display, steering wheel control module, pitch-control motorWith inclination steering wheel;
Described monocular-camera is positioned at the centre position of display back upper place, and described left lens camera and right lens camera are positioned at monocularThe right and left of video camera below, three video cameras are " product " font and put; Monocular shooting gathers display according to setpoint frequencyFront user's head image, and the image collecting is sent to vision algorithm processing module in real time; Described left lens cameraParallel with the optical axis of right lens camera, left lens camera and right lens camera make according to the frequency collection display front of setting simultaneouslyUser's head image, and two width images of synchronization collection are sent to vision algorithm processing module in real time;
Described vision algorithm processing module is carried out distortion correction to the head image number receiving, and obtains the monocular image after proofreading and correct;
Described vision algorithm processing module is carried out recognition of face and extracts facial feature points, described monocular the monocular image after proofreading and correctFacial feature points in image is left eye characteristic point and right eye characteristic point;
In described vision algorithm processing module, set up and have image coordinate system o '-x ' y '; Described image coordinate system is with monocular-camera imagingThe pixel in the plane upper left corner is initial point, and horizontal direction is x axle, and vertical direction is y axle; Described left eye characteristic point is at image thusCoordinate under coordinate system o '-x ' y ' be (x 'a，y′a), the coordinate of right eye characteristic point under image coordinate system o '-x ' y ' be (x 'b, y ' b); Work asFore head angle of heel θ is:
Described vision algorithm processing module is carried out distortion correction to the two width images that receive simultaneously, obtains the binocular image after proofreading and correctRight, be respectively the right order image after left order image and the correction after correction;
Described vision algorithm processing module is also extracted respectively face feature to the binocular image after proofreading and correct to carrying out respectively recognition of facePoint, the facial feature points of described binocular image centering is left eye characteristic point, right eye characteristic point and mouth mid point;
In described vision algorithm processing module, set up the image coordinate system o that has left lens cameral-xlyl, right lens camera image sitMark is or-xryrAnd three dimensions three-dimensional coordinate system o-xyz; Wherein said image coordinate system ol-xlyl, image coordinate system or-xryr'sMethod for building up is identical with the method for building up of image coordinate system o '-x ' y '; Described three dimensions three-dimensional coordinate system o-xyz makes a video recording with left orderThe projection centre of machine is initial point, and xyz three axes becomes right-handed coordinate system, and z axle is the right primary optical axis direction of binocular camera;
Choose the width in left order image or the right order image after correction, carry out the angle of pitch taking its three characteristic points as benchmarkCalculate: to choose left order image after correction as example, under three dimensions three-dimensional coordinate system o-xyz, the left order figure after described correctionIn picture, the coordinate of left eye characteristic point isThe coordinate of right eye characteristic point isMouth mid pointCoordinate beWherein said xalFor left eye characteristic point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xarFor left eye characteristic point in the right order image after proofreading and correct is at image coordinate system or-xryrIn abscissa,xblFor right eye characteristic point in the left order image after proofreading and correct is at image coordinate system ol-xlylIn abscissa, xbrFor the right order after proofreading and correctIn image, right eye characteristic point is at image coordinate system or-xryrIn abscissa, xclFor mouth mid point in the left order image after proofreading and correct existsImage coordinate system ol-xlylIn abscissa, xcrFor mouth mid point in the right order image after proofreading and correct is at image coordinate system or-xryrInAbscissa, T is the distance between left lens camera and right lens camera projection centre, f is left lens camera and right lens cameraFocal length;
If the mid point of left eye characteristic point and right eye characteristic point line is D point in the left order image after proofreading and correct, can obtain that D ordersCoordinate (xdl,ydl,zdl); When the fore head angle of pitchFor:Described vision algorithm processing module will be countedThe head side inclination angle, the angle of pitch that obtain send to steering wheel control module;
Described steering wheel control module obtains after head side degree of tilt, transmits control signal to rolling steering wheel, rolls steering wheel and drives demonstrationDevice rolls respective angles, makes display plane be parallel to user's face, and described steering wheel control module obtains after the head angle of pitch,Transmit control signal to pitch-control motor, pitch-control motor drives display pitching respective angles, makes display plane be parallel to userFace.
2. the intelligent display system of automatic tracking head pose as claimed in claim 1, is characterized in that,
As the actual angle of heel θ obtaining through the processing of vision algorithm processing module after the i time detectioniBetween [5 ° ,+5 °] time, willActual angle of heel sends to steering wheel control module, and using current state as original state, makes θ0=0, then enter following workCirculate, until again there is the situation of actual angle of heel between [5 ° ,+5 °];
(a) if the angle of heel θ that vision algorithm processing module calculatesjIn [15 ° ,-5 °) ∪ (+5 ° ,+15 °] scope time,Make current actual angle of heel θs＝θj+θ0, and by actual angle of heel θsSend to steering wheel control module, then enter step (b);
If vision algorithm processing module cannot complete recognition of face, send wait instruction to steering wheel control module, roll steering wheel notAction, display keeps current attitude constant, then enters step (c);
If the angle of heel θ that vision algorithm processing module calculatesjBetween [5 ° ,+5 °], make current actual angle of heelθs＝θj+θ0, and by actual angle of heel θsSend to steering wheel control module, then enter step (c);
(b) make initial angle of heel θ0＝θj+θ0, vision algorithm processing module is by the head image inclination-θ receiving next timej, soLaggard row recognition of face, the extraction of characteristic point and the calculating of angle of heel;
(c) vision algorithm processing module is directly carried out extraction and the inclination of recognition of face, characteristic point to the head image receivingThe calculating at angle.
3. the intelligent display system of automatic tracking head pose according to claim 1 and 2, is characterized in that, instituteState the right angle of pitch of monocular-camera and binocular camera and highly all adjustable.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310487330.3A CN103558910B (en) | 2013-10-17 | 2013-10-17 | A kind of intelligent display system of automatic tracking head pose |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310487330.3A CN103558910B (en) | 2013-10-17 | 2013-10-17 | A kind of intelligent display system of automatic tracking head pose |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103558910A CN103558910A (en) | 2014-02-05 |
| CN103558910B true CN103558910B (en) | 2016-05-11 |
Family
ID=50013187
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310487330.3A Active CN103558910B (en) | 2013-10-17 | 2013-10-17 | A kind of intelligent display system of automatic tracking head pose |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103558910B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105389570A (en) * | 2015-11-19 | 2016-03-09 | 吴建忠 | Face angle determination method and system |
| CN106991367B (en) | 2016-01-21 | 2019-03-19 | 腾讯科技（深圳）有限公司 | The method and apparatus for determining face rotational angle |
| CN105975076A (en) * | 2016-05-09 | 2016-09-28 | 刘瑞 | Digital art design lab |
| RU2652721C2 (en) * | 2016-06-08 | 2018-04-28 | Публичное акционерное общество "Ракетно-космическая корпорация "Энергия" имени С.П. Королёва" | Control method of astronaut's actions on spacecraft |
| CN106060514A (en) * | 2016-07-06 | 2016-10-26 | 中国计量大学 | Device and method for controlling pose of television in multi-vision mode |
| CN106354263A (en) * | 2016-09-09 | 2017-01-25 | 电子科技大学 | Real-time man-machine interaction system based on facial feature tracking and working method of real-time man-machine interaction system |
| CN106445133B (en) * | 2016-09-20 | 2020-12-15 | 惠州Tcl移动通信有限公司 | Display adjustment method and system for tracking face movement |
| CN106580470B (en) * | 2016-10-18 | 2019-04-02 | 南京医科大学附属口腔医院 | Head localization method and system based on binocular vision |
| WO2018170592A1 (en) * | 2017-03-20 | 2018-09-27 | Exact Imaging Inc. | Method and system for visually assisting an operator of an ultrasound system |
| CN106919360B (en) * | 2017-04-18 | 2020-04-14 | 珠海全志科技股份有限公司 | Head posture compensation method and device |
| CN107424392B (en) * | 2017-06-06 | 2022-12-27 | 徐君逸 | Myopia prevention reminding device |
| CN108646912A (en) * | 2018-04-17 | 2018-10-12 | 上海与德通讯技术有限公司 | Show the method for adjustment and display equipment of the hanger of equipment |
| CN110896609B (en) * | 2018-09-27 | 2023-09-26 | 武汉资联虹康科技股份有限公司 | TMS positioning navigation method for transcranial magnetic stimulation treatment |
| CN109260593B (en) * | 2018-09-27 | 2020-09-08 | 武汉资联虹康科技股份有限公司 | Transcranial magnetic stimulation treatment method and equipment |
| CN110896610A (en) * | 2018-09-27 | 2020-03-20 | 武汉资联虹康科技股份有限公司 | Transcranial magnetic stimulation diagnosis and treatment equipment |
| CN109240428B (en) * | 2018-11-10 | 2023-08-04 | 深圳市时造电子科技有限公司 | Intelligent regulation all-in-one display screen system |
| CN109376691B (en) * | 2018-11-22 | 2022-03-01 | 华南师范大学 | Face tracking method, face tracking device, storage medium and equipment |
| CN109523755A (en) * | 2018-12-17 | 2019-03-26 | 石家庄爱赛科技有限公司 | Stereoscopic vision sitting posture reminder and based reminding method |
| CN109595438A (en) * | 2019-01-28 | 2019-04-09 | 西安驰高电子科技有限公司 | Display intelligent Supports Made and its control method |
| CN110309750A (en) * | 2019-06-24 | 2019-10-08 | 火石信科(广州)科技有限公司 | A method of scene is read and write based on monocular camera automatic identification |
| CN111880577A (en) * | 2020-07-21 | 2020-11-03 | 东莞市霍晶光电科技有限公司 | Lamp steering control method, device and system and computer readable storage medium |
| CN114100085B (en) * | 2020-09-01 | 2022-12-02 | 大爱医生集团(深圳)有限公司 | Intelligent interactive motion human body decompression method and device |
| CN112107125B (en) * | 2020-09-18 | 2022-07-15 | 电子科技大学 | Intelligent sitting posture correction method based on face recognition |
| CN112432311A (en) * | 2020-11-13 | 2021-03-02 | 青岛海尔空调器有限总公司 | Air conditioner display device, air conditioner and control method thereof |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102053629A (en) * | 2010-10-29 | 2011-05-11 | 冠捷显示科技（厦门）有限公司 | Method and method for realizing position automatic adjustment |
| CN102117074A (en) * | 2009-12-31 | 2011-07-06 | 鸿富锦精密工业（深圳）有限公司 | System for regulating angle of display and using method thereof |
| CN102237013A (en) * | 2010-04-29 | 2011-11-09 | 鸿富锦精密工业（深圳）有限公司 | Display screen adjustment system and method, and advertisement board with adjustment system |
| CN102323829A (en) * | 2011-07-29 | 2012-01-18 | 青岛海信电器股份有限公司 | Display screen visual angle regulating method and display device |
| CN102547209A (en) * | 2012-02-06 | 2012-07-04 | 华为技术有限公司 | Method and device for controlling videoconferencing equipment, and videoconferencing system |
| CN102572282A (en) * | 2012-01-06 | 2012-07-11 | 鸿富锦精密工业（深圳）有限公司 | Intelligent tracking device |
| CN103235645A (en) * | 2013-04-25 | 2013-08-07 | 上海大学 | Standing type display interface self-adaption tracking regulating device and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8031272B2 (en) * | 2007-07-19 | 2011-10-04 | International Business Machines Corporation | System and method of adjusting viewing angle for display |
| CN101470446B (en) * | 2007-12-27 | 2011-06-08 | 佛山普立华科技有限公司 | Display equipment and method for automatically regulating display direction |
-
2013
- 2013-10-17 CN CN201310487330.3A patent/CN103558910B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117074A (en) * | 2009-12-31 | 2011-07-06 | 鸿富锦精密工业（深圳）有限公司 | System for regulating angle of display and using method thereof |
| CN102237013A (en) * | 2010-04-29 | 2011-11-09 | 鸿富锦精密工业（深圳）有限公司 | Display screen adjustment system and method, and advertisement board with adjustment system |
| CN102053629A (en) * | 2010-10-29 | 2011-05-11 | 冠捷显示科技（厦门）有限公司 | Method and method for realizing position automatic adjustment |
| CN102323829A (en) * | 2011-07-29 | 2012-01-18 | 青岛海信电器股份有限公司 | Display screen visual angle regulating method and display device |
| CN102572282A (en) * | 2012-01-06 | 2012-07-11 | 鸿富锦精密工业（深圳）有限公司 | Intelligent tracking device |
| CN102547209A (en) * | 2012-02-06 | 2012-07-04 | 华为技术有限公司 | Method and device for controlling videoconferencing equipment, and videoconferencing system |
| CN103235645A (en) * | 2013-04-25 | 2013-08-07 | 上海大学 | Standing type display interface self-adaption tracking regulating device and method |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于单目视觉的移动机器人室内导航方法;付梦印等;《光学技术》;20060731;第32卷(第4期);第591-593页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103558910A (en) | 2014-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103558910B (en) | A kind of intelligent display system of automatic tracking head pose | |
| CN108427503B (en) | Human eye tracking method and human eye tracking device | |
| CN106529495B (en) | Obstacle detection method and device for aircraft | |
| CN106022304B (en) | A kind of real-time body's sitting posture situation detection method based on binocular camera | |
| CN102831392B (en) | Device for remote iris tracking and acquisition, and method thereof | |
| US10201273B2 (en) | Method for determining ocular measurements using a consumer sensor | |
| CN103366157B (en) | A kind of determination methods of human eye sight distance | |
| CN105447853A (en) | Flight device, flight control system and flight control method | |
| CN104732191B (en) | The devices and methods therefor of virtual display Eye-controlling focus is realized using Cross ration invariability | |
| WO2023071884A1 (en) | Gaze detection method, control method for electronic device, and related devices | |
| WO2020125499A1 (en) | Operation prompting method and glasses | |
| CN107958479A (en) | A kind of mobile terminal 3D faces augmented reality implementation method | |
| CN106056092A (en) | Gaze estimation method for head-mounted device based on iris and pupil | |
| CN108731587A (en) | A kind of the unmanned plane dynamic target tracking and localization method of view-based access control model | |
| CN105095893A (en) | Image acquisition device and method | |
| CN110309782A (en) | It is a kind of based on infrared with visible light biocular systems living body faces detection methods | |
| CN105094337A (en) | Three-dimensional gaze estimation method based on irises and pupils | |
| CN110232389A (en) | A kind of stereoscopic vision air navigation aid based on green crop feature extraction invariance | |
| CN105631859B (en) | Three-degree-of-freedom bionic stereo visual system | |
| CN104317385A (en) | Gesture identification method and system | |
| CN105809654A (en) | Target object tracking method and device, and stereo display equipment and method | |
| CN102831601A (en) | Three-dimensional matching method based on union similarity measure and self-adaptive support weighting | |
| CN105913013A (en) | Binocular vision face recognition algorithm | |
| CN112666705A (en) | Eye movement tracking device and eye movement tracking method | |
| CN106060514A (en) | Device and method for controlling pose of television in multi-vision mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |