CN107507247A - A kind of real-time dynamic autoization scaling method of projected keyboard - Google Patents
A kind of real-time dynamic autoization scaling method of projected keyboard Download PDFInfo
- Publication number
- CN107507247A CN107507247A CN201710752495.7A CN201710752495A CN107507247A CN 107507247 A CN107507247 A CN 107507247A CN 201710752495 A CN201710752495 A CN 201710752495A CN 107507247 A CN107507247 A CN 107507247A
- Authority
- CN
- China
- Prior art keywords
- button
- projected keyboard
- coordinate
- camera
- row
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
A kind of real-time dynamic autoization scaling method of projected keyboard, it is related to a kind of automation scaling method of projected keyboard.In order to solve the problems, such as that current scaling method has larger calibrated error.The present invention shoots the picture of a LED scaling board first with identification camera, calculates the Intrinsic Matrix of identification camera and the distortion factor of distortion model；Projected keyboard is projected in a moveable plane, the relative position relation that real-time calibration goes out between identification camera and projected keyboard, and calculate the image coordinate of reference-calibrating point；Identification camera and projected keyboard relative position relation are calculated according to projected keyboard and determine the world coordinates of reference-calibrating point；Transformation equation of the reference-calibrating point from camera coordinates to world coordinate system is established according to the Transformation Relation of Projection, obtains projective transformation matrix；Then system of the world coordinate of each button is calculated according to the world coordinate system of projected keyboard size and reference-calibrating point.The automation that the present invention is applied to projected keyboard is demarcated.
Description
Technical field
The present invention relates to a kind of automation scaling method of projected keyboard.
Background technology
Input equipment (keyboard, mouse) is the important tool of computer equipment or the man-machine interaction of other smart machines,
With the miniaturization of computer equipment or wearable smart machine, the products of many electronic applications accomplishes input equipment small-sized
Change and be even miniaturized, but input equipment is smaller to cause the inconvenient trouble of input to people, but still can not meet it is mobile,
Portable demand, there are some computer equipments or wearable smart machine to employ projected keyboard and set as input at present
It is standby.
But the staking-out work of projected keyboard times so has problems, the demarcation for projected keyboard at present is substantially all
It is using manually demarcating, in actual calibration process, using manually demarcating, and needs manually in projected keyboard
Each button is demarcated, and different personnel are demarcated and inherently make calibration result different, bring error so that key
The accuracy of disk demarcation is affected.For example, everyone finger is of different sizes, finger tip coordinate can be caused to extract inconsistent,
The accuracy of calibration result can be caused to decline.
Due to being provided with the equipment of projected keyboard, when projected keyboard is installed, certain installation always be present and miss
Difference, so the projected keyboard of demarcation is frequently present of error, and equipment using when also frequently encounter to collide with and cause projected keyboard
Deviation occurs for the relative position of grenade instrumentation, and then causes projected keyboard error in real process to be present.
The content of the invention
There is larger calibrated error in the present invention, and projected keyboard projection in order to solve the problems, such as current scaling method
When deviation occurs for the relative position of device there is the problem of larger error in projected keyboard.
A kind of real-time dynamic autoization scaling method of projected keyboard, including：
Step 1, camera calibration：
Camera calibration is the demarcation of identification camera, for carrying out fingertip area identification and positioning；
Demarcation, as LED scaling boards, is shot using the small lamp plates of LED known to the distributed dimension of customization using identification camera
The picture of one LED scaling board, utilize each lamp point on Threshold Segmentation Algorithm detection LED scaling boards, the barycenter of calculating lamp point
As the coordinate of calibration point in the picture, the i.e. image coordinate of calibration point；Simultaneously because the size of LED scaling boards is, it is known that LED is marked
Known to the world coordinates of each lamp point on fixed board；Sat according to the image coordinate of calibration point on known LED scaling boards and the world
Mark, calculates the Intrinsic Matrix of identification camera and the distortion factor of distortion model, that is, completes camera calibration；
Step 2, reference-calibrating point are chosen：
During real-time dynamic calibration, projected keyboard is projected in a moveable plane, and real-time calibration goes out identification and taken the photograph
As the relative position relation between head and projected keyboard is the outer parameter of identification camera, needed when calculating relative position relation
In the projection plane of projected keyboard, 4 reference-calibrating points, four tops of position and projected keyboard pattern of 4 reference-calibrating points are set
Point is coincided, and the projection plane picture including all 4 reference-calibrating points is included using camera shooting；
Spot area corresponding to 4 reference-calibrating points difference in projection plane picture is extracted using Threshold Segmentation Algorithm,
And calculate the figure of coordinate, i.e. reference-calibrating point of the center of the minimum external square of spot area as reference-calibrating point in the picture
As coordinate；
Step 3, the world coordinates regulation of datum mark：
Projected keyboard overall dimensions and each button size are calculating identification camera, it is known that the long x0 of projected keyboard, wide y0
With, it is specified that projected keyboard left upper apex world coordinates is (x, y, z), the right vertices world is sat during projected keyboard relative position relation
Mark (x+x0, y, z), bottom left vertex world coordinates (x, y+y0, z), bottom right vertex world coordinates (x+x0, y+y0, z), and with this
World coordinates of four coordinates as four reference-calibrating points in step 2；
Step 4, the demarcation of outer dynamic state of parameters：
The image coordinate that datum mark is demarcated in step 2 is multiplied to obtain with the Intrinsic Matrix of identification camera in step 1
Coordinate of the reference-calibrating point in camera coordinates system, after being corrected according to distortion model calculating reference-calibrating point under camera coordinates system
Coordinate；
Reference-calibrating point coordinates under camera coordinate system closes with demarcating datum mark world coordinates in step 3 for projective transformation
System, establishes transformation equation of four reference-calibrating points from camera coordinates to world coordinate system, finally according to the Transformation Relation of Projection
Solve the unknowm coefficient in equation and obtain projective transformation matrix, described projective transformation matrix be used for describing identification camera with
Relative position relation between projected keyboard；
Step 5, projected keyboard calculate：
Calculated according to the world coordinate system of projected keyboard overall dimensions, each button size and four reference-calibrating points each
System of the world coordinate of individual button, the keyboard table of generation.
Step 6, coordinate matching：
Projected keyboard is operated by finger tip, infrared laser is provided with the grenade instrumentation bottom of projected keyboard, is used
In to finger tip progress light compensation；
The projecting region of finger tip retroreflective regions brightness, is calculated using Threshold segmentation in the image that identification camera captures
The retroreflective regions of finger tip are partitioned into image captured by method from identification camera, and calculate the figure of the barycenter of finger tip retroreflective regions
As coordinate；Finally by the image coordinate of finger tip retroreflective regions barycenter and the identification camera Intrinsic Matrix phase demarcated in step 1
Coordinate of the multiplied barycenter to finger tip retroreflective regions in the case where identifying camera coordinate system, using distortion model to finger tip retroreflective regions
Barycenter camera coordinate system under coordinate be corrected, by the barycenter camera coordinate system of the finger tip retroreflective regions after correction
Under coordinate obtain the world coordinates of finger tip reflective areas barycenter multiplied by with parameter matrix outside the identification camera demarcated in step 4,
Complete the positioning of fingertip area；
Tabled look-up in the keyboard table that step 5 generates, the world coordinates of button coordinate and finger tip reflective areas barycenter is entered
Row matching, realizes the response of projected keyboard and mouse.
Preferably, described in step 5 according to projected keyboard overall dimensions, each button size and four reference-calibrating points
The process of world coordinate system system of the world coordinate that calculates each button comprise the following steps：
Projected keyboard includes the mouse region on the right side of 6 row's buttons and all buttons；If 6 row's buttons of projected keyboard are set respectively
There are n1, n2, n3, n4, n5 and n6 buttons, calculate button coordinate in the following way：
First row button coordinate calculates：
The width of each button in first row n1 buttons is x1, and button is highly h1, and the gap between button level is
D1, the world coordinates of projected keyboard left upper apex is (x, y, z), and the left upper apex abscissa of first row button is (x1+d1) *
(i1-1), wherein i1 represents the sequence number of first row button, and the left upper apex ordinate of first row button is 0；First row button
Distance with second row button is s1；
Second row button coordinate calculates：
The width of n2-1 buttons is x1 before second row, and the width of last button is x2=, all buttons of second row
Highly it is h2, the gap between button level is d2；Second row button left upper apex abscissa is (x1+d2) * (i2-1), wherein
I2 represents the sequence number of second row button, and the left upper apex ordinate of second row button is (h1+s1)；
According to the 3rd length and width for draining into the 6th each button design of row, calculated with reference to second row button coordinate numerical procedure
The world coordinates of each button；
Mouse region calculates：
Mouse region is calculated according to the world coordinates on four, mouse region summit.
Preferably, the identification camera described in step 1 is wide-angle camera.
Preferably, the process institute of the Intrinsic Matrix of identification camera and the distortion factor of distortion model is calculated in step 1
The distortion model stated is Brown distortion models.
Preferably, the Intrinsic Matrix of calculating identification camera and the distortion factor of distortion model described in step 1
Process is realized using Zhang Zhengyou calibration algorithms.
Preferably, the coordinate institute after reference-calibrating point corrects under camera coordinates system is calculated according to distortion model in step 4
The distortion model stated is Brown distortion models.
Preferably, the last unknowm coefficient solved in equation described in step 4 obtains the process of projective transformation matrix
It is to be realized using Newton method.
Preferably, the identification camera is equipped with optical filter；The wavelength of light and optical filter of the infrared laser transmitting
It is consistent by wavelength.
Preferably, totally 8 rows 23 arrange the lampet on the small lamp plates of described LED, level interval between lamp and lamp and it is vertical between
Away from being 15mm.
The invention has the advantages that：
As long as camera captures a two field picture in the present invention, projected keyboard will once be demarcated, realized dynamic
State real-time calibration, much sooner, demarcation effect is more accurate for demarcation；Even if device runs into collide with after projected keyboard grenade instrumentation
When deviation occurs for relative position, or in some environment to jolt, the relative position between identification camera and projected keyboard
Constantly change, the present invention can also demarcate in time, and demarcation effect is also more accurate, possesses broader practice scene.
Meanwhile the automatic Calibration of projected keyboard is carried out using the present invention, and it is tender enough to save a part of manual operation, not only save
Human cost, while also improve demarcation efficiency, and calibration process eliminate personnel's difference caused by calibration result it is different
The problem of, largely avoid the error manually demarcated and brought.Demarcated for the projected keyboard image of 200 frames, it is right
The demarcation rate of accuracy reached of projected keyboard is to more than 99%.
And the standard of all projected keyboards is consistent, makes the standards such as the device parameter with projected keyboard consistent, be advantageous to
The automation installation and correction of equipment, further ensure equipment uniformity and qualification rate.
The variation that the position relationship between projected keyboard and camera in the present invention can be, i.e. identification camera with
Projected keyboard grenade instrumentation can separate, and putting position is any；So not only set more flexibly, projected provided with projected keyboard
The design of the equipment of device is more flexible, is also more easy to realize.
Brief description of the drawings
Fig. 1 is the LED scaling board images of identification camera shooting；
Fig. 2 is 4 reference-calibrating dot images of identification camera capture；
Fig. 3 is the real-time dynamic calibration schematic flow sheet of projected keyboard.
Embodiment
Embodiment one：Illustrate present embodiment with reference to Fig. 3,
A kind of real-time dynamic autoization scaling method of projected keyboard, including：
Step 1, camera calibration：
Camera calibration is the demarcation of identification camera, for carrying out fingertip area identification and positioning；
Demarcation, as LED scaling boards, is shot using the small lamp plates of LED known to the distributed dimension of customization using identification camera
The picture of one LED scaling board, as shown in figure 1, using each lamp point on Threshold Segmentation Algorithm detection LED scaling boards, calculate
The image coordinate of coordinate, i.e. calibration point of the barycenter of lamp point as calibration point in the picture；Simultaneously because the size of LED scaling boards
, it is known that known to the world coordinates of each lamp point on LED scaling boards；Sat according to the image of calibration point on known LED scaling boards
Mark and world coordinates, calculate the Intrinsic Matrix of identification camera and the distortion factor of distortion model, that is, complete camera calibration；
Step 2, reference-calibrating point are chosen：
During real-time dynamic calibration, it (can also be in the plane that projected keyboard, which is projected in a moveable plane,
The keyboard pattern of drafting), therefore projected keyboard and identification camera relative position are no longer fixed, it is necessary to which real-time calibration goes out identification takes the photograph
As the relative position relation between head and projected keyboard is the outer parameter of identification camera, needed when calculating relative position relation
In the projection plane of projected keyboard, 4 reference-calibrating points, four tops of position and projected keyboard pattern of 4 reference-calibrating points are set
Point coincides, and 4 reference-calibrating points can use infrared LED, facilitate identification camera to detect, using camera shooting comprising complete
Projection plane picture including 4, portion reference-calibrating point, shooting picture are as shown in Figure 2；
Spot area corresponding to 4 reference-calibrating points difference in projection plane picture is extracted using Threshold Segmentation Algorithm,
And calculate the figure of coordinate, i.e. reference-calibrating point of the center of the minimum external square of spot area as reference-calibrating point in the picture
As coordinate；
Step 3, the world coordinates regulation of datum mark：
Projected keyboard overall dimensions and each button size are calculating identification camera, it is known that the long x0 of projected keyboard, wide y0
With, it is specified that projected keyboard left upper apex world coordinates is (x, y, z), the right vertices world is sat during projected keyboard relative position relation
Mark (x+x0, y, z), bottom left vertex world coordinates (x, y+y0, z), bottom right vertex world coordinates (x+x0, y+y0, z), and with this
World coordinates of four coordinates as four reference-calibrating points in step 2；
Such as：Projected keyboard overall dimensions and each button size wide 101mm, are being counted, it is known that the long 357mm of projected keyboard
, it is specified that projected keyboard left upper apex is (in the user of projected keyboard when calculation identification camera and projected keyboard relative position relation
Direction on, the lower right corner hot spot in projected keyboard left upper apex corresponding diagram 2) world coordinates is (0,0,0), right vertices (Fig. 2
Middle lower left corner hot spot) world coordinates (357,0,0), bottom left vertex (upper right corner hot spot in Fig. 2) world coordinates (0,101,0), the right side
Lower summit (upper left corner hot spot in Fig. 2) world coordinates (357,101,0), and it is used as four corrections in step 2 using this four coordinates
The world coordinates of datum mark；
Step 4, the demarcation of outer dynamic state of parameters：
The image coordinate that datum mark is demarcated in step 2 is multiplied to obtain with the Intrinsic Matrix of identification camera in step 1
Coordinate of the reference-calibrating point in camera coordinates system, after being corrected according to distortion model calculating reference-calibrating point under camera coordinates system
Coordinate；
Reference-calibrating point coordinates under camera coordinate system closes with demarcating datum mark world coordinates in step 3 for projective transformation
System, establishes transformation equation of four reference-calibrating points from camera coordinates to world coordinate system, finally according to the Transformation Relation of Projection
Solve the unknowm coefficient in equation and obtain projective transformation matrix, described projective transformation matrix be used for describing identification camera with
Relative position relation between projected keyboard；
Step 5, projected keyboard calculate：
Calculated according to the world coordinate system of projected keyboard overall dimensions, each button size and four reference-calibrating points each
System of the world coordinate of individual button, the keyboard table of generation.
Embodiment two：
A kind of real-time dynamic autoization scaling method of projected keyboard, including：
Step 1, camera calibration：
Camera calibration is the demarcation of identification camera, for carrying out fingertip area identification and positioning；
Demarcation, as LED scaling boards, is shot using the small lamp plates of LED known to the distributed dimension of customization using identification camera
The picture of one LED scaling board, as shown in figure 1, using each lamp point on Threshold Segmentation Algorithm detection LED scaling boards, calculate
The image coordinate of coordinate, i.e. calibration point of the barycenter of lamp point as calibration point in the picture；Simultaneously because the size of LED scaling boards
, it is known that known to the world coordinates of each lamp point on LED scaling boards；Sat according to the image of calibration point on known LED scaling boards
Mark and world coordinates, calculate the Intrinsic Matrix of identification camera and the distortion factor of distortion model, that is, complete camera calibration；
Step 2, reference-calibrating point are chosen：
During real-time dynamic calibration, it (can also be in the plane that projected keyboard, which is projected in a moveable plane,
The keyboard pattern of drafting), therefore projected keyboard and identification camera relative position are no longer fixed, it is necessary to which real-time calibration goes out identification takes the photograph
As the relative position relation between head and projected keyboard is the outer parameter of identification camera, needed when calculating relative position relation
In the projection plane of projected keyboard, 4 reference-calibrating points, four tops of position and projected keyboard pattern of 4 reference-calibrating points are set
Point coincides, and 4 reference-calibrating points can use infrared LED, facilitate identification camera to detect, using camera shooting comprising complete
Projection plane picture including 4, portion reference-calibrating point, shooting picture are as shown in Figure 2；
Spot area corresponding to 4 reference-calibrating points difference in projection plane picture is extracted using Threshold Segmentation Algorithm,
And calculate the figure of coordinate, i.e. reference-calibrating point of the center of the minimum external square of spot area as reference-calibrating point in the picture
As coordinate；
Step 3, the world coordinates regulation of datum mark：
Projected keyboard overall dimensions and each button size are calculating identification camera, it is known that the long x0 of projected keyboard, wide y0
With, it is specified that projected keyboard left upper apex world coordinates is (x, y, z), the right vertices world is sat during projected keyboard relative position relation
Mark (x+x0, y, z), bottom left vertex world coordinates (x, y+y0, z), bottom right vertex world coordinates (x+x0, y+y0, z), and with this
World coordinates of four coordinates as four reference-calibrating points in step 2；
Such as：Projected keyboard overall dimensions and each button size wide 101mm, are being counted, it is known that the long 357mm of projected keyboard
, it is specified that projected keyboard left upper apex is (in the user of projected keyboard when calculation identification camera and projected keyboard relative position relation
Direction on, the lower right corner hot spot in projected keyboard left upper apex corresponding diagram 2) world coordinates is (0,0,0), right vertices (Fig. 2
Middle lower left corner hot spot) world coordinates (357,0,0), bottom left vertex (upper right corner hot spot in Fig. 2) world coordinates (0,101,0), the right side
Lower summit (upper left corner hot spot in Fig. 2) world coordinates (357,101,0), and it is used as four corrections in step 2 using this four coordinates
The world coordinates of datum mark；
Step 4, the demarcation of outer dynamic state of parameters：
The image coordinate that datum mark is demarcated in step 2 is multiplied to obtain with the Intrinsic Matrix of identification camera in step 1
Coordinate of the reference-calibrating point in camera coordinates system, after being corrected according to distortion model calculating reference-calibrating point under camera coordinates system
Coordinate；
Reference-calibrating point coordinates under camera coordinate system closes with demarcating datum mark world coordinates in step 3 for projective transformation
System, establishes transformation equation of four reference-calibrating points from camera coordinates to world coordinate system, finally according to the Transformation Relation of Projection
Solve the unknowm coefficient in equation and obtain projective transformation matrix, described projective transformation matrix be used for describing identification camera with
Relative position relation between projected keyboard；
Step 5, projected keyboard calculate：
Calculated according to the world coordinate system of projected keyboard overall dimensions, each button size and four reference-calibrating points each
System of the world coordinate of individual button, the keyboard table of generation.
Step 6, coordinate matching：
Projected keyboard is operated by finger tip, infrared laser is provided with the grenade instrumentation bottom of projected keyboard, is used
In to finger tip progress light compensation so that imaging of the finger tip in identification camera becomes apparent from；
The brightness of finger tip retroreflective regions can be significantly hotter than peripheral region in the image that identification camera captures, using threshold value
The retroreflective regions of finger tip are partitioned into image captured by partitioning algorithm from identification camera, and calculate the matter of finger tip retroreflective regions
The image coordinate of the heart；The identification camera intrinsic parameter that will finally be demarcated in the image coordinate of finger tip retroreflective regions barycenter and step 1
Matrix multiple obtains coordinate of the barycenter of finger tip retroreflective regions in the case where identifying camera coordinate system, because identification camera is wide-angle
Camera, the coordinate under the camera coordinate system of the barycenter of finger tip retroreflective regions is corrected using Brown distortion models, will
Coordinate under the barycenter camera coordinate system of finger tip retroreflective regions after correction is multiplied by with outside the identification camera demarcated in step 4
Parameter matrix obtains the world coordinates of finger tip reflective areas barycenter, completes the positioning of fingertip area；
Tabled look-up in the keyboard table that step 5 generates, the world coordinates of button coordinate and finger tip reflective areas barycenter is entered
Row matching, realizes the response of projected keyboard and mouse.
Embodiment three：
Described in present embodiment step 5 according to projected keyboard overall dimensions, each button size and four demarcation bases
The process that world coordinate system on schedule calculates system of the world coordinate of each button comprises the following steps：
Projected keyboard includes the mouse region on the right side of 6 row's buttons and all buttons；If 6 row's buttons of projected keyboard are set respectively
There are n1, n2, n3, n4, n5 and n6 buttons, calculate button coordinate in the following way：
First row button coordinate calculates：
The width of each button in first row n1 buttons is x1, and button is highly h1, and the gap between button level is
D1, the world coordinates of projected keyboard left upper apex is (x, y, z), and the left upper apex abscissa of first row button is (x1+d1) *
(i1-1), wherein i1 represents the sequence number of first row button, and the left upper apex ordinate of first row button is 0；First row button
Distance with second row button is s1；
Second row button coordinate calculates：
The width of n2-1 buttons is x1 before second row, and the width of last button is x2=, all buttons of second row
Highly it is h2, the gap between button level is d2；Second row button left upper apex abscissa is (x1+d2) * (i2-1), wherein
I2 represents the sequence number of second row button, and the left upper apex ordinate of second row button is (h1+s1)；
According to the 3rd length and width for draining into the 6th each button design of row, calculated with reference to second row button coordinate numerical procedure
The world coordinates of each button；
Mouse region calculates：
Mouse region is calculated according to the world coordinates on four, mouse region summit.
Such as projected keyboard first row has 15 buttons, second row has 14 buttons, and the 3rd row has 14 buttons, the 4th row
There are 13 buttons, the 5th row there are 13 buttons, and the 6th row there are 10 buttons；
A. first row button coordinate calculates
The width of first row button is 16.6mm, and button is highly 11mm, and the gap between button level is 1.4mm, by
The world coordinates that left upper apex is provided when identification camera and projected keyboard relative position is being calculated above is (0,0,0),
Therefore the left upper apex abscissa of first row button is (16.6+1.4) * (i1-1), and wherein i1 represents the sequence number of first row button,
I1 is more than or equal to 1 and is less than or equal to 15, and the left upper apex ordinate of first row button is 0；Often arranging the distance between button is
1.4mm；
B. second row button coordinate calculates
The width of 13 buttons is 16.6mm before second row, and the width of the 14th button is 34.6mm, and second row is all to be pressed
The height of key is 16.6mm, therefore second row button left upper apex abscissa is (16.6+1.4) * (i2-1), and wherein i2 represents the
The sequence number of two row's buttons, i2 are more than or equal to 1 and are less than or equal to 14, and the left upper apex ordinate of second row button is 12.4mm；
C. the 3rd row's button coordinate calculates
3rd to arrange the 1st button and the width of the 14th button be 25.6mm, and the width of the 2nd to the 13rd button is
16.6mm, the 3rd all buttons of row are highly 16.6mm, therefore the 3rd arranges the 1st button left upper apex abscissa as the 0, the 2nd
Abscissa to the 14th button left upper apex is 25.6+1.4+ (16.6+1.4) * (i3-2), and wherein i3 represents the 3rd row's button
Sequence number, i3 be more than or equal to 2 be less than or equal to 14, the left upper apex ordinate of the 3rd row's button is 30.4；
D. it is 30.1mm that the 4th row's button coordinate, which calculates the 4th width for arranging the 1st button, and the width of the 13rd button is
39.1mm, the width of the 2nd to the 12nd button are 16.6mm, and the 4th all buttons of row are highly 16.6mm, therefore the 4th row
1st button left upper apex abscissa is that the 0, the 2nd to the 13rd button left upper apex abscissa is 30.1+1.4+ (16.6+
1.4) * (i4-2), wherein i4 represent the sequence number of the 4th row's button, and i4 is more than or equal to 2 and is less than or equal to 13, the upper left of the 4th row's button
Summit ordinate is 48.4；
E. the 5th row's button coordinate calculates
5th width for arranging the 1st button is 39.1mm, and the width of the 11st button is 21.1mm, the width of the 13rd button
Spend for 25.6mm, the width of the 2nd to the 10th button and the 12nd button is 16.6mm, and the 5th all buttons of row are highly
16.6mm, therefore the 5th arranges the 1st button left upper apex abscissa as the 0, the 2nd to the 11st button left upper apex abscissa and is
39.1+1.4+ (16.6+1.4) * (i5-2), wherein i5 represent the sequence number of the 5th row's button, and i5 is more than or equal to 2 and is less than or equal to 11,
12nd button left upper apex abscissa is 39.1+1.4+ (16.6+1.4) * 9+ (21.1+1.6) mm, and the 13rd button upper left is pushed up
Point abscissa is 39.1+1.4+ (16.6+1.4) * 10+ (21.1+1.6) mm, and the 5th row's button left upper apex ordinate is 66.4；
F. the 6th row's button coordinate calculates
6th width for arranging the 1st button is 25.6mm, and the width of the 5th button is 88.6mm, the width of the 10th button
Spend for 25.6mm, the width of the 2nd to the 4th and the 6th to the 9th button is 16.6mm, and the height of the 6th all buttons of row is
16.6mm, therefore the 6th arranges the 1st button left upper apex abscissa as the 0, the 2nd to the 5th button left upper apex abscissa and is
25.6+1.4+ (16.6+1.4) * (i6-2), wherein i6 are the sequence number of the 6th row's button, and i6 is more than or equal to 2 and is less than or equal to the 5, the 6th
It is 25.6+1.4+ (16.6+1.4) * 3+88.6+ (16.6+1.4) * (i7-6) to the 9th button left upper apex abscissa, wherein
I7 is the sequence number of the 6th row's button, and it is 25.6+88.6 that i7, which is more than or equal to 6 less than or equal to the 9, the 10th button left upper apex abscissa,
+ 1.4*9+16.6*7, the 6th row's button left upper apex ordinate are 84.4；
G. mouse region calculates
Mouse region, which calculates, only to be needed to provide the world coordinates on four summits in mouse region, according to mouse region upper left
Summit world coordinates is (270,0), and right vertices world coordinates is (357,0), and bottom left vertex world coordinates is (270,101),
Bottom right vertex world coordinates is (357,101).
Other steps and parameter are identical with embodiment one or two.
Embodiment four：
Identification camera described in present embodiment step 1 is 120 degree of wide-angle cameras.
Other steps and parameter are identical with one of embodiment one to three.
Embodiment five：
The process of the Intrinsic Matrix of identification camera and the distortion factor of distortion model is calculated in present embodiment step 1
Described distortion model is Brown distortion models.
Other steps and parameter are identical with one of embodiment one to four.
Embodiment six：
The Intrinsic Matrix of calculating identification camera and the distortion factor of distortion model described in present embodiment step 1
Process using Zhang Zhengyou calibration algorithms realize, i.e., using Zhang Zhengyou calibration algorithms calculate identification camera Intrinsic Matrix and
The distortion factor of distortion model, that is, complete camera calibration.
Other steps and parameter are identical with one of embodiment one to five.
Embodiment seven：
The coordinate after reference-calibrating point corrects under camera coordinates system is calculated according to distortion model in present embodiment step 4
Described distortion model is Brown distortion models.Because identification camera is 120 degree of wide-angle cameras, distortion be present, so root
The coordinate after reference-calibrating point corrects under camera coordinates system is calculated according to Brown distortion models.
Other steps and parameter are identical with one of embodiment one to six.
Embodiment eight：
The last unknowm coefficient solved in equation described in present embodiment step 4 obtains the mistake of projective transformation matrix
Journey is realized using Newton method.
Other steps and parameter are identical with one of embodiment one to seven.
Embodiment nine：
Identification camera described in present embodiment is equipped with optical filter, and the effect of filter plate is to make in particular range of wavelengths
Light can be captured to, can be with the interference of filtering environmental light；The wavelength of light and optical filter of the infrared laser transmitting
It is consistent by wavelength.
Other steps and parameter are identical with one of embodiment one to eight.
Embodiment ten：
Totally 8 rows 23 arrange the lampet on the small lamp plates of LED described in present embodiment, level interval between lamp and lamp and vertical
Spacing is 15mm.
Other steps and parameter are identical with one of embodiment one to nine.
Claims (10)
- A kind of 1. real-time dynamic autoization scaling method of projected keyboard, it is characterised in that including：Step 1, camera calibration：Camera calibration is the demarcation of identification camera, for carrying out fingertip area identification and positioning；Demarcation, as LED scaling boards, one is shot using identification camera using the small lamp plates of LED known to the distributed dimension of customization The picture of LED scaling boards, using each lamp point on Threshold Segmentation Algorithm detection LED scaling boards, calculate the barycenter conduct of lamp point The coordinate of calibration point in the picture, the i.e. image coordinate of calibration point；Simultaneously because the size of LED scaling boards is, it is known that LED scaling boards On each lamp point world coordinates known to；According to the image coordinate and world coordinates of calibration point on known LED scaling boards, meter The Intrinsic Matrix of identification camera and the distortion factor of distortion model are calculated, that is, completes camera calibration；Step 2, reference-calibrating point are chosen：During real-time dynamic calibration, projected keyboard is projected in a moveable plane, and real-time calibration goes out identification camera Relative position relation between projected keyboard is the outer parameter of identification camera, needs throwing when calculating relative position relation The projection plane of shadow keyboard sets 4 reference-calibrating points, four summit phases of position and projected keyboard pattern of 4 reference-calibrating points Overlap, the projection plane picture including all 4 reference-calibrating points is included using camera shooting；Spot area corresponding to 4 reference-calibrating points difference in projection plane picture is extracted using Threshold Segmentation Algorithm, and counted Coordinate of the center of the minimum external square of spot area as reference-calibrating point in the picture is calculated, i.e. the image of reference-calibrating point is sat Mark；Step 3, the world coordinates regulation of datum mark：Projected keyboard overall dimensions and each button size are calculating identification camera with throwing, it is known that the long x0 of projected keyboard, wide y0 , it is specified that projected keyboard left upper apex world coordinates is (x, y, z) during shadow keyboard relative position relation, right vertices world coordinates (x + x0, y, z), bottom left vertex world coordinates (x, y+y0, z), bottom right vertex world coordinates (x+x0, y+y0, z), and with this four World coordinates of the coordinate as four reference-calibrating points in step 2；Step 4, the demarcation of outer dynamic state of parameters：The image coordinate that datum mark is demarcated in step 2 is multiplied with the Intrinsic Matrix of identification camera in step 1 and demarcated Coordinate of the datum mark in camera coordinates system, the seat after reference-calibrating point corrects under camera coordinates system is calculated according to distortion model Mark；Reference-calibrating point coordinates under camera coordinate system and demarcation datum mark world coordinates in step 3 are the Transformation Relation of Projection, Transformation equation of four reference-calibrating points from camera coordinates to world coordinate system is established according to the Transformation Relation of Projection, finally solved Unknowm coefficient in equation obtains projective transformation matrix, and described projective transformation matrix is used for describing identification camera and projection Relative position relation between keyboard；Step 5, projected keyboard calculate：Each press is calculated according to the world coordinate system of projected keyboard overall dimensions, each button size and four reference-calibrating points System of the world coordinate of key, the keyboard table of generation.
- 2. the real-time dynamic autoization scaling method of a kind of projected keyboard according to claim 1, it is characterised in that also wrap Include：Step 6, coordinate matching：Projected keyboard is operated by finger tip, infrared laser is provided with the grenade instrumentation bottom of projected keyboard, for pair Finger tip carries out light compensation；The projecting region of finger tip retroreflective regions brightness in the image that identification camera captures, using Threshold Segmentation Algorithm from The retroreflective regions of finger tip are partitioned into image captured by identification camera, and the image for calculating the barycenter of finger tip retroreflective regions is sat Mark；It is finally that the image coordinate of finger tip retroreflective regions barycenter and the identification camera Intrinsic Matrix demarcated in step 1 is mutually multiplied To coordinate of the barycenter in the case where identifying camera coordinate system of finger tip retroreflective regions, the matter using distortion model to finger tip retroreflective regions Coordinate under the camera coordinate system of the heart is corrected, by under the barycenter camera coordinate system of the finger tip retroreflective regions after correction Coordinate obtains the world coordinates of finger tip reflective areas barycenter multiplied by with parameter matrix outside the identification camera demarcated in step 4, completes The positioning of fingertip area；Tabled look-up in the keyboard table that step 5 generates, by the progress of the world coordinates of button coordinate and finger tip reflective areas barycenter Match somebody with somebody, realize the response of projected keyboard and mouse.
- A kind of 3. real-time dynamic autoization scaling method of projected keyboard according to claim 1 or 2, it is characterised in that The world coordinate system meter according to projected keyboard overall dimensions, each button size and four reference-calibrating points described in step 5 The process for calculating system of the world coordinate of each button comprises the following steps：Projected keyboard includes the mouse region on the right side of 6 row's buttons and all buttons；If 6 row's buttons of projected keyboard are respectively equipped with N1, n2, n3, n4, n5 and n6 buttons, button coordinate is calculated in the following way：First row button coordinate calculates：The width of each button in first row n1 buttons is x1, and button is highly h1, and the gap between button level is d1, is thrown The world coordinates of shadow keyboard left upper apex is (x, y, z), and the left upper apex abscissa of first row button is (x1+d1) * (i1-1), Wherein i1 represents the sequence number of first row button, and the left upper apex ordinate of first row button is 0；First row button and second row The distance of button is s1；Second row button coordinate calculates：The width of n2-1 buttons is x1 before second row, and the width of last button is x2=, the height of all buttons of second row For h2, the gap between button level is d2；Second row button left upper apex abscissa is (x1+d2) * (i2-1), wherein i2 tables Show the sequence number of second row button, the left upper apex ordinate of second row button is (h1+s1)；According to the 3rd length and width for draining into the 6th each button design of row, calculated with reference to second row button coordinate numerical procedure each The world coordinates of button；Mouse region calculates：Mouse region is calculated according to the world coordinates on four, mouse region summit.
- A kind of 4. real-time dynamic autoization scaling method of projected keyboard according to claim 3, it is characterised in that step Identification camera described in 1 is wide-angle camera.
- A kind of 5. real-time dynamic autoization scaling method of projected keyboard according to claim 4, it is characterised in that step The distortion model calculated in 1 described in the process of the Intrinsic Matrix of identification camera and the distortion factor of distortion model is Brown Distortion model.
- A kind of 6. real-time dynamic autoization scaling method of projected keyboard according to claim 5, it is characterised in that step The process of the Intrinsic Matrix of calculating identification camera described in 1 and the distortion factor of distortion model is calculated using Zhang Zhengyou demarcation Method is realized.
- A kind of 7. real-time dynamic autoization scaling method of projected keyboard according to claim 6, it is characterised in that step Distortion model in 4 according to distortion model calculates the coordinate after reference-calibrating point corrects under camera coordinates system is Brown Distortion model.
- A kind of 8. real-time dynamic autoization scaling method of projected keyboard according to claim 7, it is characterised in that step The process that the last unknowm coefficient solved in equation described in 4 obtains projective transformation matrix is realized using Newton method.
- 9. the real-time dynamic autoization scaling method of a kind of projected keyboard according to claim 8, it is characterised in that described Identification camera is equipped with optical filter；The wavelength of light of the infrared laser transmitting is consistent by wavelength with optical filter.
- A kind of 10. real-time dynamic autoization scaling method of projected keyboard according to claim 9, it is characterised in that institute Totally 8 rows 23 arrange the lampet on the small lamp plates of LED stated, and the level interval and vertical interval between lamp and lamp are 15mm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710752495.7A CN107507247B (en) | 2017-08-28 | 2017-08-28 | A kind of real-time dynamic autoization scaling method of projected keyboard |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710752495.7A CN107507247B (en) | 2017-08-28 | 2017-08-28 | A kind of real-time dynamic autoization scaling method of projected keyboard |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107507247A true CN107507247A (en) | 2017-12-22 |
| CN107507247B CN107507247B (en) | 2018-09-11 |
Family
ID=60692808
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710752495.7A Active CN107507247B (en) | 2017-08-28 | 2017-08-28 | A kind of real-time dynamic autoization scaling method of projected keyboard |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107507247B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108495104A (en) * | 2018-03-21 | 2018-09-04 | 联想(北京)有限公司 | A kind of information processing method and device |
| CN108921797A (en) * | 2018-06-14 | 2018-11-30 | 合肥市商巨智能装备有限公司 | The calibration method of fault image |
| CN109978958A (en) * | 2019-03-27 | 2019-07-05 | 哈尔滨拓博科技有限公司 | A kind of interactive projection system calibrated error compensation method |
| CN110310332A (en) * | 2019-06-26 | 2019-10-08 | 珠海格力智能装备有限公司 | The scaling method and device of key |
| CN110413111A (en) * | 2019-07-09 | 2019-11-05 | 南京大学 | A kind of target keyboard tracing system and method |
| CN111340891A (en) * | 2020-02-25 | 2020-06-26 | 卡莱特（深圳）云科技有限公司 | Method and system for calibrating camera by using LED screen |
| CN112432593A (en) * | 2019-08-24 | 2021-03-02 | 上海翊威半导体有限公司 | Measuring method based on image recognition |
| CN112862895A (en) * | 2019-11-27 | 2021-05-28 | 杭州海康威视数字技术股份有限公司 | Fisheye camera calibration method, device and system |
| TWI789283B (en) * | 2022-03-31 | 2023-01-01 | 群光電子股份有限公司 | Inputting system for children assisted teaching and teaching order inputting device thereof |
| CN115580716A (en) * | 2022-12-09 | 2023-01-06 | 普赞加信息科技南京有限公司 | Projection picture output method, system and equipment based on physical module |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521822A (en) * | 2011-10-25 | 2012-06-27 | 南京大学 | Active light-emitting type target for automatic calibration based on machine vision and calibrating method thereof |
| US20150093042A1 (en) * | 2012-06-08 | 2015-04-02 | Huawei Technologies Co., Ltd. | Parameter calibration method and apparatus |
| CN106600648A (en) * | 2016-12-06 | 2017-04-26 | 合肥工业大学 | Stereo coding target for calibrating internal parameter and distortion coefficient of camera and calibration method thereof |
| CN106920261A (en) * | 2017-03-02 | 2017-07-04 | 江南大学 | A kind of Robot Hand-eye static demarcating method |
| CN106933376A (en) * | 2017-03-23 | 2017-07-07 | 哈尔滨拓博科技有限公司 | A kind of scaling method of smooth projected keyboard |
-
2017
- 2017-08-28 CN CN201710752495.7A patent/CN107507247B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521822A (en) * | 2011-10-25 | 2012-06-27 | 南京大学 | Active light-emitting type target for automatic calibration based on machine vision and calibrating method thereof |
| US20150093042A1 (en) * | 2012-06-08 | 2015-04-02 | Huawei Technologies Co., Ltd. | Parameter calibration method and apparatus |
| CN106600648A (en) * | 2016-12-06 | 2017-04-26 | 合肥工业大学 | Stereo coding target for calibrating internal parameter and distortion coefficient of camera and calibration method thereof |
| CN106920261A (en) * | 2017-03-02 | 2017-07-04 | 江南大学 | A kind of Robot Hand-eye static demarcating method |
| CN106933376A (en) * | 2017-03-23 | 2017-07-07 | 哈尔滨拓博科技有限公司 | A kind of scaling method of smooth projected keyboard |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108495104A (en) * | 2018-03-21 | 2018-09-04 | 联想(北京)有限公司 | A kind of information processing method and device |
| CN108921797A (en) * | 2018-06-14 | 2018-11-30 | 合肥市商巨智能装备有限公司 | The calibration method of fault image |
| CN108921797B (en) * | 2018-06-14 | 2021-07-13 | 合肥市商巨智能装备有限公司 | Method for calibrating distorted image |
| CN109978958A (en) * | 2019-03-27 | 2019-07-05 | 哈尔滨拓博科技有限公司 | A kind of interactive projection system calibrated error compensation method |
| CN109978958B (en) * | 2019-03-27 | 2020-05-22 | 哈尔滨拓博科技有限公司 | Calibration error compensation method for interactive projection system |
| CN110310332A (en) * | 2019-06-26 | 2019-10-08 | 珠海格力智能装备有限公司 | The scaling method and device of key |
| CN110413111A (en) * | 2019-07-09 | 2019-11-05 | 南京大学 | A kind of target keyboard tracing system and method |
| CN112432593A (en) * | 2019-08-24 | 2021-03-02 | 上海翊威半导体有限公司 | Measuring method based on image recognition |
| CN112862895A (en) * | 2019-11-27 | 2021-05-28 | 杭州海康威视数字技术股份有限公司 | Fisheye camera calibration method, device and system |
| CN112862895B (en) * | 2019-11-27 | 2023-10-10 | 杭州海康威视数字技术股份有限公司 | Fisheye camera calibration method, device and system |
| CN111340891B (en) * | 2020-02-25 | 2021-02-19 | 卡莱特（深圳）云科技有限公司 | Method and system for calibrating camera by using LED screen |
| CN111340891A (en) * | 2020-02-25 | 2020-06-26 | 卡莱特（深圳）云科技有限公司 | Method and system for calibrating camera by using LED screen |
| TWI789283B (en) * | 2022-03-31 | 2023-01-01 | 群光電子股份有限公司 | Inputting system for children assisted teaching and teaching order inputting device thereof |
| CN115580716A (en) * | 2022-12-09 | 2023-01-06 | 普赞加信息科技南京有限公司 | Projection picture output method, system and equipment based on physical module |
| CN115580716B (en) * | 2022-12-09 | 2023-09-05 | 普赞加信息科技南京有限公司 | Projection picture output method, system and equipment based on object module |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107507247B (en) | 2018-09-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107507247B (en) | A kind of real-time dynamic autoization scaling method of projected keyboard | |
| CN107656637B (en) | A kind of automation scaling method using the projected keyboard for choosing manually at 4 points | |
| CN107657642B (en) | A kind of automation scaling method carrying out projected keyboard using external camera | |
| US9800846B2 (en) | Gestural control of visual projectors | |
| US8870389B2 (en) | Image Projection system and image geometric correction device | |
| KR101187500B1 (en) | Light projection device and illumination device | |
| JP6896204B2 (en) | Devices to generate computer programs and how to generate computer programs | |
| CN102103435B (en) | Interactive electronic whiteboard device and positioning method thereof | |
| TWI577172B (en) | Image calibration system and calibration method of a stereo camera | |
| CN107018316B (en) | Image processing apparatus, image processing method, and storage medium | |
| CN106446888B (en) | Multi-identifier identification method and equipment for camera module | |
| US8937593B2 (en) | Interactive projection system and method for calibrating position of light point thereof | |
| US11380063B2 (en) | Three-dimensional distortion display method, terminal device, and storage medium | |
| JP2015043086A (en) | Projector clustering method, and management device and management system using the same | |
| JP6775179B2 (en) | Picking system | |
| CN105488807A (en) | Method for calibrating and rectifying telecentric lens | |
| US10812764B2 (en) | Display apparatus, display system, and method for controlling display apparatus | |
| CN111627073B (en) | Calibration method, calibration device and storage medium based on man-machine interaction | |
| CN104865893B (en) | Motion platform control system and motion platform error calculation method | |
| CN114663495A (en) | Calibration method and apparatus, head-mounted display device, and computer-readable storage medium | |
| JPWO2017179543A1 (en) | Information processing apparatus, information processing method, and program recording medium | |
| JP2016149678A (en) | Camera calibration unit, camera calibration method and camera calibration program | |
| CN103529939A (en) | Man-machine interaction method based on color, computer vision and grid division | |
| CN116330305A (en) | Multi-mode man-machine interaction assembly method, system, equipment and medium thereof | |
| CN104570557B (en) | Display apparatus and display method using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |