CN104881641A - Mobile device based digital recognition method and system for questionnaires and tables - Google Patents

Abstract

The invention provides a mobile device based digital recognition method and system for questionnaires and tables. First, based on angular point features of images of paper questionnaires and tables, sample images are trained and questionnaires or tables are separated from a background. Then, based on coordinate conversion in an OpenGL system, object coordinates are converted to screen coordinates and questionnaires and tables in different positions are corrected through affine transformation. Next, image processing is performed; options in the tables are digitalized in sequence; whether an option is selected or not is judged according to the proportion of a black zone of a selection frame in a whole zone; and a serial number of the selection option is output. Finally, the selection option and content are output according to the serial number and a locally constructed SQL library of option contents of the questionnaires or tables. The method and system can be used for recognizing questionnaires and tables in different backgrounds and exposed to different lights. The robustness is good. By using the method and system provided by the invention, one questionnaire can be generally recognized within a few seconds at a nearly 100 percent accuracy. Therefore, quickness and high efficiency are both achieved.

Description

Based on the questionnaire of mobile device and form Digital identification method and system

Technical field

What the present invention relates to is the system of a kind of machine learning and technical field of image processing, specifically a kind of questionnaire rapidly and efficiently based on mobile device and form Digital identification method and system.

Background technology

Questionnaire and form are widely used in society life.Such as carry out a new project or produce a new product, all will come the feasibility of analysis project or the demand of product by market survey.Or same in school of bank, usually need to fill in some forms to register information.Although network surveying and registration have become more and more popular now, the questionnaire of papery and form have also been absolutely necessary for investigation and statistical study.Owing to market identifying the equipment of papery questionnaire and form or applying very little, after these questionnaires or form are filled, the statistical study of questionnaire and form can only by manually carrying out, need a large amount of man power and materials, this had both consumed a large amount of man power and materials, also valuable in the waste time.So design needs one questionnaire rapidly and efficiently and Table recognition help staff complete statistics to the data of papery questionnaire and form and typing work.

Realize the digitizing identification of questionnaire and form, need to realize the segmentation of questionnaire and form, demarcation and identification three parts.For Iamge Segmentation, traditional method has based on color segmentation, differential motion detects segmentation and Adaboost training classifier is split.Due to papery questionnaire and form general adularescent and black two kinds of colors, the colouring information comprised is few, so adopt the effect of color segmentation bad; And identify that questionnaire and form generally adopt static identification by taking pictures, do not comprise the information of motion, so the effect that differential motion detects is also not satisfactory; Finally, questionnaire and form is identified by training Adaboost sorter, Adaboost sorter is a kind of training method inside machine learning, but a large amount of samples pictures of this need of work could determine good effect, and sorter is under complex environment and when illumination variation is larger, discrimination is lower; For image calibration, the general coordinate being calculated the questionnaire of papery and four angles of form by the intersection point of detection of straight lines, then picture is calibrated by affined transformation, but under complex environment, straight-line detection less stable, and the coordinate of the angle point calculated exists deviation, serious have impact on subsequent step.Quick Response Code knows the method that method for distinguishing demarcates picture in addition, and setting mark demarcates picture, but for questionnaire and form, established standards seems and do not sound feasible too complex feasible.Table recognition part, general idea judges selected option by Text region, but too complicated for the identification of word, and the discrimination of existing technology to word is lower, and feasibility is lower.

In existing patent, if application number is CN201310455065.0 Chinese invention patent, the patent provides a kind of Table recognition method and system, but this technology is by being partitioned into the element figure of form, adopt non-directed graph, extract the image in the page, the point of crossing of detection level and vertical line, the external envelope matrix of detected intersection, whether described segmentation line of text is fallen into external envelope matrix as local relation feature, then uses the method establishment Table Model of the machine learning such as cluster and SVM vector machine to identify form.The method just finds form in whole document, and any identification is not done for the content in form, be similar to the segmentation to form, and document vertically must be placed in the identifying of form, for document rotation with when blocking, discrimination is lower.And this technology is the pure identification to form, and does not do any process for the content in form, and the use for the data statistics of form is little.

Based on above-mentioned, need design a kind of new recognition methods and system, be first partitioned into questionnaire and form at complex environment, and to identify in form and questionnaire those by the content selected to facilitate the statistical study of data, increase practicality.

Summary of the invention

For above-mentioned the deficiencies in the prior art, the invention provides a kind of mobile device questionnaire rapidly and efficiently and form Digital identification method and system, whole questionnaire or form can be identified fast and efficiently, accuracy rate almost reaches absolutely, facilitates staff to the statistics of questionnaire and form data and typing.

For achieving the above object, the present invention is by the following technical solutions:

According to an aspect of the present invention, a kind of questionnaire based on mobile device and form Digital identification method are provided, comprise the steps:

Step one, the segmentation of questionnaire or form: according to the Corner Feature of papery questionnaire or form Image, uses VuforiaSDK training sample picture, in background complicated and changeable rapidly and efficiently be partitioned into questionnaire or form;

Step 2, the calibration of questionnaire or form: according to the coordinate transform in OpenGL system, is converted to screen coordinate by the object coordinates being partitioned into questionnaire or form, and calibrates questionnaire or the form of various position by affined transformation;

Step 3, the identification of questionnaire or form: image procossing is carried out to the questionnaire calibrated or form, in order by the option digitizing in table, whether the ratio accounting for whole region according to black region in choice box carrys out this option of interpretation selected, exports the number designation of selected option; The questionnaire set up according to number designation and this locality or the SQL storehouse of form option content, export selected option and content.

Preferably, described step, be specially: from complex background, be partitioned into questionnaire, obtain four apex coordinates of questionnaire or form, first corresponding storehouse is trained according to the Corner Feature of questionnaire or form, then starting Vuforia SDK in a mobile device uses the storehouse of training to identify questionnaire or form, adopts OpenGL to play up the interface of mobile device, calculate four three-dimensional coordinates of summit under the coordinate system set up centered by questionnaire or form of questionnaire or form in identifying.

Preferably, described step 2, be specially: the screen coordinate three-dimensional vertices coordinate of questionnaire or form being converted to mobile device by matrixing, namely the conversion of coordinate system in OpenGL is utilized, set up mould and look matrix (ModelviewMatrix) and projection matrix (Projection Matrix), and by the viewport transform (Viewport Transform), the three-dimensional coordinate under object coordinates system is converted to screen coordinate, then according to affined transformation, the questionnaire of diverse location or form are calibrated.

Further, described step 2, comprises following operation steps:

First the apex coordinate of questionnaire is obtained, namely at the coordinate (x of model coordinate systems _obj, y _obj, z _obj, w _obj), the coordinate (x of camera coordinate system _eye, y _eye, z _eye, w _eye) be multiplied by mould by model coordinate to obtain depending on matrix:

$\left(\begin{array}{c}{x}_{\mathrm{eye}}\\ y_{\mathrm{eye}}\\ z_{\mathrm{eye}}\\ w_{\mathrm{eye}}\end{array}\right)=M_{\mathrm{Modelview}}*\left(\begin{array}{c}{x}_{\mathrm{obj}}\\ y_{\mathrm{opj}}\\ z_{\mathrm{obj}}\\ w_{\mathrm{obj}}\end{array}\right)---\left(1\right)$

Eye coordinates obtains the coordinate (x on summit under cutting coordinate system by being multiplied by projection matrix _clip, y _clip, z _clip, w _clip):

$\left(\begin{array}{c}{x}_{\mathrm{clip}}\\ y_{\mathrm{clip}}\\ z_{\mathrm{clip}}\\ w_{\mathrm{clip}}\end{array}\right)=M_{\mathrm{projection}}*\left(\begin{array}{c}{x}_{\mathrm{eye}}\\ y_{\mathrm{eye}}\\ z_{\mathrm{eye}}\\ w_{\mathrm{eye}}\end{array}\right)---\left(2\right)$

By the coordinate under reduction coordinate system divided by w _clip, namely obtain normalized device coordinate (x _ndc, y _ndc, z _ndc), this coordinate just obtains screen coordinate through Pan and Zoom, just this coordinate conversion is become screen coordinate (x by the viewport transform (Viewport Transform) _w, y _w, z _w):

$\left(\begin{array}{c}{x}_w\\ y_w\\ z_w\end{array}\right)=\left(\begin{array}{c}\frac{w}{2}{x}_{\mathrm{ndc}}+(x+\frac{w}{2})\\ \frac{h}{2}{y}_{\mathrm{ndc}}+(y+\frac{h}{2})\\ \frac{f-n}{2}{z}_{\mathrm{ndc}}+\left(\frac{f+n}{2}\right)\end{array}\right)---\left(3\right)$

Wherein (x, y, w, h) is starting point and the size of screen window, (x, y) is the coordinate of screen window starting point, and w is the width of screen, and h is the height of screen, (n, f) is the depth range of screen, n →-1 under linear case, f → 1.

By affined transformation on mobile device screen by the questionnaire of diverse location or form calibration.The matrix of the affined transformation wherein used is:

Translation transformation: point (x, y) on screen is moved to point (x+dx, y+dy), dx is the distance of horizontal ordinate translation, and dy is the distance of ordinate translation, and transformation matrix is:

$M_{\mathrm{translate}}=\left(\begin{array}{ccc}1& 0& \mathrm{dx}\\ 0& 1& \mathrm{dy}\\ 0& 0& 1\end{array}\right)---\left(4\right)$

Rotational transform: point (x, y) on screen is rotated counterclockwise θ around far point, and transformation matrix is:

$M_{\mathrm{rotate}}=\left(\begin{array}{ccc}\cos \mathrm{\θ}& -\sin \mathrm{\θ}& 0\\ \sin \mathrm{\θ}& \cos \mathrm{\θ}& 0\\ 0& 0& 1\end{array}\right)---\left(5\right)$

Scale transformation: the horizontal ordinate of point (x, y) on screen is zoomed in or out original sx doubly, ordinate zooms in or out original sy doubly, and transformation matrix is:

$M_{\mathrm{scale}}=\left(\begin{array}{ccc}\mathrm{sx}& 0& 0\\ 0& \mathrm{sy}& 0\\ 0& 0& 1\end{array}\right)---\left(6\right)$

Preferably, described step 3, be specially: first by the content data of whole questionnaire and form, namely the database of questionnaire content is set up according to order from top to bottom and from left to right, the option of questionnaire is represented with numeral, then the position of the choice box of each option is calibrated, save the data in local file, finally by the calibrated picture of adaptive thresholding, pass through threshold process, corrosion, expand, the morphological operations such as corrosion are by picture binaryzation, then calculate blacking part in each option area and account for the ratio in whole region, if ratio is greater than a certain threshold value, judge that this option is selected, export the numeral representing this option, then option and the content thereof of this digitized representation is matched in a database, and Output rusults.

According to a further aspect in the invention, provide a kind of questionnaire based on mobile device and form digitizing recognition system, described system comprises: the partitioning portion of questionnaire or form, calibrated section and identification division, wherein:

The partitioning portion of described questionnaire or form: according to the Corner Feature of papery questionnaire or form Image, uses VuforiaSDK training sample picture, in background complicated and changeable rapidly and efficiently be partitioned into questionnaire or form;

The calibrated section of described questionnaire or form: according to the coordinate transform in OpenGL system, is converted to screen coordinate by the object coordinates being partitioned into questionnaire or form, and calibrates questionnaire or the form of various position by affined transformation;

The identification division of described questionnaire or form: image procossing is carried out to the questionnaire calibrated or form, in order by the option digitizing in table, whether the ratio accounting for whole region according to black region in choice box carrys out this option of interpretation selected, exports the number designation of selected option; The questionnaire set up according to number designation and this locality or the SQL storehouse of form option content, export selected option and content.

Compared with prior art, the present invention has following beneficial effect:

The present invention processes for interference of different nature, and speed and discrimination all improve a lot.Whole system has stronger robustness, can split and questionnaire under identifying complex background and different light situation; Also have rapidly and efficiently property, the identification for a questionnaire and form generally only spends the time in a few second simultaneously, accurately almost reaches absolutely.

Accompanying drawing explanation

By reading the detailed description done non-limiting example with reference to the following drawings, other features, objects and advantages of the present invention will become more obvious:

Fig. 1 be in one embodiment of the invention under varying environment to the segmentation schematic diagram of papery questionnaire;

Fig. 2 is the calibration schematic diagram to papery questionnaire under varying environment in one embodiment of the invention;

Fig. 3 is digitizing and the recognition result schematic diagram of questionnaire in one embodiment of the invention;

Fig. 4 identifies schematic diagram in one embodiment of the invention;

Fig. 5 is method identification process figure in one embodiment of the invention;

Fig. 6 is the angle point that in one embodiment of the invention, Vuforia SDK trains;

Fig. 7 is coordinate transform procedure Procedure figure in one embodiment of the invention.

Embodiment

Below in conjunction with specific embodiment, the present invention is described in detail.Following examples will contribute to those skilled in the art and understand the present invention further, but not limit the present invention in any form.It should be pointed out that to those skilled in the art, without departing from the inventive concept of the premise, some distortion and improvement can also be made.These all belong to protection scope of the present invention.

As shown in Figure 5, method flow of the present invention:

First according to picture Corner Feature, Vuforia SDK is adopted to set up the Corner Feature storehouse of questionnaire and form, for being partitioned into questionnaire and form in complex background;

Then according to the conversion of coordinate system in OpenGL the three-dimensional coordinate of object is converted to the coordinate on mobile device screen, then by affined transformation by the questionnaire that splits or form calibration;

Then the picture of calibration is processed, by morphological operations such as binaryzation, corrosion and expansions, calculate each option blacking part and account for the ratio of whole Option Box to judge that whether option is selected, and export the number designation of selected option, then the option digitizing of whole questionnaire, setting up local data base, carrying out output intent option content by mating selected number designation, and then the information of statistics and typing questionnaire and form.

Based on above flow process, system corresponding to the present invention comprises following components: the partitioning portion of questionnaire or form, calibrated section and identification division.Below in conjunction with specific embodiment, various piece is described in detail.

Partitioning portion: as a shown in Figure 6, adopts Vuforia SDK to identify form according to the unique point of questionnaire and form, and then be partitioned into questionnaire and form from the background of complexity; Vuforia SDK is mainly used in virtual reality, but it also may be used for some identification arranging object or the picture be made up of plane and tracking.Different with traditional matrixing and QR coupling, it is not identify picture according to the specific mark designed, but identifies picture by unique point that in picture, those are changed significantly and angle point.Such as a rectangle contains four unique points, and circle does not have unique point, as shown in Figure 6.By the coupling of these unique points, can identify questionnaire or form in various environment, be that system has very strong robustness.

In Fig. 1: under (a) represents normal illumination, to the identification of questionnaire under complex background and form; B () represents normal illumination under, the questionnaire blocked there being object and the identification of form; C when () represents that illumination is partially dark, to the identification having questionnaire and the form blocked.

Calibrated section: as Fig. 2, shown in 7, four three-dimensional coordinates of summit under the coordinate system set up centered by questionnaire of questionnaire can be obtained by partitioning portion, in order to reduce the error of coordinate produced due to detection, the present invention is by the method for mathematics, utilize the conversion of coordinate system in OpenGL, set up mould and look matrix (Model-View Matrix) and projection matrix (Projection Matrix), and by the viewport transform (Viewport Transform), the three-dimensional coordinate under object coordinates system is converted to screen coordinate, then by affined transformation, picture is calibrated,

In Fig. 2: (a), to the calibration result of (a) in Fig. 1, (b), to the calibration result of (b) in Fig. 1, (c) is to the calibration result of (c) in Fig. 1.

As shown in Fig. 5,7, as one preferably, the calibration of questionnaire or form, is implemented as follows:

The first step: the apex coordinate first obtaining questionnaire, namely at the coordinate (x of model coordinate systems _obj, y _obj, z _obj, w _obj), the coordinate (x of camera coordinate system _eye, y _eye, z _eye, w _eye) be multiplied by mould by model coordinate to obtain depending on matrix:

$\left(\begin{array}{c}{x}_{\mathrm{eye}}\\ y_{\mathrm{eye}}\\ z_{\mathrm{eye}}\\ w_{\mathrm{eye}}\end{array}\right)=M_{\mathrm{Modelview}}*\left(\begin{array}{c}{x}_{\mathrm{obj}}\\ y_{\mathrm{opj}}\\ z_{\mathrm{obj}}\\ w_{\mathrm{obj}}\end{array}\right)---\left(1\right)$

The mould used in calibration process depending on matrix is:

$M_{\mathrm{Modelview}}=\left(\begin{array}{cccc}-23.314& -579.434& -10.635& 0\\ -545.794& 20.371& 86.618& 0\\ -0.516& 0.024& -0.987& 0\\ 11.484& 16.001& 917.402& 1\end{array}\right)---\left(2\right)$

Second step: eye coordinates obtains the coordinate (x on summit under cutting coordinate system by being multiplied by projection matrix _clip, y _clip, z _clip, w _clip):

$\left(\begin{array}{c}{x}_{\mathrm{clip}}\\ y_{\mathrm{clip}}\\ z_{\mathrm{clip}}\\ w_{\mathrm{clip}}\end{array}\right)=M_{\mathrm{projection}}*\left(\begin{array}{c}{x}_{\mathrm{eye}}\\ y_{\mathrm{eye}}\\ z_{\mathrm{eye}}\\ w_{\mathrm{eye}}\end{array}\right)---\left(3\right)$

The projection matrix used in calibration process is:

$M_{\mathrm{projection}}=\left(\begin{array}{cccc}0& -1.690& 0& 0\\ -3.005& 0& 0& 0\\ 0& -0.002& 1.004& 1.0\\ 0& 0& -20.040& 0\end{array}\right)---\left(4\right)$

3rd step: by the coordinate under reduction coordinate system divided by w _clip, namely obtain normalized device coordinate (x _ndc, y _ndc, z _ndc), this coordinate just obtains screen coordinate through Pan and Zoom, just this coordinate conversion is become screen coordinate (x by the viewport transform (Viewport Transform) _w, y _w, z _w):

$\left(\begin{array}{c}{x}_w\\ y_w\\ z_w\end{array}\right)=\left(\begin{array}{c}\frac{w}{2}{x}_{\mathrm{ndc}}+(x+\frac{w}{2})\\ \frac{h}{2}{y}_{\mathrm{ndc}}+(y+\frac{h}{2})\\ \frac{f-n}{2}{z}_{\mathrm{ndc}}+\left(\frac{f+n}{2}\right)\end{array}\right)---\left(5\right)$

Wherein (x, y, w, h) is starting point and the size of screen window, (x, y) is the coordinate of screen window starting point, and w is the width of screen, and h is the height of screen, (n, f) is the depth range of screen, n →-1 under linear case, f → 1.

4th step: by affined transformation on mobile device screen by the questionnaire of diverse location or form calibration, the matrix of the affined transformation wherein used is:

Translation transformation: point (x, y) on screen is moved to point (x+dx, y+dy), dx is the distance of horizontal ordinate translation, and dy is the distance of ordinate translation, and transformation matrix is:

$M_{\mathrm{translate}}=\left(\begin{array}{ccc}1& 0& \mathrm{dx}\\ 0& 1& \mathrm{dy}\\ 0& 0& 1\end{array}\right)---\left(6\right)$

Rotational transform: point (x, y) is rotated counterclockwise θ around far point, transformation matrix is:

$M_{\mathrm{rotate}}=\left(\begin{array}{ccc}\cos \mathrm{\θ}& -\sin \mathrm{\θ}& 0\\ \sin \mathrm{\θ}& \cos \mathrm{\θ}& 0\\ 0& 0& 1\end{array}\right)---\left(7\right)$

Scale transformation: the horizontal ordinate of point (x, y) is zoomed in or out original sx doubly, ordinate zooms in or out original sy doubly, and transformation matrix is:

$M_{\mathrm{scale}}=\left(\begin{array}{ccc}\mathrm{sx}& 0& 0\\ 0& \mathrm{sy}& 0\\ 0& 0& 1\end{array}\right)---\left(8\right)$

Identification division: as shown in Figure 3, by the option of questionnaire and content data building database, then demarcates the position of the option of the questionnaire after calibrating or form and carries out image procossing to the picture after calibration, identifying by the option selected.

In Fig. 3: (a), according to the content of whole questionnaire or form, sets up the questionnaire corresponding with numeral or table database; B () demarcates whole questionnaire and all option position of form; (c) digitizing recognition result.

Fig. 4 illustrates the identification process of whole system: what (a) represented is partitioned into questionnaire or form under the background of complexity; B () illustrates the result after the questionnaire be partitioned into or form being calibrated; (c) represent to the questionnaire after calibration or the recognition result of form.

As one preferably, picture recognition comprises following step:

The first step: according to from top to bottom, the whole questionnaire of order digitizing from left to right or the option of form, set up local data base;

Second step: by the method for image procossing, demarcates the position of choice box before each option in questionnaire or form, then sets a certain size check box, whether selectedly carry out detection option;

3rd step: the picture after calibration is carried out adaptive thresholding binaryzation picture, then picture is processed by the expansion of morphological operation burn into, corrosion, and calculate blacking part in each check box and account for the scale of whole check box, if this ratio is greater than a certain threshold value, represent that this option is selected, it is digital accordingly that output represents this option.Due to the error of the interference and binary conversion treatment that there is illumination in experiment, threshold value gets 0.6;

4th step: the numeral exported in the 3rd step mated with local database, finds the option that numeral is corresponding, then the related content of output intent option and option in a mobile device, and then completes the identification of questionnaire and form.

Implementation result:

According to above-mentioned effect, the online questionnaire downloaded is identified, all experiments tangible mobile device Huawei AscendP7 carries out, the major parameter of this mobile device is: operating system: Android OS 4.4, cpu frequency 1.8GHz, post-positioned pick-up head 1,300 ten thousand pixel;

Under complex background, identify that the time of a questionnaire is within 5 seconds, accuracy is 100%;

The identification process of whole system and the results are shown in Figure 4;

Compared with prior art, invent the requirement of equipment simple, simultaneously convenient to operation, interference of different nature is processed, speed and discrimination all improve a lot.Whole system has stronger robustness, can split and questionnaire under identifying complex background and different light situation; Also have rapidly and efficiently property, the identification for a questionnaire and form generally only spends the time in a few second simultaneously, accurately almost reaches absolutely.

Above specific embodiments of the invention are described.It is to be appreciated that the present invention is not limited to above-mentioned particular implementation, those skilled in the art can make various distortion or amendment within the scope of the claims, and this does not affect flesh and blood of the present invention.

Claims (9)

1., based on questionnaire and the form Digital identification method of mobile device, it is characterized in that: comprise the steps:

Step one, the segmentation of questionnaire or form: according to the Corner Feature of papery questionnaire or form Image, uses VuforiaSDK training sample picture, in background complicated and changeable rapidly and efficiently be partitioned into questionnaire or form;

Step 2, the calibration of questionnaire or form: according to the coordinate transform in OpenGL system, is converted to screen coordinate by the object coordinates being partitioned into questionnaire or form, and calibrates questionnaire or the form of various position by affined transformation;

Step 3, the identification of questionnaire or form: image procossing is carried out to the questionnaire calibrated or form, in order by the option digitizing in table, whether the ratio accounting for whole region according to black region in choice box carrys out this option of interpretation selected, exports the number designation of selected option; The questionnaire set up according to number designation and this locality or the SQL storehouse of form option content, export selected option and content.

2. the questionnaire based on mobile device according to claim 1 and form Digital identification method, it is characterized in that: described step one, be specially: from complex background, be partitioned into questionnaire, obtain four apex coordinates of questionnaire or form, first corresponding storehouse is trained according to the Corner Feature of questionnaire or form, then starting Vuforia SDK in a mobile device uses the storehouse of training to identify questionnaire or form, OpenGL is adopted to play up the interface of mobile device in identifying, calculate four three-dimensional coordinates of summit under the coordinate system set up centered by questionnaire or form of questionnaire or form.

3. the questionnaire based on mobile device according to claim 1 and form Digital identification method, it is characterized in that: described step 2, be specially: the screen coordinate three-dimensional vertices coordinate of questionnaire or form being converted to mobile device by matrixing, namely the conversion of coordinate system in OpenGL is utilized, set up mould and look matrix and projection matrix, and by the viewport transform, the three-dimensional coordinate under object coordinates system is converted to screen coordinate, then according to affined transformation, the questionnaire of diverse location or form are calibrated.

4. the questionnaire based on mobile device according to claim 1 and form Digital identification method, is characterized in that: described step 2, comprises following operation steps:

The first step: the apex coordinate first obtaining questionnaire, namely at the coordinate (x of model coordinate systems _obj, y _obj, z _obj, w _obj), the coordinate (x of camera coordinate system _eye, y _eye, z _eye, w _eye) be multiplied by mould by model coordinate to obtain depending on matrix:

$\left(\begin{array}{c}{x}_{\mathrm{eye}}\\ y_{\mathrm{eye}}\\ z_{\mathrm{eye}}\\ w_{\mathrm{eye}}\end{array}\right)=M_{\mathrm{Modelview}}*\left(\begin{array}{c}{x}_{\mathrm{obj}}\\ y_{\mathrm{obj}}\\ z_{\mathrm{obj}}\\ w_{\text{obj}}\end{array}\right)---\left(1\right)$

Second step: the coordinate (x under camera coordinate system _eye, y _eye, z _eye, w _eye) coordinate (x on summit under cutting coordinate system is obtained by being multiplied by projection matrix _clip, y _clip, z _clip, w _clip):

$\left(\begin{array}{c}{x}_{\mathrm{clip}}\\ y_{\mathrm{clip}}\\ z_{\mathrm{clip}}\\ w_{\mathrm{clip}}\end{array}\right)=M_{\mathrm{projection}}*\left(\begin{array}{c}{x}_{\mathrm{eye}}\\ y_{\mathrm{eye}}\\ z_{\mathrm{eye}}\\ w_{\text{eye}}\end{array}\right)---\left(2\right)$

3rd step: by the coordinate under reduction coordinate system divided by w _clip, namely obtain normalized device coordinate (x _ndc, y _ndc, z _ndc), this coordinate just obtains screen coordinate through Pan and Zoom, by the viewport transform, this coordinate conversion is become screen coordinate (x _w, y _w, z _w):

$\left(\begin{array}{c}{x}_w\\ y_w\\ z_w\end{array}\right)=\left(\begin{array}{c}\frac{w}{2}{x}_{\mathrm{ndc}}+(x+\frac{w}{2})\\ \frac{h}{2}{y}_{\mathrm{ndc}}+(y+\frac{h}{2})\\ \frac{f-n}{2}{z}_{\mathrm{ndc}}+\left(\frac{f+n}{2}\right)\end{array}\right)---\left(3\right)$

Wherein (x, y, w, h) is starting point and the size of screen window, (x, y) is the coordinate of screen window starting point, and w is the width of screen, and h is the height of screen, (n, f) is the depth range of screen, n →-1 under linear case, f → 1;

4th step: by affined transformation on mobile device screen by the questionnaire of diverse location or form calibration, the matrix of the affined transformation wherein used is:

Translation transformation: point (x, y) on screen is moved to point (x+dx, y+dy), dx is the distance of horizontal ordinate translation, and dy is the distance of ordinate translation, and transformation matrix is:

$M_{\mathrm{translate}}=\left(\begin{array}{ccc}1& 0& \mathrm{dx}\\ 0& 1& \mathrm{dy}\\ 0& 0& 1\end{array}\right)---\left(4\right)$

Rotational transform: point (x, y) on screen is rotated counterclockwise θ around far point, and transformation matrix is:

$M_{\mathrm{rotate}}=\left(\begin{array}{ccc}\cos \mathrm{\θ}& -\sin \mathrm{\θ}& 0\\ \sin \mathrm{\θ}& \cos \mathrm{\θ}& 0\\ 0& 0& 1\end{array}\right)---\left(5\right)$

Scale transformation: the horizontal ordinate of point (x, y) on screen is zoomed in or out original sx doubly, ordinate zooms in or out original sy doubly, and transformation matrix is:

$M_{\mathrm{scale}}=\left(\begin{array}{ccc}\mathrm{sx}& 0& 0\\ 0& \mathrm{sy}& 0\\ 0& 0& 1\end{array}\right)---\left(6\right).$

5. the questionnaire based on mobile device according to any one of claim 1-4 and form Digital identification method, it is characterized in that: described step 3, be specially: first by the content data of whole questionnaire and form, namely the database of questionnaire content is set up according to order from top to bottom and from left to right, the option of questionnaire is represented with numeral, then the position of the choice box of each option is calibrated, save the data in local file, finally by the calibrated picture of adaptive thresholding, pass through threshold process, corrosion, expand, etching operation is by picture binaryzation, then calculate blacking part in each option area and account for the ratio in whole region, if ratio is greater than a certain threshold value, judge that this option is selected, export the numeral representing this option, then option and the content thereof of this digitized representation is matched in a database, and Output rusults.

6. for realizing the questionnaire based on mobile device and the form digitizing recognition system of method described in any one of the claims 1-5, it is characterized in that: comprising: the partitioning portion of questionnaire or form, calibrated section and identification division, wherein:

The partitioning portion of described questionnaire or form: according to the Corner Feature of papery questionnaire or form Image, uses VuforiaSDK training sample picture, in background complicated and changeable rapidly and efficiently be partitioned into questionnaire or form;

The calibrated section of described questionnaire or form: according to the coordinate transform in OpenGL system, is converted to screen coordinate by the object coordinates being partitioned into questionnaire or form, and calibrates questionnaire or the form of various position by affined transformation;

The identification division of described questionnaire or form: image procossing is carried out to the questionnaire calibrated or form, in order by the option digitizing in table, whether the ratio accounting for whole region according to black region in choice box carrys out this option of interpretation selected, exports the number designation of selected option; The questionnaire set up according to number designation and this locality or the SQL storehouse of form option content, export selected option and content.

7. the questionnaire based on mobile device according to claim 6 and form digitizing recognition system, it is characterized in that: the partitioning portion of described questionnaire or form: be partitioned into questionnaire from complex background, obtain four apex coordinates of questionnaire or form, first corresponding storehouse is trained according to the Corner Feature of questionnaire or form, then starting Vuforia SDK in a mobile device uses the storehouse of training to identify questionnaire or form, OpenGL is adopted to play up the interface of mobile device in identifying, calculate four three-dimensional coordinates of summit under the coordinate system set up centered by questionnaire or form identifying questionnaire or form.

8. the questionnaire based on mobile device according to claim 6 and form digitizing recognition system, it is characterized in that: the calibrated section of described questionnaire or form: the screen coordinate three-dimensional vertices coordinate of questionnaire or form being converted to mobile device by matrixing, namely the conversion of coordinate system in OpenGL is utilized, formwork erection looks matrix and projection matrix, and by the viewport transform, the three-dimensional coordinate under object coordinates system is converted to screen coordinate, then according to affined transformation, the questionnaire of diverse location or form are calibrated.

9. the questionnaire based on mobile device according to claim 6 and form digitizing recognition system, it is characterized in that: the identification division of described questionnaire or form: first by the content data of whole questionnaire and form, namely the database of questionnaire content is set up according to order from top to bottom and from left to right, the option of questionnaire is represented with numeral, then the position of the choice box of each option is calibrated, save the data in local file, finally by the calibrated picture of adaptive thresholding, pass through threshold process, corrosion, expand, the morphological operations such as corrosion are by picture binaryzation, then calculate blacking part in each option area and account for the ratio in whole region, if ratio is greater than a certain threshold value, judge that this option is selected, export the numeral representing this option, then option and the content thereof of this digitized representation is matched in a database, and Output rusults.