WO2012068902A1

WO2012068902A1 - Method and system for enhancing text image clarity

Info

Publication number: WO2012068902A1
Application number: PCT/CN2011/077904
Authority: WO
Inventors: 黄灿; 龙腾; 镇立新
Original assignee: 上海合合信息科技发展有限公司
Priority date: 2010-11-25
Filing date: 2011-08-02
Publication date: 2012-05-31
Also published as: CN102013094A; CN102013094B

Abstract

Disclosed is a method and system for enhancing text image clarity. The method comprises: first taking an image of the document, and then taking images of each local area of the document at a close range; extracting the characteristic points of these clear local area images and the original document image and then matching to obtain the corresponding matched characteristic points between the local area images and the original document image; calculating the perspective transformation matrix from the local areas images to the original document image according to the characteristic point pairs, then transforming the clear local area images in accordance with the perspective transformation matrix, and replacing the sections of the original document image with the transformed local areas images. The present invention employs techniques in the fields of image processing, computer vision and the like, utilizing multiple clear local areas images to replace sections of the original document image, enhancing image clarity through this replacement method and making the text more legible. The present invention solves the problem of unclear text image when a user takes an image of a large document using a camera.

Description

Method and system for improving text image clarity

The invention belongs to the technical field of image processing, and relates to a method for improving the sharpness of an image, in particular to a method for improving the sharpness of a text image. Meanwhile, the present invention also relates to a system for improving the sharpness of a text image. Background technique

As the performance of smart cameras has increased, the digital cameras that come with them have become the standard configuration for smartphones. People often use a camera on their mobile phone to scan or take text images. The current scanner function on the smartphone is to first take the text picture with the camera, and then add some image pre-processing to get the final scan result. One of the obvious disadvantages of this kind of mobile phone scanner is that when the text (document) is relatively large, because the camera is relatively far away, the image obtained in this image has lower resolution and louder noise, resulting in a text picture. A lot of the text is not very clear.

The main reasons for the font blur are:

(1) The camera of a mobile phone has a limited number of pixels. The average pixel of a mobile phone camera is between 3 million and 5 million. Therefore, for a large document, you need to take all the details of the document. , is unlikely.

(2) Since the complete document is to be taken, the camera must be far away when the document is large, so the focus of the lens on the flat document cannot be very accurate when it is far away, which will cause the text image to be blurred. .

In the method of improving the sharpness and resolution of an image, the patent "United States Patent 7106914: Bayesian image super resolution" and the patent "United States Patent 7613363: Image superresolution through edge extraction and contrast enhancement" introduce an increase in image resolution to make the image more A clear method. Chinese patent CN200910153544.0 also discloses a video super-resolution method suitable for the compressed domain, which fully utilizes the information of multiple frames before and after to reconstruct the target frame by super-resolution, mainly including the following steps: First, the decompression is low. Distinguish the video and get various information; then, using the obtained information, use the Bayesian framework to get each of the current windows Single super-resolution image; Finally, the final super-resolution image of the target frame is reconstructed using each single super-resolution image within the current window.

All of the above schemes improve the sharpness of text images by taking multiple images of the same resolution and then processing them by a certain algorithm. A major disadvantage of such methods is that they take a long time and improve the sharpness of the text image. Not very obvious, not suitable for mobile platforms, nor suitable for processing text images. Summary of the invention

The technical problem to be solved by the present invention is to provide a method for improving the sharpness of a text image, which can improve the sharpness of the entire document image.

Further, the present invention further provides a system for improving the sharpness of a text image, which can improve the sharpness of the entire document image.

In order to solve the above technical problems, the present invention uses the following technical solutions:

A method for improving the sharpness of a text image, first taking a document image, then taking a close-up shot of each partial region of the document, and then extracting these clear local region images and feature points of the original document image, and then matching to obtain a partial image Matching feature points with the original document image, calculating a perspective transformation matrix of the partial image to the original document image according to the feature point pair, and then transforming the clear partial image according to the perspective change matrix, and replacing the transformed partial image with the original document The area in which the image is located, using this alternative to ultimately improve the clarity of the entire document image.

A method of improving the sharpness of a text image, the method comprising the steps of:

51. Take the entire text image;

52. Shooting various partial areas of the text;

53. Extracting a local area image and a feature point of the original entire image, and performing matching to obtain a corresponding matching feature point of the partial image and the original text image;

54. Calculate a perspective transformation matrix of the partial image to the original text image according to the feature point pair;

55, transforming the clear partial image according to the perspective change matrix;

S6. Substituting the transformed partial image for the corresponding region in the entire text image.

As a preferred solution of the present invention, in the step S1, a method for capturing an entire text image To: Adjust the distance of the camera from the text. When the text to be shot just fills the entire screen of the phone, press the capture button to get the initial text image.

As a preferred solution of the present invention, in step S2, the distance of the camera is adjusted to make the camera closer to the text; when the local area of the text to be captured occupies the setting range of the entire text area, the shooting button is pressed; At this time, since the camera is closer to the text, the text in the obtained partial image will be more clear.

As a preferred solution of the present invention, in the step S3, the method for performing feature matching between the partial image and the entire text image includes:

S31, determining a feature key point of interest; S32, extracting a feature vector descriptor of a region around the key point; S33, matching each feature vector descriptor by the Euclidean distance of the feature point;

In step S33, the matching strategy adopts the nearest neighbor proportional matching: for the feature point matching of the two images, to find the corresponding matching point with a certain feature point in the first image, find the feature in the second image. The two feature points closest to the Euclidean distance, if the distance of the nearest point a ' is divided by the distance of the second near point °^. ^ If the value is less than the set value, the nearest point is considered to be the matching point, otherwise it will not be received.

As a preferred solution of the present invention, in the step S4, the method for calculating the perspective transformation matrix according to the matched feature point pairs is:

Calculating a perspective change matrix between planes of two text images according to feature point pairs on the matching of the two images;

Set src_points to the coordinate of the matching point of the plane in the whole text image, the size is 2xN, where N is the number of points; set dst_points is the matching point coordinate of the plane of the local image, the size is 2x;

The perspective change matrix is a 3 x 3 matrix, making

Where , . , 1) is the dst_points—the coordinates of a point, (x , 1) is the src__point—the 3x3 perspective change matrix of the output of the point, so that the sum of the back projection errors is the smallest, that is, the following formula is the smallest:

Change 26

As a preferred solution of the present invention, in the step S5, the method for transforming the partial image by the perspective transformation matrix is:

After obtaining the perspective change matrix, each pixel of the partial image is transformed according to the perspective change matrix to obtain a transformed partial image, and the changed partial image is in the same coordinate system as the entire text image.

As a preferred solution of the present invention, the step S6 includes: calculating an effective area, and pasting the transformed partial image according to the effective area;

The effective area is calculated as: Four vertices of the partial image before the change, the upper left point, the upper right point, the lower left point, and the lower right point. The four points are transformed by the perspective change matrix to obtain the transformed position coordinates, and then the effective inscribed rectangles of the four transformed vertices are calculated, and the inscribed rectangle represents the effective area to be pasted;

The method of pasting a partial image according to the effective area is as follows: By using the calculated pasted area, the pixel of the original text image is directly replaced with the partial image pixel in the area to be pasted. A method of improving the sharpness of a text image, the method comprising the steps of:

Step 110: Obtain a full picture of the text;

Step 120: The camera is moved closer to a local area of the text to obtain a clear local image to be pasted;

Step 130: Perform feature matching on the partial image and the text full image;

Step 140: judging whether the feature matching is successful; judging criterion: whether the feature point pair on the matching reaches the set value, and if the perspective change matrix cannot be calculated if the value is lower than the set value, the judgment is a failure, and the process proceeds to the step.

170, if the number of points of the feature matching pair reaches or exceeds the set value, determining that the matching is successful, go to step 150; Step 150, calculate the perspective change matrix between the two images by using the feature points on the matching obtained in step 130, and Transforming the partial image according to a perspective change matrix;

Step 160, replacing the transformed partial image with a corresponding area of the original text full image;

Step 170, judge: whether there are other partial areas that need to be photographed; if still, go to the step

120, shooting the next area of text, if there is no local area to be photographed, go to step 180;

Replacement page (Article 26) Step 180, the end. A system for improving the sharpness of a text image, the system comprising:

a camera unit for taking an entire text image and for capturing various local areas of the text;

The feature point matching unit is configured to extract the local area image and the feature points of the original whole image, perform matching, and obtain corresponding matching feature points of the partial image and the original text image;

a perspective transformation matrix calculation unit, configured to calculate a perspective transformation matrix of the partial image to the original text image according to the feature point pair;

a partial image transforming unit for transforming a clear partial image according to a perspective change matrix; and an integrating unit for replacing the transformed partial image with a corresponding region in the entire text image. As a preferred solution of the present invention, the method for performing feature matching between a partial image and a whole text image by the feature point matching unit includes:

Step 131: Determine a feature key point of interest; Step 132: Extract a feature vector descriptor of a region around the key point; Step 133, match each feature vector descriptor by a Euclidean distance of the feature point;

The matching strategy adopts the nearest neighbor proportional matching: For the feature point matching of the two images, to find the corresponding matching point with a certain feature point in the first image, find the closest to the feature point in the second image. Two feature points, if the distance d of the nearest point divided by the distance d _s of the second near point is less than the set threshold, the nearest point is considered to be a matching point, otherwise it is not received;

The method for calculating the perspective transformation matrix by the perspective transformation matrix calculation unit according to the matched feature point pairs is: calculating a perspective change matrix between planes of two text images according to the feature point pairs on the matching of the two images; setting src_points The coordinate of the matching point of the plane in the whole text image, the size is 2xN, where N is the number of points; the dstjpoints is the matching point coordinate of the plane where the partial image is located, the size is 2xN; the perspective change matrix is 3 χ 3 Matrix, making

Replacement page (Article 26) Where ( , , ¹ ) is the coordinates of a point in dst_points, (^, , 1) is the coordinates of a point in src_point; the 3x3 perspective change matrix of the output is such that the sum of back projection errors is the smallest, that is, the following formula is the smallest:

y + yi + ^ _{3 )} 2 , ^x i + yi + y ) .

The partial image transform unit transforms the partial image by the perspective transformation matrix: after obtaining the perspective change matrix, each pixel of the partial image is transformed according to the perspective change matrix to obtain the transformed partial image, and the changed The partial image will be in the same coordinate system as the entire text image;

The integration unit includes: an effective area calculation unit, and an attachment unit for pasting the transformed partial image according to the effective area;

The calculation method of the effective area calculation unit is: changing four vertices of the partial image before, changing the upper left point, the upper right point, the lower left point, and the lower right point; the four points are transformed by the perspective change matrix to obtain the transformed position coordinates. Then calculating a valid inscribed rectangle of the four transformed vertices, the inscribed rectangle representing the effective area to be pasted;

The method for the pasting unit to paste the partial image according to the effective area is: replacing the pixels of the original text image with the partial image pixels by using the calculated pasting area and the area to be pasted. In order to implement the present invention, it is generally required to have the following hardware conditions: a smartphone or a digital camera, which requires a general arithmetic and storage device, including a CPU of a certain frequency (central processing unit), a memory for use in computing, and a use. To store system software, application software and storage space for various data. A smart phone or digital camera should have an auto focus function. The invention has the following advantages: the method and the system for improving the sharpness of the text image proposed by the invention adopt the techniques of image processing, computer vision and the like, and use multiple clear partial document images to replace the original document area, through this Alternative methods improve the clarity of the image and make the text easier to distinguish. The invention solves the problem that the user photographs when shooting a large document using the camera

Replacement page (Article 26) The text picture is ambiguous. DRAWINGS

1 is a flow chart of a method for improving the sharpness of a text image according to the present invention.

Figure 2 is a schematic diagram of acquiring an entire text image.

FIG. 3 is a schematic diagram of acquiring a partial text image.

Figure 4 is a schematic diagram of the acquired partial text image.

FIG. 5 is a schematic diagram of feature matching between a partial image and an original image of the document. detailed description

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Embodiment 1

The present invention discloses a method for improving the sharpness of a text image by first taking a document image, then photographing each partial region of the document at a close distance, and then extracting these clear local region images and feature points of the original document image. Then, matching is performed to obtain corresponding matching feature points of the partial image and the original document image, and according to the feature point pair, the perspective transformation matrix of the partial image to the original document image is calculated, and then the clear partial image is transformed according to the perspective change matrix, and the transformation is performed. The subsequent partial image replaces the area where the original document image is located, and this alternative method finally improves the sharpness of the entire document image.

Referring to FIG. 1, in the embodiment, the specific steps of the method for improving the sharpness of the text image are as follows: [Step 110] Obtain a full text of the text.

The way to get the initial text image is:

Adjust the distance of the camera from the document. When the document to be shot just fills the entire screen of the phone, press the capture button to get the initial text image. An example of initial text image acquisition is shown in Figure 1.

[Step 120] The camera is moved closer to the local area of the text to obtain a clear partial image to be pasted.

The way to get a partial image is:

Adjust the distance of the camera so that the camera is closer to the document, when the local area of the document to be captured is occupied When the document area is 1/6 to 1/3 (the specific size is determined by the user), press the shooting button. At this time, the text in the partial image will be more clear because the camera is closer to the document. Examples of partial image capture are shown in Figures 2 and 3.

[Step 130] The partial image is matched with the full text of the text.

The method for feature matching between a partial image and an initial text image is:

In the prior art, there are many methods for extracting feature points in an image and then matching according to the descriptors of the feature points, wherein SIFT (scale invariant Features) is a good scale-invariant local feature, which is for translation, Rotation, scale, and brightness variations are invariant, while maintaining a certain degree of robustness over a range of noise, affine transformations, and illumination variations. (Lowe, D. Distinctive image features from scale-invariant keypoints, IJCV, volume 60, pages 91 110, 2004). SIFT-based feature matching involves three steps: First, determine the feature detection of interest. Second, extract the feature vector descriptor of the area around the key point. Third, the feature vectors are described by feature vectors. The method of measurement generally uses Euclidean distance.

Matching strategy uses nearest neighbor proportional matching: For example, for feature point matching of two images, to find the corresponding matching point with a feature point in the first image, find the feature point in the second image. From the nearest two feature points, if the distance of the nearest point _£ ^ divided by the distance of the second near point ∞ is less than the set threshold, the nearest point is considered to be the matching point, otherwise it is not received. The accuracy of this matching method is relatively high. Because it is a matching point, the first neighboring point represents the correct matching point, and the second neighboring point is the incorrect matching point. In general, the distance of the incorrect point is greater than the distance of the correct point. This can be launched. The ratio of ^ is relatively small. If it is not a matching point, since the first near and second near feature vectors are all mismatched, the distance difference between the two is relatively small, so the ratio will be compared.

Nearly 1. With the nearest neighbor matching, set a reasonable proportional value, generally set to 0.7, you can find the matching point very well. An example of feature matching between images is shown in Figure 4.

[Step 140] It is judged whether the feature matching is successful. Judging criteria: Whether the feature point pairs on the matching reach more than four, such as less than four, the perspective change matrix cannot be calculated, then it is judged as failure, go to step 170, if the number of points of the feature matching pair exceeds four, it is judged as successful. , Go to step 150.

[Step 150] Calculate the feature points on the matching obtained by step 130, and calculate between the two images. The perspective matrix is transformed and the partial image is transformed according to the perspective change matrix.

The method for calculating the perspective transformation matrix from the matched feature point pairs is:

According to the feature point pairs on the matching of the two images, the through-change matrix (homgraphy matrix) between the planes of the two text images is calculated.

It is assumed here that srcjDoints is the matching point coordinate of the plane in the initial text image, and the size is 2xN, where N is the number of points. Suppose dst_points is the matching point coordinate of the plane where the partial image is located, and the size is 2xN.

Homography is a 3 x 3 matrix, making

Where O, 1) is dst_points—the coordinates of a point, which is src_point—the 3x3 homography matrix of the point output, which minimizes the sum of back projection errors, ie, the minimum:

y (( _χ ' , ^x i + ^ > I ₍ .' + ³ ) ² ) The method of transforming a partial image through a perspective transformation matrix is:

After obtaining the perspective change matrix (homography matrix), each pixel of the partial image is transformed according to the homography matrix to obtain a transformed partial image, so that the partial image after the sub-variation is in the same coordinate system as the initial text image.

[Step 160] replacing the transformed partial image with the corresponding region of the original document full image; comprising: calculating the effective region, and pasting the transformed partial image according to the effective region.

The method of calculating the effective area is:

The four vertices of the partial image before the change, the upper left point, the upper right point, the lower left point, and the lower right point. The four points are transformed by the perspective change matrix to obtain the transformed position coordinates, and then the effective inscribed rectangles of the four transformed vertices are calculated. The inscribed rectangle represents the effective area to be pasted.

The method of pasting the transformed partial image according to the effective area is as follows:

By using the pasted area calculated above, the area to be pasted, directly using the partial image

Replacement page (Article 26) The pixels replace the pixels of the original text image.

[Step 170] Judging: Whether there are other partial areas that need to be photographed. If so, go to step 120 and take the next area of the text. If there is no local area to be shot, go to step 180.

[Step 180] End. In summary, the method for improving the sharpness of a text image proposed by the present invention uses a technique of image processing, computer vision, and the like to replace a region of an original document with a plurality of clear partial document images. Improves the sharpness of the image and makes the text easier to distinguish. The present invention solves the problem that a user takes a picture that is blurred when shooting a large document using the camera. Embodiment 2

The embodiment discloses a system for improving the sharpness of a text image, the system comprising: an imaging unit, a feature point matching unit, a perspective transformation matrix calculation unit, a partial image transformation unit, and an integration unit.

The camera unit is used to capture the entire text image while simultaneously capturing various local areas of the text. The feature point matching unit is configured to extract the local area image and the feature points of the original whole image, and perform matching to obtain corresponding matching feature points of the partial image and the original text image.

The perspective transformation matrix calculation unit is configured to calculate a perspective transformation matrix of the partial image to the original text image according to the feature point pair.

The partial image transform unit is configured to transform the clear partial image according to the perspective change matrix. The integration unit is used to replace the transformed partial image with the corresponding region in the entire text image.

The method for the feature matching unit to perform feature matching between the partial image and the whole text image includes: Step 131: determining a feature key point of interest; Step 132: Extracting a feature vector descriptor of a region around the key point; Step 133, The Euclidean distance of the feature points matches each feature vector descriptor.

Matching strategy using nearest neighbor proportional match: For feature point matching of two images, to find and Corresponding matching points of a feature point in an image, and finding two feature points closest to the Euclidean distance of the feature point in the second image, if the distance d of the closest point is divided by the distance d of the second near point _{If s∞ond is} less than the set threshold, the nearest point is considered to be a matching point, otherwise it is not received.

The perspective transformation matrix calculation unit calculates the perspective transformation matrix according to the matched feature point pairs as follows: According to the feature point pairs on the matching of the two images, the perspective change matrix between the planes of the two text images is calculated.

Set src_points to the coordinates of the matching points of the plane in the initial text image, the size is 2xN, where N is the number of points; set dst_points to the matching point coordinates of the plane where the partial image is located, the size is 2xN; the perspective change matrix is 3 x 3 matrix, making

Where (χ,., y _t , 1) is the homogeneous coordinate corresponding to the dst_points point, and (x _t ' , , 1) is the homogeneous coordinate corresponding to the src_points point.

In the stage of calculating the matching points, it is obtained that src_points and dst_points are Cartesian coordinates, and for N points, the size is 2 χ Ν. When calculating the perspective change matrix Η, the homogeneous coordinates are used. Homogeneous coordinates use Ν + 1 component to describe the Cartesian coordinates of the Ν dimension. For example, the 2D homogeneous coordinate is a new component 1 added to the Cartesian coordinates (x, y), which becomes (x, y, l). For example: The point (1, 2) in Cartesian coordinates is (1, 2, 1) in homogeneous coordinates.

The output 3x3 perspective change matrix minimizes the sum of back projection errors, ie the following minimum: Υ ((χ' ^ ^x ' ^+h >^' ⁺ ^y _h( 3⁄4 ₁ +/? ₂₂ + _{3⁄4 3) 2)} The partial image transform unit transforms the partial image by the perspective transformation matrix: after obtaining the perspective change matrix, each pixel of the partial image is transformed according to the perspective change matrix to obtain the transformed partial image, and the changed The partial image will be in the same coordinate system as the entire text image.

The integration unit includes: an effective area calculation unit, and an attachment unit for pasting the transformed partial image according to the effective area.

The calculation method of the effective area calculation unit is: changing four vertices of the partial image before, left

Replacement page (Article 26) Upper point, upper right point, lower left point, lower right point; the four points are transformed by the perspective change matrix to obtain the transformed position coordinates, and then the effective inscribed rectangles of the four transformed vertices are calculated, and the inscribed rectangle represents The valid area to paste.

The method for the pasting unit to paste the partial image according to the effective area is: replacing the pixels of the original text image with the partial image pixels by using the calculated pasting area and the area to be pasted. The description and application of the present invention are intended to be illustrative, and not intended to limit the scope of the invention. Variations and modifications of the embodiments disclosed herein are possible, and various alternative and equivalent components of the embodiments are well known to those of ordinary skill in the art. It will be apparent to those skilled in the art that the present invention may be embodied in other forms, structures, arrangements, ratios, and other components, materials and components without departing from the spirit or essential characteristics of the invention. Other variations and modifications of the embodiments disclosed herein may be made without departing from the scope and spirit of the invention.

Claims

The invention provides a method for improving the sharpness of a text image, wherein the method comprises the following steps: Step 110: Obtain a full text of the text; the method is: adjusting a distance of the camera from the text, when the text to be photographed is just full The entire phone screen, at this point press the capture button to get the initial text image;

Step 120: Adjust a distance between the camera and the file, and capture a partial area of the text to obtain a clear partial image to be pasted;

Step 130: Perform feature matching on the partial image and the text full image; the method of feature matching includes: Step 131: determining a feature key point of interest; Step 132, extracting a feature vector descriptor of a region around the key point; Step 133, passing the feature The Euclidean distance of the point is used to match each feature vector descriptor; in step 133, the matching strategy adopts nearest neighbor proportional matching: For feature point matching of two images, to find a corresponding matching point with a feature point in the first image, Then, in the second image, find two feature points closest to the Euclidean distance of the feature point. If the distance of the nearest point is ^ ₍ the distance _e divided by the second near point is less than the set threshold, the nearest point is considered to be Match points, otherwise they will not receive;

Step 140: Determine whether the feature matching is successful. Judging criterion: whether the feature point pair on the matching reaches the set value. If the perspective change matrix cannot be calculated if the value is lower than the set value, the determination is a failure, and the process proceeds to step 170, such as feature matching. If the number of points reaches or exceeds the set value, it is judged that the matching is successful, and the process proceeds to step 150;

Step 150: Calculate the perspective change matrix between the two images by using the feature points on the matching obtained in step 130, and transform the partial image according to the perspective change matrix; wherein, the method for calculating the perspective transformation matrix according to the matched feature point pairs For: the feature point pairs on the matching of the two images, calculate the perspective change matrix between the planes of the two text images; set src_points as the matching point coordinates of the plane in the initial text image, the size is 2xN, where, N Indicates the number of points; set dst_points to the coordinates of the matching points of the plane where the partial image is located, the size is 2xN; the perspective change matrix is a matrix of 3 x 3, making _Si

13 Replacement page (Article 26) Where Ο,, ,Ι) is the coordinate of a point in dst_points, (x;, ., 1) is the coordinate of a point in src_point; the output matrix of 3x3 is the smallest, so that the sum of back projection errors is the smallest, that is, the following formula is the smallest :

((Χ' ^ + ^{+ hl3} f I (γ' + ⁺ ^ γ) ; where the transformation of the partial image by the perspective transformation matrix is: After obtaining the perspective change matrix, each pixel of the partial image is in accordance with the perspective The change matrix is transformed to obtain a transformed partial image, and the changed partial image will be in the same coordinate system as the initial text image;

Step 160: Substituting the transformed partial image for the corresponding region of the original text full image; Step 160 includes: calculating an effective region, and pasting the transformed partial image according to the effective region; the effective region is calculated as: Four vertices, upper left point, upper right point, lower left point, lower right point; the four points are transformed by the perspective change matrix to obtain the transformed position coordinates, and then the effective inscribed rectangles of the four transformed vertices are calculated. The inscribed rectangle represents the effective area to be pasted; the method of pasting the partial image according to the effective area is: by using the calculated pasting area, the area of the original text image is directly replaced by the partial image pixel in the area to be pasted;

Step 170, judge: whether there are other partial areas that need to be photographed; if still, go to step 120, take the next area of the text, if there is no local area to be photographed, go to step 180;

Step 180, the end. A method for improving the sharpness of a text image, the method comprising the steps of:

51. Take the entire text image;

52. Shooting various partial areas of the text;

53. Extracting a local area image and feature points of the original entire image, and performing matching to obtain corresponding matching feature points of the local image and the original text image;

Replacement page (Article 2 of the Rules) S6. Substituting the transformed partial image for the corresponding region in the entire text image. The method for improving the sharpness of a text image according to claim 2, wherein:

In the step S1, the method for capturing the entire text image comprises: adjusting the distance of the camera from the text, and when the text to be photographed just fills the entire mobile phone screen, pressing the shooting button to obtain the initial text image;

In the step S2, the distance of the camera is adjusted to make the camera closer to the text; when the local area of the text to be captured occupies the setting range of the entire text area, the shooting button is pressed; at this time, because the camera is closer to the text, The text in the obtained partial image will be more clear. The method for improving the sharpness of a text image according to claim 2, wherein:

In the step S3, the method for performing feature matching between the partial image and the whole text image includes: S31, determining a feature key point of interest; S32, extracting a feature vector descriptor of a region around the key point; S33, passing the feature point of the European style Distance to match each feature vector descriptor;

In step S33, the matching strategy uses the nearest neighbor proportional matching: for the feature point matching of the two images, to find the corresponding matching point with a certain feature point in the first image, find the same in the second image. The feature point Euclidean distance is the closest two feature points. If the closest point distance d _nearst divided by the second near point distance _{ee is} less than the set threshold, the nearest point is considered to be a matching point, otherwise it is not received. The method for improving the sharpness of a text image according to claim 2, wherein:

In the step S4, the method for calculating the perspective transformation matrix according to the matched feature point pairs includes:

Calculating a perspective change matrix between the planes of the two text images according to the feature point pairs on the matching of the two images;

Set src_points to the coordinate of the matching point of the plane in the whole text image, the size is 2xN, where N is the number of points; set dst_points is the matching point coordinate of the plane of the local image, the size is 2xN; The perspective change matrix is a 3 x 3 matrix, making

Where ( , , 1) is the dst_points—the coordinates of a point, ( , , 1) is the src_point—the coordinates of a point;

The output 3x3 perspective change matrix minimizes the sum of back projection errors, ie the following formula:

The method for improving the sharpness of a text image according to claim 2, wherein:

In the step S5, the method for transforming the partial image by the perspective transformation matrix comprises: after obtaining the perspective change matrix, transforming each pixel of the partial image according to the perspective change matrix to obtain the transformed partial image, and the changed partial image. The partial image will be in the same coordinate system as the entire text image. The method for improving the sharpness of a text image according to claim 2, wherein:

The step S6 includes: calculating an effective area, and pasting the transformed partial image according to the effective area;

The effective area is calculated as: four vertices of the partial image before the change, the upper left point, the upper right point, the lower left point, and the lower right point; the four points are transformed by the perspective change matrix to obtain the transformed position coordinates, and then the four are calculated. a valid inscribed rectangle of the transformed vertex, this inscribed rectangle represents the valid area to be pasted;

The method of pasting a partial image according to the effective area is as follows: By using the calculated pasted area, the pixels of the original text image are directly replaced with the partial image pixels in the area to be pasted. A method for improving the sharpness of a text image, the method comprising the steps of:

16

Replacement page (Article 26) Step 110, obtaining a full picture of the text;

Step 120: Move the camera closer, and capture a partial area of the text to obtain a clear partial image to be pasted;

Step 150: Calculate a perspective change matrix between the two images by using the feature points on the matching obtained in step 130, and transform the partial image according to the perspective change matrix;

Step 180, the end. The method for improving the sharpness of a text image according to claim 8, wherein:

In the step 130, the method for performing feature matching between the partial image and the initial text image includes:

Step 131: Determine a feature key point of interest; Step 132: Extract a feature vector descriptor of a region around the key point; Step 133: Match each feature vector descriptor by the Euclidean distance of the feature point;

In step 133, the matching strategy uses the nearest neighbor proportional matching: For the feature point matching of the two images, to find the corresponding matching point with a certain feature point in the first image, find the same in the second image. The feature point Euclidean distance is the closest two feature points. If the closest point distance d _nearst divided by the second near point distance _{ee is} less than the set threshold, the nearest point is considered to be a matching point, otherwise it is not received. The method for improving the sharpness of a text image according to claim 8, wherein: in the step 150, the method for calculating a perspective transformation matrix according to the matched feature point pairs comprises:

Set src_points to the coordinate of the matching point of the plane in the initial text image, the size is 2xN, where N is the number of points; set dst_points is the matching point coordinate of the plane where the partial image is located, the size is 2xN;

The perspective change matrix is a 3 x 3 matrix, making

Where ( , , 1) is the coordinate of a point in dst_points, and 1) is the coordinates of a point in src_point;

The output of the 3x3 perspective change matrix minimizes the sum of back projection errors, ie the following formula:

The method for improving the sharpness of a text image according to claim 8, wherein:

In the step 150, the method for transforming the partial image by the perspective transformation matrix includes:

After obtaining the perspective change matrix, each pixel of the partial image is transformed according to the perspective change matrix to obtain a transformed partial image, and the changed partial image will be in the same coordinate system as the initial text image. The method for improving the sharpness of a text image according to claim 8, wherein:

The step 160 includes: calculating an effective area, and pasting the transformed partial image according to the effective area;

The effective area is calculated as: Four vertices of the partial image before the change, upper left point, right

18

Replacement page (Article 26) Upper point, lower left point, lower right point; the four points are transformed by the perspective change matrix to obtain the transformed position coordinates, and then the effective inscribed rectangles of the four transformed vertices are calculated, and the inscribed rectangle represents the paste to be pasted. Effective area

The method of pasting a partial image according to the effective area is as follows: By using the calculated pasted area, the pixels of the original text image are directly replaced with the partial image pixels in the area to be pasted. A system for improving the sharpness of a text image, wherein the system comprises:

a partial image transforming unit for converting a clear partial image according to a perspective change matrix;

An integration unit for replacing the transformed partial image with a corresponding region in the entire text image. The system for improving the sharpness of a text image according to claim 13, wherein: the feature point matching unit performs feature matching on the partial image and the entire text image, including:

The matching strategy uses the nearest neighbor proportional match: For the feature point matching of the two images, to find the corresponding matching point with a feature point in the first image, find the Euclidean distance from the feature point in the second image. The two most recent feature points, if the distance of the nearest point is divided by the second closest point Distance _e . "If rf is smaller than the set threshold, the nearest point is considered to be a matching point, otherwise it is not received; the perspective transformation matrix calculation unit calculates the perspective transformation matrix according to the matched feature point pairs as follows: According to the matching of the two images Feature point pair, calculate the perspective change matrix between the planes of the two text images; set src_points to the coordinates of the matching points of the plane in the whole text image, the size is 2xN, where N is the number of points; set dst_points to The coordinates of the matching points of the plane where the partial image is located, the size is 2xN; the perspective change matrix is a matrix of 3 x 3,

Where (W,, ¹ ) is the coordinates of a point in dst_points, which is the coordinates of a point in src_point; the 3x3 perspective change matrix of the output, so that the sum of back projection errors is the smallest, that is, the following formula is the smallest:

The integration unit includes: an effective area calculation unit, and a paste unit for pasting the transformed partial image according to the effective area;

The method for pasting the partial image according to the effective area by the pasting unit is: replacing the pixels of the original text image with the partial image pixels in the area to be pasted by calculating the pasted area.

20

Replacement page (Article 26)