CN107368829B - Method and apparatus for determining rectangular target area in input image - Google Patents
Method and apparatus for determining rectangular target area in input image Download PDFInfo
- Publication number
- CN107368829B CN107368829B CN201610307727.3A CN201610307727A CN107368829B CN 107368829 B CN107368829 B CN 107368829B CN 201610307727 A CN201610307727 A CN 201610307727A CN 107368829 B CN107368829 B CN 107368829B
- Authority
- CN
- China
- Prior art keywords
- line segment
- segments
- aspect ratio
- candidate
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Abstract
The invention discloses a method and equipment for determining a rectangular target area in an input image. The method comprises the following steps: acquiring a horizontal line segment and a vertical line segment in an input image; generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
Description
Technical Field
The present invention relates generally to the field of image processing. In particular, the present invention relates to a method and apparatus capable of determining a rectangular target region in an input image for perspective correction in a complex background.
Background
In modern society, there are a variety of documents, cards, documents, etc., such as identification cards, business cards, bank cards, house books, drivers' licenses, passports, household management documents of a party, etc. Some entities or individuals need to collect or archive such information frequently, requiring such documents, cards, documents, etc. to be retained in electronic form. The usual way of doing this electronically, with the exception of some special reading tools, is to take a picture and then store the image or to identify the image and then store the identified information.
In the process of taking and keeping, the problem of perspective transformation is often needed to be solved. This is because: due to environmental or equipment limitations, when these documents, cards, documents, etc. (rectangular objects) are photographed, they may not be photographed right against the surface of the photographic subject, but may have a certain angle with the normal direction of the surface of the photographic subject, resulting in the influence of the inclination, which is generally called perspective transformation. For the next step of identification and storage, perspective correction must be performed on the image, and then subsequent processing such as layout analysis and identification can be performed.
The traditional method is to analyze the shot image, find out the edge and corner points of the shot image, combine the width and height information of a rectangular target, establish a perspective transformation formula and carry out perspective correction.
However, the conventional method requires accurate edge and corner positions. Under the condition that the background of a rectangular target in an input image is complex, the positions of edges and corners are difficult to accurately position, and the traditional method is not suitable any more.
The present invention is directed to solving the above-mentioned problems, and can accurately determine a rectangular target region for perspective correction even when the background in an input image is complicated.
Disclosure of Invention
The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. It should be understood that this summary is not an exhaustive overview of the invention. It is not intended to determine the key or critical elements of the present invention, nor is it intended to limit the scope of the present invention. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.
The invention aims to provide a method and equipment for determining a rectangular target area in an input image.
In order to achieve the above object, according to one aspect of the present invention, there is provided a method of determining a rectangular target region in an input image, the method comprising: acquiring a horizontal line segment and a vertical line segment in an input image; generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
According to another aspect of the present invention, there is provided an apparatus for determining a rectangular target region in an input image, the apparatus comprising: a line segment obtaining device configured to: acquiring a horizontal line segment and a vertical line segment in an input image; a candidate line segment combination generating device configured to: generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; an aspect ratio calculation device configured to: calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; a selection device configured to: selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and a rectangular region determination device configured to: and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
According to still another aspect of the present invention, there is provided an information processing apparatus. The information processing apparatus includes: a controller configured to: acquiring a horizontal line segment and a vertical line segment in an input image; generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
In addition, according to another aspect of the present invention, there is also provided a storage medium. The storage medium includes a program code readable by a machine, which, when executed on an information processing apparatus, causes the information processing apparatus to execute the above-described method according to the present invention.
Further, according to still another aspect of the present invention, there is provided a program product. The program product comprises machine-executable instructions which, when executed on an information processing apparatus, cause the information processing apparatus to perform the above-described method according to the invention.
It should be noted that: the invention can solve the technical problem that the traditional method can not process the image with the complex background, and the invention can process the image with the simple background.
Drawings
The above and other objects, features and advantages of the present invention will be more readily understood by reference to the following description of the embodiments of the present invention taken in conjunction with the accompanying drawings. The components in the figures are meant to illustrate the principles of the present invention. In the drawings, the same or similar technical features or components will be denoted by the same or similar reference numerals. In the drawings:
fig. 1 shows a flow chart of a method of determining a rectangular target region in an input image according to an embodiment of the invention.
Fig. 2 shows an embodiment of step S1.
Fig. 3 shows an embodiment of step S3.
Fig. 4 shows an example of an input image.
Fig. 5 shows all edge line segments detected.
Fig. 6 shows a rectangular target area found according to the method of the invention.
Fig. 7 shows the result of perspective correction based on the present invention.
Fig. 8 shows a block diagram of the structure of an apparatus according to an embodiment of the invention.
Fig. 9 shows a block diagram of the structure of an information processing apparatus according to the present invention.
FIG. 10 shows a schematic block diagram of a computer that may be used to implement methods and apparatus according to embodiments of the invention.
Detailed Description
Exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in the specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the device structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted. In addition, it is also noted that elements and features depicted in one drawing or one embodiment of the invention may be combined with elements and features depicted in one or more other drawings or embodiments.
A flow of a method of determining a rectangular target region in an input image according to an embodiment of the present invention will be described below with reference to fig. 1.
Fig. 1 shows a flow chart of a method of determining a rectangular target region in an input image according to an embodiment of the invention. As shown in fig. 1, a method according to an embodiment of the invention comprises the steps of: obtaining a horizontal line-like segment and a vertical line-like segment in the input image (step S1); generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points (step S2); calculating the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination according to the four intersections corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the camera internal parameters of the input image (step S3); selecting one candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio (step S4); and determining an area surrounded by four line segments in the selected candidate line segment combination or extensions thereof as a rectangular target area (step S5).
In step S1, a horizontal line-like segment and a vertical line-like segment in the input image are obtained.
The input image is, for example, an image containing a rectangular object such as a certificate, a card, a document, or the like.
Fig. 4 shows an example of an input image including a rectangular target area (lower card) and a complicated background noise (upper N patches having two rectangular-like structures), and when perspective correction is performed by a conventional method, it is easy to recognize the background as the rectangular target area, and thus the perspective correction is erroneously performed.
First, for example, the input image may be pre-processed, including but not limited to noise removal, scaling size, image enhancement, and the like.
Then, as shown in fig. 2, in step S21, line segments in the input image are acquired.
Specifically, edges in the input image are detected; and detecting line segments from the edges.
Methods of detecting edges in images and methods of detecting straight-line segments from edges are well known to those skilled in the art, and edges may be detected based on CANNY operators or on gradients, for example. Detecting line segments from edges may utilize, for example, hough line detection algorithms and the like. The present invention is not limited thereto.
In step S22, the acquired line segments are classified into horizontal line-like segments and vertical line-like segments.
Firstly, all the acquired line segments are merged, and the line segments with close distance, consistent slope or close distance (for example, within 5 degrees of difference) are merged into one line segment. Wherein, the distance between the line segments with inconsistent slopes and no closeness is defined as the maximum value. The merging method is, for example, fitting a plurality of line segments, selecting a line segment at an intermediate position, representing by an average slope, clustering, etc. It should be noted that the merging step is an optional step.
Then, all the merged line segments are classified into horizontal line segments and vertical line segments according to the slope of the merged line segments.
The horizontal line-like segment is a substantially horizontal line segment, for example, a line segment having a difference in slope from the horizontal direction within a predetermined range. The vertical line-like segment is a substantially vertical line segment, for example, a line segment having a difference in slope from the vertical direction within a predetermined range.
Of course, the order of the step of merging segments and the step of classifying segments into horizontal and vertical line segments may be interchanged.
FIG. 5 shows all edge line segments detected, including four edges of the card, 7 edges of the N patches.
In step S2, candidate line segment combinations are generated, each of which includes two horizontal line segments and two vertical line segments, and the line segments or their extensions in each of the candidate line segment combinations intersect to obtain four intersections. It should be noted that the line segment candidate combinations herein are, for example, exhaustive.
Since the target region is rectangular, which includes two horizontal sides and two vertical sides perpendicular to each other, the candidate line segment combination should include two horizontal line-like segments and two vertical line-like segments.
The intersection of the line segments or their extensions in each generated combination of candidate line segments may result in four intersection points. It should be noted that such a combination of candidate line segments is discarded if at least one of the obtained four intersection points is not within the range of the input image.
In step S3, the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination is calculated based on the four intersections corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image.
Specifically, as shown in fig. 3, in step S31, a corresponding homography matrix for perspective correction is obtained according to the predetermined reference aspect ratio and the four intersections corresponding to each candidate line segment combination.
Suppose the resulting horizontal line segment is sh1、sh2And the like. VerticalStraight line segment is sv1,sv2And the like. Line segment combination candidates(s)h1、sh2、sv1,sv2) Corresponding to four intersections of(s)h1，sv1)，(sh1，sv2)，(sh2，sv1)，(sh2，sv2) The intersection of (2) is named p1、p2、p3、p4。
The present invention is based on the following assumptions: the aspect ratio of the target is known, referred to as a predetermined reference aspect ratio R.
Based on a predetermined reference aspect ratio R, a unit target rectangle may be constructed with four vertex coordinates pm1(0,0)、pm2(1,0)、pm3(0,R)、pm4(1,R)。
For each combination of candidate line segments, there are four intersections p1、p2、p3、p4. As known to those skilled in the art, the homography matrix H for perspective correction is:
wherein h is33＝1。
Accordingly, the perspective transformation formula is:
h21xix+h22xiy+h23＝(h31xix+h32xiy+h33)miy
h11xix+h12xiy+h13＝(h31xix+h32xiy+h33)mix
wherein x isix、xiyIs p1、p2、p3、p4Abscissa and ordinate of (a), mix、miyIs pm1(0,0)、pm2(1,0)、pm3(0,R)、pm4The abscissa and the ordinate of (1, R). Thus, a total of 8 equations are obtained, with R being known, and 8 unknowns, i.e., h11、h12、h13、h21、h22、h23、h31、h32. The homography matrix H can be solved. It should be noted that this homography matrix corresponds to a unit target rectangle in projective space (i.e., after perspective transformation) based on a predetermined reference aspect ratio R, and is not the homography matrix ultimately used for perspective correction. And, this homography matrix corresponds to one combination of candidate line segments. Thus, each candidate line segment combination corresponds to a homography matrix H.
Furthermore, when the aspect ratio r in projective space (i.e. after perspective transformation) is unknown, the inclusion h can be obtained by the derivation step described above11、h12、h13、h21、h22、h23、h31、h32A total of 8 equations of 8 unknowns can be used to solve the system of equations with 4 intersections P corresponding to each candidate segment combination and a unit target rectangle based on the unknown aspect ratio r. Thus, there are 9 unknowns in total.
In addition, based on the corresponding relationship between the world coordinates of the four points before perspective correction and the projective coordinates of the four points after perspective correction in the projective space (i.e., the perspective transformation formula), and the known characteristics of the external reference matrix of the camera, the constraint relationship between the perspective transformation matrix and the camera internal parameters can be known:
wherein, the camera intrinsic parameter matrix A is:
f is the focal length of the camera, w, h are the resolution of the camera, and a, b are the sensor size of the camera.
Thus, there are 9 equations and 9 unknowns.
One can solve:
where each candidate segment combination may solve for one r. The above formula is an example of a predetermined relationship.
Therefore, in step S32, the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination is calculated based on the predetermined relationship from the homography matrix corresponding to each candidate line segment combination and the in-camera parameters of the input image.
In step S4, one candidate line segment combination is selected based on a comparison of the aspect ratio R corresponding to each candidate line segment combination with a predetermined reference aspect ratio R.
Specifically, a candidate line segment combination whose aspect ratio is the smallest in difference from a predetermined reference aspect ratio is selected. Since there is only one rectangular target area in the input image, R corresponding to only one candidate line segment combination is correct, i.e. closest to the predetermined reference aspect ratio R.
In step S5, an area surrounded by four line segments or extensions thereof in the selected combination of candidate line segments is determined as a rectangular target area.
Fig. 6 shows a rectangular target area found according to the method of the invention, with four corners identified with dots.
In addition, perspective correction may be performed on the rectangular target region determined in step S5 according to the homography matrix corresponding to the finally selected aspect ratio r to obtain a corrected image.
Fig. 7 shows the result of perspective correction based on the present invention. The lower card has been corrected to the standard card shape and no longer has the effect of the perspective transformation, while the upper N patches are corrected to the deformity because the homography matrix on which the perspective correction of the invention is based is generated based on the correct target, i.e. the card. In the conventional method, since it is difficult to determine which of the card and the N patches is the target area, it is likely that the N patches are corrected well and the card is corrected to be malformed.
Next, an apparatus for determining a rectangular target area in an input image according to an embodiment of the present invention will be described with reference to fig. 8.
Fig. 8 shows a block diagram of the structure of an apparatus according to an embodiment of the invention. As shown in fig. 8, an apparatus 800 for determining a rectangular target area in an input image according to the present invention includes: a line segment obtaining device 81 configured to: acquiring a horizontal line segment and a vertical line segment in an input image; a candidate line segment combination generating device 82 configured to: generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; an aspect ratio calculation device 83 configured to: calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; a selection device 84 configured to: selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and a rectangular region determination device 85 configured to: and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
In one embodiment, the aspect ratio calculation means 83 comprises: a homography matrix obtaining unit configured to: obtaining a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a preset reference aspect ratio; an aspect ratio calculation unit configured to: and calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination based on a predetermined relation according to the homography matrix corresponding to each candidate line segment combination and the camera internal parameters of the input image.
In one embodiment, the predetermined relationship is constructed based on the correspondence between the world coordinates of the four points before perspective correction and the projective coordinates of the four points in the projective space after perspective correction, and the characteristics of the external parameter matrix of the camera.
In one embodiment, the line segment obtaining device 81 includes: an acquisition unit configured to: acquiring line segments in an input image; and a classification unit configured to: the acquired line segments are classified into horizontal line-like segments and vertical line-like segments.
In one embodiment, the line segment obtaining device 81 further includes: a merging unit configured to: and respectively combining at least part of the horizontal line-like segments and/or the vertical line-like segments according to a preset rule.
In one embodiment, the obtaining unit is further configured to: detecting an edge in an input image; and detecting line segments from the edges.
In one embodiment, the obtaining unit is further configured to: detecting edges in the input image based on the gradients; and detecting line segments from the edges by using Hough transform.
In one embodiment, the selection means 84 is further configured to: a candidate line segment combination having the smallest difference between the aspect ratio and the predetermined reference aspect ratio is selected.
Since the processes in the respective devices and units included in the apparatus 800 according to the present invention are similar to those in the respective steps included in the above-described method, respectively, a detailed description of these devices and units is omitted herein for the sake of brevity.
Further, according to still another aspect of the present invention, as shown in fig. 9, there is provided an information processing apparatus. The information processing apparatus 100 includes a controller 1001. The controller 1001 is configured to: acquiring a horizontal line segment and a vertical line segment in an input image; generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
Further, it should be noted that each constituent device and unit in the above-described apparatus may be configured by software, firmware, hardware, or a combination thereof. The specific means or manner in which the configuration can be used is well known to those skilled in the art and will not be described further herein. In the case of implementation by software or firmware, a program constituting the software is installed from a storage medium or a network to a computer (for example, a general-purpose computer 900 shown in fig. 10) having a dedicated hardware configuration, and the computer can execute various functions and the like when various programs are installed.
FIG. 10 shows a schematic block diagram of a computer that may be used to implement methods and apparatus according to embodiments of the invention.
In fig. 10, a Central Processing Unit (CPU)901 performs various processes in accordance with a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage section 908 to a Random Access Memory (RAM) 903. In the RAM 903, data necessary when the CPU 901 executes various processes and the like is also stored as necessary. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output interface 905 is also connected to bus 904.
The following components are connected to the input/output interface 905: an input section 906 (including a keyboard, a mouse, and the like), an output section 907 (including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like), a storage section 908 (including a hard disk, and the like), a communication section 909 (including a network interface card such as a LAN card, a modem, and the like). The communication section 909 performs communication processing via a network such as the internet. The driver 910 may also be connected to the input/output interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like can be mounted on the drive 910 as needed, so that a computer program read out therefrom is installed in the storage section 908 as needed.
In the case where the series of processes described above is realized by software, a program constituting the software is installed from a network such as the internet or a storage medium such as the removable medium 911.
It will be understood by those skilled in the art that such a storage medium is not limited to the removable medium 911 shown in fig. 10 in which the program is stored, distributed separately from the apparatus to provide the program to the user. Examples of the removable medium 911 include a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including a compact disc-read only memory (CD-ROM) and a Digital Versatile Disc (DVD)), a magneto-optical disk (including a mini-disk (MD) (registered trademark)), and a semiconductor memory. Alternatively, the storage medium may be the ROM 902, a hard disk included in the storage section 908, or the like, in which programs are stored, and which is distributed to users together with the device including them.
The invention also provides a program product with machine readable instruction codes stored. The instruction codes are read and executed by a machine, and can execute the method according to the embodiment of the invention.
Accordingly, a storage medium carrying the above-described program product having machine-readable instruction code stored thereon is also included in the present disclosure. Including, but not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.
In the foregoing description of specific embodiments of the invention, features described and/or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, in combination with or instead of the features of the other embodiments.
It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
In addition, the method of the present invention is not limited to be performed in the time sequence described in the specification, and may be performed in other time sequences, in parallel, or independently. Therefore, the order of execution of the methods described in this specification does not limit the technical scope of the present invention.
While the present invention has been disclosed above by the description of specific embodiments thereof, it should be understood that all of the embodiments and examples described above are illustrative and not restrictive. Various modifications, improvements and equivalents of the invention may be devised by those skilled in the art within the spirit and scope of the appended claims. Such modifications, improvements and equivalents are also intended to be included within the scope of the present invention.
Supplementary note
1. A method of determining a rectangular target region in an input image, comprising:
acquiring a horizontal line segment and a vertical line segment in an input image;
generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points;
calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image;
selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and
and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
2. The method according to supplementary note 1, wherein calculating the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination based on the four intersections corresponding to each candidate line segment combination, a predetermined reference aspect ratio, and the in-camera parameters of the input image includes:
obtaining a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a preset reference aspect ratio;
and calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination based on a predetermined relation according to the homography matrix corresponding to each candidate line segment combination and the camera internal parameters of the input image.
3. The method according to supplementary note 2, wherein the predetermined relationship is constructed based on a correspondence relationship between world coordinates of four points before perspective correction and projective coordinates of four points in the projective space after perspective correction, and characteristics of an external parameter matrix of the camera.
4. The method of supplementary note 1, wherein obtaining the horizontal line-like segments and the vertical line-like segments in the input image comprises:
acquiring line segments in an input image; and
the acquired line segments are classified into horizontal line-like segments and vertical line-like segments.
5. The method of supplementary note 4, wherein obtaining the horizontal line-like segments and the vertical line-like segments in the input image further comprises:
and respectively combining at least part of the horizontal line-like segments and/or the vertical line-like segments according to a preset rule.
6. The method according to supplementary note 4, wherein the acquiring of the line segment in the input image includes:
detecting an edge in an input image; and
line segments are detected from the edges.
7. The method according to supplementary note 6, wherein,
detecting edges in the input image based on the gradients; and
using hough transform, line segments are detected from the edges.
8. The method according to supplementary note 1, wherein selecting one of the candidate line segment combinations based on a comparison of the aspect ratio corresponding to each of the candidate line segment combinations with a predetermined reference aspect ratio comprises:
a candidate line segment combination having the smallest difference between the aspect ratio and the predetermined reference aspect ratio is selected.
9. An apparatus for determining a rectangular target area in an input image, comprising:
a line segment obtaining device configured to: acquiring a horizontal line segment and a vertical line segment in an input image;
a candidate line segment combination generating device configured to: generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points;
an aspect ratio calculation device configured to: calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image;
a selection device configured to: selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and
a rectangular region determination device configured to: and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
10. The apparatus according to supplementary note 9, the aspect ratio calculation means includes:
a homography matrix obtaining unit configured to: obtaining a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a preset reference aspect ratio;
an aspect ratio calculation unit configured to: and calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination based on a predetermined relation according to the homography matrix corresponding to each candidate line segment combination and the camera internal parameters of the input image.
11. The apparatus according to supplementary note 10, wherein the predetermined relationship is constructed based on a correspondence relationship between world coordinates of the four points before the perspective correction and projective coordinates of the four points in the projective space after the perspective correction, and characteristics of an external parameter matrix of the camera.
12. The apparatus according to supplementary note 9, wherein the line segment obtaining means includes:
an acquisition unit configured to: acquiring line segments in an input image; and
a classification unit configured to: the acquired line segments are classified into horizontal line-like segments and vertical line-like segments.
13. The apparatus according to supplementary note 12, wherein the line segment obtaining means further includes:
a merging unit configured to: and respectively combining at least part of the horizontal line-like segments and/or the vertical line-like segments according to a preset rule.
14. The apparatus of supplementary note 12, wherein the acquisition unit is further configured to:
detecting an edge in an input image; and
line segments are detected from the edges.
15. The apparatus of supplementary note 14, wherein the acquisition unit is further configured to:
detecting edges in the input image based on the gradients; and
using hough transform, line segments are detected from the edges.
16. The apparatus according to supplementary note 9, wherein the selection means is further configured to:
a candidate line segment combination having the smallest difference between the aspect ratio and the predetermined reference aspect ratio is selected.
17. An information processing apparatus comprising:
a controller configured to:
acquiring a horizontal line segment and a vertical line segment in an input image;
generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points;
calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image;
selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and
and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
Claims (8)
1. A method of determining a rectangular target region in an input image, comprising:
obtaining a horizontal line-like segment and a vertical line-like segment in the input image, wherein the horizontal line-like segment is a segment whose difference in slope from the horizontal direction is within a first predetermined range, and the vertical line-like segment is a segment whose difference in slope from the vertical direction is within a second predetermined range;
generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points;
obtaining a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a preset reference aspect ratio;
calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination based on a predetermined relation according to the homography matrix corresponding to each candidate line segment combination and the camera internal parameters of the input image, wherein the predetermined relation is constructed based on the corresponding relation of the world coordinates of four points before perspective correction and the projective coordinates of the four points in the projective space after perspective correction and the characteristics of the external parameter matrix of the camera;
selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and
and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
2. The method of claim 1, wherein obtaining horizontal line-like segments and vertical line-like segments in the input image comprises:
acquiring line segments in an input image; and
the acquired line segments are classified into horizontal line-like segments and vertical line-like segments.
3. The method of claim 2, obtaining horizontal line-like segments and vertical line-like segments in the input image further comprising:
and respectively combining at least part of the horizontal line-like segments and/or the vertical line-like segments according to a preset rule.
4. The method of claim 2, acquiring line segments in the input image comprising:
detecting an edge in an input image; and
line segments are detected from the edges.
5. The method of claim 4, wherein,
detecting edges in the input image based on the gradients; and
using hough transform, line segments are detected from the edges.
6. The method of claim 1, wherein selecting one of the candidate line segment combinations based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio comprises:
a candidate line segment combination having the smallest difference between the aspect ratio and the predetermined reference aspect ratio is selected.
7. An apparatus for determining a rectangular target area in an input image, comprising:
a line segment obtaining device configured to: obtaining a horizontal line-like segment and a vertical line-like segment in the input image, wherein the horizontal line-like segment is a segment whose difference in slope from the horizontal direction is within a first predetermined range, and the vertical line-like segment is a segment whose difference in slope from the vertical direction is within a second predetermined range;
a candidate line segment combination generating device configured to: generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points;
an aspect ratio calculation device configured to: obtaining a corresponding homography matrix for perspective correction according to four intersection points corresponding to each candidate line segment combination and a preset reference aspect ratio, and calculating the aspect ratio of a perspective corrected rectangular region corresponding to each candidate line segment combination based on a preset relation according to the homography matrix corresponding to each candidate line segment combination and camera internal parameters of an input image, wherein the preset relation is constructed based on the corresponding relation of world coordinates of four points before perspective correction and projective coordinates of the four points after perspective correction in a projective space and the characteristics of an external reference matrix of a camera;
a selection device configured to: selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and
a rectangular region determination device configured to: and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
8. An information processing apparatus comprising:
a controller configured to:
obtaining a horizontal line-like segment and a vertical line-like segment in the input image, wherein the horizontal line-like segment is a segment whose difference in slope from the horizontal direction is within a first predetermined range, and the vertical line-like segment is a segment whose difference in slope from the vertical direction is within a second predetermined range;
generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points;
obtaining a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a preset reference aspect ratio;
calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination based on a predetermined relation according to the homography matrix corresponding to each candidate line segment combination and the camera internal parameters of the input image, wherein the predetermined relation is constructed based on the corresponding relation of the world coordinates of four points before perspective correction and the projective coordinates of the four points in the projective space after perspective correction and the characteristics of the external parameter matrix of the camera;
selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and
and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610307727.3A CN107368829B (en) | 2016-05-11 | 2016-05-11 | Method and apparatus for determining rectangular target area in input image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610307727.3A CN107368829B (en) | 2016-05-11 | 2016-05-11 | Method and apparatus for determining rectangular target area in input image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107368829A CN107368829A (en) | 2017-11-21 |
| CN107368829B true CN107368829B (en) | 2020-08-14 |
Family
ID=60304464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610307727.3A Active CN107368829B (en) | 2016-05-11 | 2016-05-11 | Method and apparatus for determining rectangular target area in input image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107368829B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108805823B (en) * | 2018-04-12 | 2021-03-12 | 上海扩博智能技术有限公司 | Commodity image correction method, system, equipment and storage medium |
| CN108734162B (en) * | 2018-04-12 | 2021-02-09 | 上海扩博智能技术有限公司 | Method, system, equipment and storage medium for identifying target in commodity image |
| CN108564557B (en) * | 2018-05-31 | 2020-08-25 | 京东方科技集团股份有限公司 | Image correction method and device |
| CN109409366B (en) * | 2018-10-30 | 2022-04-05 | 四川长虹电器股份有限公司 | Distorted image correction method and device based on angular point detection |
| TWI731484B (en) * | 2019-11-26 | 2021-06-21 | 台灣基督長老教會馬偕醫療財團法人馬偕紀念醫院 | Method and system for building medication library and managing medication via the image of its blister package |
| CN110930300B (en) * | 2019-12-09 | 2023-09-29 | 上海扩博智能技术有限公司 | Image correction method, system, device and storage medium |
| CN115760620B (en) * | 2022-11-18 | 2023-10-20 | 荣耀终端有限公司 | Document correction method and device and electronic equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4067957B2 (en) * | 2002-12-20 | 2008-03-26 | 富士通株式会社 | Boundary detection method, program, and image processing apparatus |
| JP4006601B2 (en) * | 2004-03-29 | 2007-11-14 | セイコーエプソン株式会社 | Image processing system, projector, program, information storage medium, and image processing method |
| JP2008059081A (en) * | 2006-08-29 | 2008-03-13 | Sony Corp | Image processing apparatus, image processing method and computer program |
| US8897538B1 (en) * | 2013-08-26 | 2014-11-25 | Vertifi Software, LLC | Document image capturing and processing |
-
2016
- 2016-05-11 CN CN201610307727.3A patent/CN107368829B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN107368829A (en) | 2017-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107368829B (en) | Method and apparatus for determining rectangular target area in input image | |
| US10187546B2 (en) | Method and device for correcting document image captured by image pick-up device | |
| US9230382B2 (en) | Document image capturing and processing | |
| Kakar et al. | Exposing digital image forgeries by detecting discrepancies in motion blur | |
| US11682225B2 (en) | Image processing to detect a rectangular object | |
| US8811751B1 (en) | Method and system for correcting projective distortions with elimination steps on multiple levels | |
| US8897600B1 (en) | Method and system for determining vanishing point candidates for projective correction | |
| EP3764318A1 (en) | Systems and methods for three dimensional geometric reconstruction of captured image data | |
| JP2016517587A (en) | Classification of objects in digital images captured using mobile devices | |
| JP6642970B2 (en) | Attention area detection device, attention area detection method, and program | |
| US8913836B1 (en) | Method and system for correcting projective distortions using eigenpoints | |
| US9418446B2 (en) | Method and apparatus for determining a building location based on a building image | |
| US20180365837A1 (en) | Image processing apparatus, image processing method, and storage medium | |
| US10586099B2 (en) | Information processing apparatus for tracking processing | |
| EP2536123B1 (en) | Image processing method and image processing apparatus | |
| JP6542230B2 (en) | Method and system for correcting projected distortion | |
| Maiwald et al. | Feature matching of historical images based on geometry of quadrilaterals | |
| CN110135288B (en) | Method and device for quickly checking electronic certificate | |
| CN103839059A (en) | Device and method used for extracting document boundary in document image and electronic device | |
| US20150235399A1 (en) | Variable Patch Shape Synthesis | |
| Koo | Segmentation and rectification of pictures in the camera-captured images of printed documents | |
| JP2011107878A (en) | Position detection apparatus and position detection method | |
| CN106803269B (en) | Method and device for perspective correction of document image | |
| GB2532537A (en) | Aligning multi-view scans | |
| CN107305683B (en) | Image processing method, image processing device and image processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |