KR20180059108A - Apparatus for correcting video and method for using the same - Google Patents

Abstract

An image correction apparatus and method are disclosed. The image correction apparatus according to an aspect of the present invention comprises a feature point extracting part that acquires the image of a target photographed from a camera, selects a mode according to the type of target, and extracts a plurality of feature points forming a quadrangle; a geometry changing part for obtaining a rectangular image by performing a geometry change based on the plurality of feature points on the image of the target; and a color changing part for changing the color of the quadrangle based on a predetermined preset parameter and outputting a corrected image. It is possible to perform geometric and color correction of the image.

Description

TECHNICAL FIELD [0001] The present invention relates to an image correcting apparatus and method,

The present invention relates to image correction techniques, and more particularly, to image geometry and color correction techniques.

Conventional smartphone-based image correction techniques are used in various fields such as correcting images, scanning books, and so on to recognize business cards.

As the various learning methods are implemented in the school scene, the image correction technology should be operated quickly and stably.

Generally, when the image is acquired, the image is distorted according to the inclination of the camera. That is, it is necessary to correct the rectangle again through the geometry change by acquiring images other than the rectangle even when the rectangle is photographed.

Also, in the school education field, lessons are being held indoors and the lighting environment is almost constant. Therefore, you can use preset-based color change for color change.

In other words, it is important to change the color depending on the surrounding environment when there is a change in shadows, colors, etc. at the time of image acquisition with a camera at a school site.

Korean Patent Application No. 10-2013-0142027 entitled " Image correction system, image correction method and smart image acquisition apparatus "relates to an image correction system, an image correction method, and a smart image acquisition apparatus. More particularly, An image correction method using this system, and a smart image acquisition apparatus for correcting an image through this system.

However, in Korean Patent Laid-Open No. 10-2013-0142027, image correction is performed by analyzing a user on the basis of large-volume unstructured data and a database, which makes it difficult to perform image correction easily and quickly in a school education field.

An object of the present invention is to perform geometric and color correction of an image captured by a teacher or a student who is a subject of education in an indoor classroom environment using a smartphone camera in a quasi-real-time manner.

It is another object of the present invention to maximize the effect of education by projecting an image subjected to image geometry and color correction to an electronic blackboard of an indoor classroom.

According to an aspect of the present invention, there is provided an apparatus for measuring a passive intermodulation distortion (PIMD) signal, the apparatus comprising: a central processing unit for transmitting a parameter stored for each index for measuring a passive intermodulation distortion (PIMD) signal; A signal transmitter for transmitting a signal corresponding to a center frequency based on the parameter to a feeder line and a signal receiver for generating a reference signal based on the parameter and calculating a position of a passive element that generates the passive intermodulation distortion signal received from the feeder line .

The present invention can perform geometric and color correction of an image captured by a teacher or a student who is a subject of education in an indoor classroom environment using a smartphone camera in a quasi-real-time manner.

In addition, the present invention can maximize the effect of education by projecting the image subjected to the image geometry and color correction to the electronic blackboard of the indoor classroom.

1 is a block diagram illustrating an image correction apparatus according to an exemplary embodiment of the present invention.
2 to 4 are views showing objects to be subjected to image correction according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a feature point extraction process of a BOOK object according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 6 is a diagram illustrating a region partitioning process of a BOOK object according to an exemplary embodiment of the present invention. Referring to FIG.
7 is a flowchart illustrating an image correction method according to an embodiment of the present invention.
FIG. 8 is an operation flowchart illustrating in detail an example of the feature point extracting step shown in FIG.
FIG. 9 is an operation flowchart illustrating an exemplary FLAT mode execution step shown in FIG. 8 in detail.
FIG. 10 is an operation flow chart illustrating an example of the BOARD mode execution step shown in FIG. 8 in detail.
FIG. 11 is a flowchart illustrating an example of the BOOK mode execution step shown in FIG. 8 in detail.
12 is a block diagram illustrating a computer system in accordance with an embodiment of the present invention.

The present invention will now be described in detail with reference to the accompanying drawings. Hereinafter, a repeated description, a known function that may obscure the gist of the present invention, and a detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an image correction apparatus according to an exemplary embodiment of the present invention. 2 to 4 are views showing objects to be subjected to image correction according to an embodiment of the present invention. FIG. 5 is a diagram illustrating a feature point extraction process of a BOOK object according to an exemplary embodiment of the present invention. Referring to FIG. FIG. 6 is a diagram illustrating a region partitioning process of a BOOK object according to an exemplary embodiment of the present invention. Referring to FIG.

Referring to FIG. 1, an image correction apparatus according to an embodiment of the present invention includes a feature point extracting unit 110, a geometry changing unit 120, and a color changing unit 130.

The feature point extracting unit 110 may extract a plurality of feature points that form a quadrangle by acquiring an image of a photographed object from a camera and selecting a mode according to the object type.

The camera can also correspond to a variety of image and image acquisition devices such as smart phones, tablets, camcorders and PC cams.

The feature point extraction unit 110 may filter an image using a high pass filter for edge extraction.

In this case, the feature point extraction unit 110 may generate filter matrices of NxM type to perform filter operation.

In addition, the feature point extraction unit 110 may perform adaptive binarization.

At this time, the feature point extraction unit 110 can adaptively calculate the binarization threshold T _th . For example, assuming that various characters, figures, tables, and the like are created on a white background of the input image, the feature point extracting unit 110 may have a low value for the adaptive binarization and a low value for the high frequency filter, T _th can be set lower than other regions. To this end, the feature point extracting unit 110 may calculate the T _th for each region by dividing the entire region into tiles. T _th may be set in inverse proportion to the average brightness value of each region.

In addition, the minutiae point extraction unit 110 can select a mode according to the type of target object for minutiae extraction.

At this time, the feature point extraction unit 110 can select one of the FLAT mode, the BOARD mode, and the BOOK mode according to the type of the object.

Referring to FIG. 2, it can be seen that the camera has photographed A4 paper.

As shown in FIG. 2, the feature point extracting unit 110 can select the FLAT mode when the input image is a general A4 paper.

In addition, the feature point extraction unit 110 can perform the selected FLAT mode.

At this time, the feature point extraction unit 110 may first perform line fitting.

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

In addition, the feature point extraction unit 110 may perform line alignment.

The feature point extraction unit 110 may generate a plurality of line segments by fitting an image to a line having a predetermined length or longer. You can sort these segments in order

At this time, the minutiae point extracting unit 110 can select two aligned lines to calculate an intersection point.

At this time, the feature point extraction unit 110 can define the calculated intersection point as a feature point candidate.

In this case, the feature point extracting unit 110 may exclude feature point candidates whose angle formed by the two line segments out of the pair of line segments defining the feature point candidate is greater than or equal to a predetermined value.

That is, since the feature point extracting unit 110 tilts the camera too far, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the feature point candidate that is equal to or less than a predetermined value can be eliminated.

At this time, the feature point extracting unit 110 can extract each of the feature points as a plurality of final feature points by judging whether or not the remaining feature point candidates form a quadrangle, excluding the feature point candidates.

The geometry changing unit 120 may obtain a quadrangle image by performing a geometry change based on the plurality of feature points on an image of a target object.

That is, the geometry changing unit 120 captures an image of a quadrangle, but can change it to a quadrangle when geometric deformation occurs.

At this time, the geometry changing unit 120 may receive the coordinates of a plurality of minutiae points of four coordinates or more than four minutiae (when the end portion of the image is warped and the area-specific correction is necessary).

In this case, the geometry changing unit 120 may generate a rectangular image based on the coordinates of the input minutiae points.

The color changing unit 130 may output a corrected image by changing a color based on a predetermined preset parameter of a rectangular image.

If the indoor environment is changed, the color changing unit 130 may change the preset parameter based on the predetermined indoor environment setting value.

The color changing unit 130 may change the image of the rectangle geometrically corrected to a value such as predetermined saturation, hue, contrast, contrast, etc. in order to reset the preset, or may be manually set by the user.

Thereafter, the color changing unit 130 can output a quadrangle-modified image using the image output device.

The video output device may correspond to a monitor, a smartphone screen, a projector, and the like.

Referring to FIG. 3, it can be seen that the camera photographs a blackboard having an outline.

As shown in FIG. 3, the feature point extraction unit 110 can select a BOARD mode when the input image is a plane having an outline.

In addition, the feature point extraction unit 110 may perform the selected BOARD mode.

At this time, the feature point extraction unit 110 may first select any one of the pattern matching mode and the line fitting mode.

At this time, the feature point extracting unit 110 can perform the selected pattern matching mode.

At this time, the feature point extracting unit 110 can determine the feature points existing at the four corners of the inputted object as the most important feature points.

At this time, the feature point extracting unit 110 may insert the existing feature points of the four corners into the training set of the pattern matching.

At this time, the feature point extracting unit 110 may insert information corresponding to various predetermined slopes.

At this time, the feature point extracting unit 110 may target the binarized image, so it may not be necessary to perform pattern matching for various kinds of illumination.

However, the feature point extracting unit 110 must input various slope information for pattern matching, and performs the pattern matching on the image according to the algorithm used in the present invention and then cuts it.

That is, when the feature point extracting unit 110 performs pattern matching with the binarized image by the binarization method used in the present invention, the search space is reduced, and the feature point can be searched precisely.

In this case, the feature point extracting unit 110 may insert a circular shape as an input as shown in the blackboard shown in FIG. 3, and if the end is at right angles, the same shape can be actually photographed at various angles have. The end portion (edge) may be any shape, as long as it can be recognized by pattern matching.

At this time, the feature point extracting unit 110 can use various learning algorithms used for pattern matching.

In addition, the feature point extracting unit 110 may perform a candidate group post-processing.

That is, after the feature point extracting unit 110 recognizes the shape of the target object by pattern matching, it can perform post-processing of the candidate group to locate the feature point.

At this time, the feature point extracting unit 110 finds the positions of the four patterns constituting the quadrangle among the plurality of pattern-matched candidate groups, and extracts a plurality of final feature points at the corner positions based on the positions.

In addition, the feature point extraction unit 110 may perform the selected line fitting mode.

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

In addition, the feature point extraction unit 110 may perform line alignment.

The feature point extraction unit 110 may generate a plurality of line segments by fitting an image to a line having a predetermined length or longer. You can sort these segments in order

At this time, the minutiae point extracting unit 110 can select two aligned lines to calculate an intersection point.

At this time, the feature point extraction unit 110 can define the calculated intersection point as a feature point candidate.

In this case, the feature point extracting unit 110 may exclude feature point candidates whose angle formed by the two line segments out of the pair of line segments defining the feature point candidate is greater than or equal to a predetermined value.

That is, since the feature point extracting unit 110 tilts the camera too far, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the feature point candidate that is equal to or less than a predetermined value can be eliminated.

At this time, the feature point extracting unit 110 can extract the feature points as a plurality of feature points by judging whether the remaining feature point candidates form a quadrangle, excluding the feature point candidates.

In addition, the feature point extraction unit 110 can search for a dual structure.

That is, as shown in FIG. 3, the feature point extracting unit 110 can search for a quadrangle at a position similar to two quadrangles, because two quadrangles (inside and outside) are generated based on the outline.

At this time, the feature point extracting unit 110 may generate two rectangles by selecting two aligned lines.

At this time, the feature point extracting unit 110 may return an inner square among the two rectangles.

At this time, the feature point extracting unit 110 can extract final feature points based on the corners of the inner square.

In addition, the feature point extraction unit 110 may process the two rectangles at once using the approxPolyDP function, and analyze the output tree structure to extract final feature points.

Further, the feature point extracting unit 110 may determine the feature points having high accuracy by comparing the extracted final feature points with the extracted final feature points by performing the line fitting mode.

The geometry changing unit 120 may obtain a quadrangle image by performing a geometry change based on the plurality of feature points on an image of a target object.

That is, the geometry changing unit 120 captures an image of a quadrangle, but can change it to a quadrangle when geometric deformation occurs.

At this time, the geometry changing unit 120 may receive the coordinates of a plurality of minutiae points of four coordinates or more than four minutiae (when the end portion of the image is warped and the area-specific correction is necessary).

In this case, the geometry changing unit 120 may generate a rectangular image based on the coordinates of the input minutiae points.

The color changing unit 130 may output a corrected image by changing a color based on a predetermined preset parameter of a rectangular image.

If the indoor environment is changed, the color changing unit 130 may change the preset parameter based on the predetermined indoor environment setting value.

The color changing unit 130 may change the image of the rectangle geometrically corrected to a value such as predetermined saturation, hue, contrast, contrast, etc. in order to reset the preset, or may be manually set by the user.

Thereafter, the color changing unit 130 can output a quadrangle-modified image using the image output device.

The video output device may correspond to a monitor, a smartphone screen, a projector, and the like.

Referring to FIG. 4, it can be seen that the camera has taken the opened book.

As shown in FIG. 4, the feature point extracting unit 110 can select the BOOK mode when the input image is a book.

In addition, the feature point extracting unit 110 may perform the selected BOOK mode.

At this time, the feature point extraction unit 110 can divide the input image first.

As shown in FIG. 5, the feature point extracting unit 110 may divide the area by dividing the left side into the center part of the book and the outer part of the right side book with reference to the area dividing line.

At this time, the feature point extracting unit 110 can extract a center line and a corner.

That is, the feature point extracting unit 110 may first extract the corners of the center line and the right portion of the left part using the above-described line fitting and pattern matching.

At this time, since the feature point extracting unit 110 corrects only one side of the book, only one center line and one corner can be extracted.

In addition, the feature point extraction unit 110 can extract a line edge.

At this time, the feature point extracting unit 110 can extract the line edge on the center line and the line edge on the corner, respectively.

At this time, as shown in FIG. 5, the feature point extracting unit 110 may extract an incorrect corner, so that it is possible to newly extract an edge based on the amount of change of the line edge.

In addition, the feature point extraction unit 110 may insert a control point.

As shown in FIG. 6, the feature point extracting unit 110 can insert the control point narrowly because the line edge on the center line is a portion where the book is bent much.

At this time, the feature point extracting unit 110 can insert the control point widely because the corner line edge is a portion in which the book is slightly curved.

At this time, the feature point extracting unit 110 may set the area to be inserted with the control point of the center line side and the edge side area separation to the same width for the purpose of improving the speed.

In addition, the feature point extracting unit 110 can separate regions.

In this case, the feature point extracting unit 110 may separate the center line control point insertion area and the edge control point insertion area and input them to the geometry changing unit 120, respectively.

 The geometry changing unit 120 may perform geometric curve fitting based on the input control point.

At this time, the geometry changing unit 120 may perform geometric curve fitting for the center line control point and the corner control point, respectively.

At this time, since the center line control point is inserted narrowly, the geometry changing unit 120 can perform three-dimensional geometric curve fitting.

For example, as shown in FIG. 6, the geometry changing unit 120 may perform geometric curve fitting by dividing a rectangular area of the control points (c1-c2-c7-c6) have.

At this time, the geometry changing unit 120 may perform the geometric curve fitting by dividing the rectangular area of the control points c2-c3-c8-c7.

That is, the geometry changing unit 120 may perform geometric curve fitting by dividing the control point into a rectangular area while selecting the next control point by predetermined points in the right direction from the control point closest to the center line.

In addition, the geometry changing unit 120 can perform the two-dimensional geometric curve fitting since the corner control point is inserted widely.

Further, the geometry changing unit 120 may combine the image of the rectangle completing the geometric curve fitting on the center line and the image of the rectangle completing the fitting of the geometric curve on the corner to obtain a rectangle having the same geometric curve fitting as the black region shown in FIG. 5 Images can be generated.

The color changing unit 130 may output a corrected image by changing a color based on a predetermined preset parameter of a rectangular image.

If the indoor environment is changed, the color changing unit 130 may change the preset parameter based on the predetermined indoor environment setting value.

The color changing unit 130 may change the image of the rectangle geometrically corrected to a value such as predetermined saturation, hue, contrast, contrast, etc. in order to reset the preset, or may be manually set by the user.

Thereafter, the color changing unit 130 can output a quadrangle-modified image using the image output device.

The video output device may correspond to a monitor, a smartphone screen, a projector, and the like.

7 is a flowchart illustrating an image correction method according to an embodiment of the present invention.

Referring to FIG. 7, an image correction method according to an exemplary embodiment of the present invention may first receive an image (S210).

That is, the step S210 can acquire the image of the object photographed from the camera.

The camera can also correspond to a variety of image and image acquisition devices such as smart phones, tablets, camcorders and PC cams.

In addition, the image correction method according to an embodiment of the present invention can extract feature points (S220).

That is, in step S220, a plurality of minutiae forming a quadrangle can be extracted by selecting a mode according to the object type.

At this time, the step S220 may first select the mode (S221).

At this time, in step S221, an image can be filtered using a high pass filter for edge extraction.

At this time, the step S221 may generate the filter matrix of the NxM type to perform the filter operation.

Also, step S221 may perform adaptive binarization.

At this time, the binarization threshold T _th may be adaptively calculated in step S221. For example, since the input image is because assuming that the various character or figure, table created on white background, to the step (S221) is adaptive binarization, the dark area can lower the result of the high-frequency filter, T _th Can be set lower than other regions. For this purpose, in step S221, the entire area can be divided into tiles to calculate T _th for each area. T _th may be set in inverse proportion to the average brightness value of each region.

At this time, the step S221 can select a mode according to the type of object for extracting the minutiae.

That is, in step S221, any one of the FLAT mode, the BOARD mode, and the BOOK mode may be selected according to the type of the object.

Referring to FIG. 2, it can be seen that the camera has photographed A4 paper.

As shown in FIG. 2, in step S221, the FLAT mode can be selected when the input image is a general A4 paper (S222).

In addition, the step S220 can perform the selected FLAT mode (S224).

In this case, step S224 may first perform line fitting (S224a).

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

In addition, step S224 may perform line alignment (S224b).

That is, in step S224b, if an image is fitted to a line having a predetermined length or more, a plurality of line segments can be generated. Step S224b may sort these lines in order

In addition, step S224 can select the intersection of line segments (S224c)

That is, in step S224c, it is possible to calculate the intersection by selecting two aligned lines.

At this time, the calculated intersection point can be defined as a minutia point candidate in step S224c.

At this time, step S224c may exclude minutiae candidates whose angle formed by the two line segments out of the pair of line segments defining the minutiae candidate is equal to or less than a preset value.

In other words, since the camera is too tilted in step S224c, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the minutiae point candidates that are more or less than a predetermined value can be removed by considering them as meaningless minutiae candidates.

At this time, the step S224c can extract the minutiae candidates as a plurality of final minutiae points, judging whether the remaining minutiae candidates form a quadrangle.

Referring to FIG. 3, it can be seen that the camera photographs a blackboard having an outline.

As shown in FIG. 3, in step S220, if the input image is a plane having an outline, the BOARD mode can be selected (S223).

In addition, step S220 may perform the selected BOARD mode (S225).

That is, the step S225 can select either the pattern matching mode or the line fitting mode (S225a).

At this time, the step S225 can determine whether the selected mode is the pattern matching mode or the line fitting mode (S225b).

In this case, step S225 can perform the selected pattern matching mode (S225c).

In this case, the step S225c may determine the feature points existing at the four corners of the input object as the most important feature points.

At this time, step S225c may insert the existing minutiae of the four corners into the training set of the pattern matching.

At this time, the step S225c may insert information corresponding to various preset slopes.

At this time, since the binarized image is targeted at step S225c, it may not be necessary to perform pattern matching for various illumination.

However, in step S225c, various gradient information must be input for pattern matching, and the pattern-matched image is performed according to an algorithm used in the present invention and then cut and inserted.

That is, if the pattern matching is performed using the binarized image by the binarization method used in the present invention, the search space is reduced and the minutiae can be searched precisely in step S225c.

At this time, in step S225c, as shown in the blackboard shown in FIG. 3, a rounded shape is inserted as an input, and when the end is right-angled, the same shape can be actually photographed at various angles. The end portion (edge) may be any shape, as long as it can be recognized by pattern matching.

At this time, in step S225c, various learning algorithms used for pattern matching can be used.

In addition, the step S225 may perform the post-candidate group post-processing (S225d).

That is, after recognizing the shape of the target object by pattern matching, the candidate group post-process is performed to find the location of the minutiae.

In this case, in step S225d, the positions of the four patterns constituting the quadrangular pattern among the pattern matching candidates may be found, and a plurality of final feature points at the corner position may be extracted based on the positions.

In addition, the step S225 can perform the selected line fitting mode (S225e).

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

Step S225e may also perform line alignment.

That is, in step S225e, if an image is fitted to a line having a predetermined length or more, a plurality of line segments can be generated. Step S225e may sort the lines in order

At this time, in step S225e, it is possible to calculate the intersection by selecting two aligned lines.

At this time, the calculated intersection point can be defined as a minutia point candidate in step S225e.

In this case, in step S225e, minutiae candidates having an angle formed by two line segments out of pairs of line segments that define the minutiae candidate are equal to or less than a predetermined value may be excluded.

In other words, since the camera is too tilted in step S225e, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the minutiae point candidate that is more or less than a preset value can be eliminated.

In this case, in step S225e, it is possible to extract each of the minutiae candidates by determining whether the remaining minutiae candidates form a quadrangle, excluding the minutiae candidate.

Also, step S225 can retrieve the dual structure (S225f).

That is, as shown in FIG. 3, in step S225f, since two rectangles (inside and outside) are generated with respect to the outline, it is possible to retrieve a rectangle at a position similar to two rectangles.

At this time, in step S225f, two aligned lines may be selected to generate two rectangles.

At this time, step S225f may return an inner square among the two squares.

In this case, the step S225f may extract the final feature points based on the corners of the inner square.

In addition, in step S225f, the approxPolyDP function may be used to process two squares at a time, and the output tree structure may be analyzed to extract final feature points.

In addition, the step S225 may determine the minutiae with high accuracy by comparing the extracted final minutiae by performing the pattern matching mode with the accuracy of the extracted minutiae by performing the line fitting mode.

Referring to FIG. 4, it can be seen that the camera has taken the opened book.

As shown in FIG. 4, in step S220, the BOOK mode can be selected when the input image is a book (S223).

In addition, the step S220 can perform the selected BOOK mode (S226).

In this case, in step S226, the input image can be divided first (S226a).

As shown in Fig. 5, in step S226a, the area can be divided by dividing the center part of the book, the outer part of the book on the right side, and the outer part of the book on the basis of the area dividing line.

At this time, the center line and the corner can be extracted in step S226 (S226b).

That is, in step S226b, first, the edge of the center line and the right part of the left part can be extracted using the above-described line fitting and pattern matching.

At this time, since only one side of the book is corrected in step S226b, only one center line and one corner can be extracted.

In addition, the line edge can be extracted in step S226 (S226c).

In other words, step S226c can extract the line edge on the center line and the line edge on the corner, respectively.

At this time, since the erroneous edge may be extracted as shown in FIG. 5, the edge may be newly extracted based on the amount of change of the line edge in step S226c.

In addition, the control point can be inserted in the step S226 (S226d).

As shown in Fig. 6, in step S226d, the control point can be inserted narrowly because the line edge on the center line is a portion where the book is bent much.

At this time, in step S226d, since the corner edge line is a portion where the book is slightly curved, the control point can be inserted widely.

At this time, in order to improve the speed, the step S226d may set the area for inserting the control points of the center line side and the edge side area separation to the same width.

In addition, the step S226 can separate the regions (S226e).

That is, in step S226e, the area where the center line control point is inserted and the area where the corner side control point is inserted may be separately input.

In addition, the image correction method according to an embodiment of the present invention may perform the geometric change (S230).

That is, the step S230 may perform the geometric curve fitting based on the plurality of minutiae points and control points input according to the mode selection.

In this case, when the mode selected for image correction is one of the FLAT mode and the BOARD mode, the image of the object is subjected to the geometric change based on the plurality of feature points to acquire a quadrangle image .

At this time, if a rectangular image is taken in step S230 but a geometric distortion has occurred, it may be changed to a rectangle.

At this time, the coordinates of the plurality of minutiae points of the four coordinates or four or more minutiae of the image (when the end portion of the image is warped and the region-specific correction is necessary) may be inputted in step S230.

At this time, the step S230 may generate a quadrangle image based on the coordinates of the input minutiae points.

In step S230, if the mode selected for image correction is the BOOK mode, geometric curve fitting may be performed based on the input control point.

In this case, step S230 may perform geometric curve fitting for the center line control point and the corner control point, respectively.

At this time, since the control point on the center line is inserted narrowly in step S230, fitting of the three-dimensional geometric curve can be performed.

For example, in step S230, as shown in FIG. 6, geometric curve fitting can be performed by dividing the rectangular area of the control points c1-c2-c7-c6 in the center line area.

In this case, step S230 may perform geometric curve fitting by dividing the rectangular area of the control points c2-c3-c8-c7.

That is, in step S230, geometric curve fitting can be performed by dividing the control point into a rectangular area while selecting the next control point by preset points from the control point closest to the center line to the right.

In step S230, the two-dimensional geometric curve fitting can be performed since the corner control point is widely inserted.

Further, in step S230, a rectangle image of which the fitting of the center curve is completed is combined with a rectangle image of which the fitting of the geometric curve on the corner side is completed, and a rectangle image of which geometric curve fitting like the black area shown in FIG. Can be generated.

In addition, the image correction method according to an embodiment of the present invention may change the color (S240).

That is, in step S240, the color of the quadrangle image can be changed based on preset preset parameters.

In this case, in step S240, if the indoor environment is changed, the preset parameter may be changed based on the predetermined indoor environment setting value.

At this time, the step S240 may change the image of the rectangle geometrically corrected to a predetermined saturation, hue, contrast, contrast or the like so as to reset the preset, or may be manually set by the user.

In addition, the image correction method according to an embodiment of the present invention may output an image (S250).

That is, in step S250, the image of the quadrangled color can be output using the image output device.

The video output device may correspond to a monitor, a smartphone screen, a projector, and the like.

FIG. 8 is an operation flowchart illustrating in detail an example of the feature point extracting step shown in FIG.

Referring to FIG. 8, step S220 may first select a mode (S221).

At this time, in step S221, an image can be filtered using a high pass filter for edge extraction.

At this time, the step S221 may generate the filter matrix of the NxM type to perform the filter operation.

Also, step S221 may perform adaptive binarization.

At this time, the binarization threshold T _th may be adaptively calculated in step S221. For example, since the input image is because assuming that the various character or figure, table created on white background, to the step (S221) is adaptive binarization, the dark area can lower the result of the high-frequency filter, T _th Can be set lower than other regions. For this purpose, in step S221, the entire area can be divided into tiles to calculate T _th for each area. T _th may be set in inverse proportion to the average brightness value of each region.

At this time, the step S221 can select a mode according to the type of object for extracting the minutiae.

That is, in step S221, any one of the FLAT mode, the BOARD mode, and the BOOK mode may be selected according to the type of the object.

Referring to FIG. 2, it can be seen that the camera has photographed A4 paper.

As shown in FIG. 2, in step S221, the FLAT mode can be selected when the input image is a general A4 paper (S222).

In addition, the step S220 can perform the selected FLAT mode (S224).

In this case, step S224 may first perform line fitting (S224a).

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

In addition, step S224 may perform line alignment (S224b).

That is, in step S224b, if an image is fitted to a line having a predetermined length or more, a plurality of line segments can be generated. Step S224b may sort these lines in order

In addition, step S224 can select the intersection of line segments (S224c)

That is, in step S224c, it is possible to calculate the intersection by selecting two aligned lines.

At this time, the calculated intersection point can be defined as a minutia point candidate in step S224c.

At this time, step S224c may exclude minutiae candidates whose angle formed by the two line segments out of the pair of line segments defining the minutiae candidate is equal to or less than a preset value.

In other words, since the camera is too tilted in step S224c, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the minutiae point candidates that are more or less than a predetermined value can be removed by considering them as meaningless minutiae candidates.

At this time, the step S224c can extract the minutiae candidates as a plurality of final minutiae points, judging whether the remaining minutiae candidates form a quadrangle.

Referring to FIG. 3, it can be seen that the camera photographs a blackboard having an outline.

As shown in FIG. 3, in step S220, if the input image is a plane having an outline, the BOARD mode can be selected (S223).

In addition, step S220 may perform the selected BOARD mode (S225).

That is, the step S225 can select either the pattern matching mode or the line fitting mode (S225a).

At this time, the step S225 can determine whether the selected mode is the pattern matching mode or the line fitting mode (S225b).

In this case, step S225 can perform the selected pattern matching mode (S225c).

In this case, the step S225c may determine the feature points existing at the four corners of the input object as the most important feature points.

At this time, step S225c may insert the existing minutiae of the four corners into the training set of the pattern matching.

At this time, the step S225c may insert information corresponding to various preset slopes.

At this time, since the binarized image is targeted at step S225c, it may not be necessary to perform pattern matching for various illumination.

However, in step S225c, various gradient information must be input for pattern matching, and the pattern-matched image is performed according to an algorithm used in the present invention and then cut and inserted.

That is, if the pattern matching is performed using the binarized image by the binarization method used in the present invention, the search space is reduced and the minutiae can be searched precisely in step S225c.

At this time, in step S225c, as shown in the blackboard shown in FIG. 3, a rounded shape is inserted as an input, and when the end is right-angled, the same shape can be actually photographed at various angles. The end portion (edge) may be any shape, as long as it can be recognized by pattern matching.

At this time, in step S225c, various learning algorithms used for pattern matching can be used.

In addition, the step S225 may perform the post-candidate group post-processing (S225d).

That is, after recognizing the shape of the target object by pattern matching, the candidate group post-process is performed to find the location of the minutiae.

In this case, in step S225d, the positions of the four patterns constituting the quadrangular pattern among the pattern matching candidates may be found, and a plurality of final feature points at the corner position may be extracted based on the positions.

In addition, the step S225 can perform the selected line fitting mode (S225e).

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

Step S225e may also perform line alignment.

That is, in step S225e, if an image is fitted to a line having a predetermined length or more, a plurality of line segments can be generated. Step S225e may sort the lines in order

At this time, in step S225e, it is possible to calculate the intersection by selecting two aligned lines.

At this time, the calculated intersection point can be defined as a minutia point candidate in step S225e.

In this case, in step S225e, minutiae candidates having an angle formed by two line segments out of pairs of line segments that define the minutiae candidate are equal to or less than a predetermined value may be excluded.

In other words, since the camera is too tilted in step S225e, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the minutiae point candidate that is more or less than a preset value can be eliminated.

In this case, in step S225e, it is possible to extract each of the minutiae candidates by determining whether the remaining minutiae candidates form a quadrangle, excluding the minutiae candidate.

Also, step S225 can retrieve the dual structure (S225f).

That is, as shown in FIG. 3, in step S225f, since two rectangles (inside and outside) are generated with respect to the outline, it is possible to retrieve a rectangle at a position similar to two rectangles.

At this time, in step S225f, two aligned lines may be selected to generate two rectangles.

At this time, step S225f may return an inner square among the two squares.

In this case, the step S225f may extract the final feature points based on the corners of the inner square.

In addition, in step S225f, the approxPolyDP function may be used to process two squares at a time, and the output tree structure may be analyzed to extract final feature points.

In addition, the step S225 may determine the minutiae with high accuracy by comparing the extracted final minutiae by performing the pattern matching mode with the accuracy of the extracted minutiae by performing the line fitting mode.

Referring to FIG. 4, it can be seen that the camera has taken the opened book.

As shown in FIG. 4, in step S220, the BOOK mode can be selected when the input image is a book (S223).

In addition, the step S220 can perform the selected BOOK mode (S226).

In this case, in step S226, the input image can be divided first (S226a).

As shown in Fig. 5, in step S226a, the area can be divided by dividing the center part of the book, the outer part of the book on the right side, and the outer part of the book on the basis of the area dividing line.

At this time, the center line and the corner can be extracted in step S226 (S226b).

That is, in step S226b, first, the edge of the center line and the right part of the left part can be extracted using the above-described line fitting and pattern matching.

At this time, since only one side of the book is corrected in step S226b, only one center line and one corner can be extracted.

In addition, the line edge can be extracted in step S226 (S226c).

In other words, step S226c can extract the line edge on the center line and the line edge on the corner, respectively.

At this time, since the erroneous edge may be extracted as shown in FIG. 5, the edge may be newly extracted based on the amount of change of the line edge in step S226c.

In addition, the control point can be inserted in the step S226 (S226d).

As shown in Fig. 6, in step S226d, the control point can be inserted narrowly because the line edge on the center line is a portion where the book is bent much.

At this time, in step S226d, since the corner edge line is a portion where the book is slightly curved, the control point can be inserted widely.

At this time, in order to improve the speed, the step S226d may set the area for inserting the control points of the center line side and the edge side area separation to the same width.

In addition, the step S226 can separate the regions (S226e).

That is, in step S226e, the area where the center line control point is inserted and the area where the corner side control point is inserted may be separately input.

FIG. 9 is an operation flowchart illustrating an exemplary FLAT mode execution step shown in FIG. 8 in detail.

Referring to FIG. 9, step S224 may first perform line fitting (S224a).

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

In addition, step S224 may perform line alignment (S224b).

That is, in step S224b, if an image is fitted to a line having a predetermined length or more, a plurality of line segments can be generated. Step S224b may sort these lines in order

In addition, step S224 can select the intersection of line segments (S224c)

That is, in step S224c, it is possible to calculate the intersection by selecting two aligned lines.

At this time, the calculated intersection point can be defined as a minutia point candidate in step S224c.

At this time, step S224c may exclude minutiae candidates whose angle formed by the two line segments out of the pair of line segments defining the minutiae candidate is equal to or less than a preset value.

In other words, since the camera is too tilted in step S224c, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the minutiae point candidates that are more or less than a predetermined value can be removed by considering them as meaningless minutiae candidates.

At this time, the step S224c can extract the minutiae candidates as a plurality of final minutiae points, judging whether the remaining minutiae candidates form a quadrangle.

FIG. 10 is an operation flow chart illustrating an example of the BOARD mode execution step shown in FIG. 8 in detail.

Referring to FIG. 10, in step S225, the pattern matching mode and the line fitting mode may be selected first (S225a).

At this time, the step S225 can determine whether the selected mode is the pattern matching mode or the line fitting mode (S225b).

In this case, step S225 can perform the selected pattern matching mode (S225c).

In this case, the step S225c may determine the feature points existing at the four corners of the input object as the most important feature points.

At this time, step S225c may insert the existing minutiae of the four corners into the training set of the pattern matching.

At this time, the step S225c may insert information corresponding to various preset slopes.

At this time, since the binarized image is targeted at step S225c, it may not be necessary to perform pattern matching for various illumination.

However, in step S225c, various gradient information must be input for pattern matching, and the pattern-matched image is performed according to an algorithm used in the present invention and then cut and inserted.

That is, if the pattern matching is performed using the binarized image by the binarization method used in the present invention, the search space is reduced and the minutiae can be searched precisely in step S225c.

At this time, in step S225c, as shown in the blackboard shown in FIG. 3, a rounded shape is inserted as an input, and when the end is right-angled, the same shape can be actually photographed at various angles. The end portion (edge) may be any shape, as long as it can be recognized by pattern matching.

At this time, in step S225c, various learning algorithms used for pattern matching can be used.

In addition, the step S225 may perform the post-candidate group post-processing (S225d).

That is, after recognizing the shape of the target object by pattern matching, the candidate group post-process is performed to find the location of the minutiae.

In this case, in step S225d, the positions of the four patterns constituting the quadrangular pattern among the pattern matching candidates may be found, and a plurality of final feature points at the corner position may be extracted based on the positions.

In addition, the step S225 can perform the selected line fitting mode (S225e).

The line fitting can be an optimization algorithm that uses a Hough Transform or maps to arbitrary straight lines. At this time, the length of the mapped line can be used only when the length is equal to or larger than a predetermined length. For example, one-fifth of the width and length of the image may be used.

Step S225e may also perform line alignment.

That is, in step S225e, if an image is fitted to a line having a predetermined length or more, a plurality of line segments can be generated. Step S225e may sort the lines in order

At this time, in step S225e, it is possible to calculate the intersection by selecting two aligned lines.

At this time, the calculated intersection point can be defined as a minutia point candidate in step S225e.

In this case, in step S225e, minutiae candidates having an angle formed by two line segments out of pairs of line segments that define the minutiae candidate are equal to or less than a predetermined value may be excluded.

In other words, since the camera is too tilted in step S225e, the image correction can not be successfully performed even if the geometric correction is performed. Therefore, the minutiae point candidate that is more or less than a preset value can be eliminated.

In this case, in step S225e, it is possible to extract each of the minutiae candidates by determining whether the remaining minutiae candidates form a quadrangle, excluding the minutiae candidate.

Also, step S225 can retrieve the dual structure (S225f).

That is, as shown in FIG. 3, in step S225f, since two rectangles (inside and outside) are generated with respect to the outline, it is possible to retrieve a rectangle at a position similar to two rectangles.

At this time, in step S225f, two aligned lines may be selected to generate two rectangles.

At this time, step S225f may return an inner square among the two squares.

In this case, the step S225f may extract the final feature points based on the corners of the inner square.

In addition, in step S225f, the approxPolyDP function may be used to process two squares at a time, and the output tree structure may be analyzed to extract final feature points.

In addition, the step S225 may determine the minutiae with high accuracy by comparing the extracted final minutiae by performing the pattern matching mode with the accuracy of the extracted minutiae by performing the line fitting mode.

FIG. 11 is a flowchart illustrating an example of the BOOK mode execution step shown in FIG. 8 in detail.

Referring to FIG. 11, in step S226, the input image can be divided first (S226a).

As shown in Fig. 5, in step S226a, the area can be divided by dividing the center part of the book, the outer part of the book on the right side, and the outer part of the book on the basis of the area dividing line.

At this time, the center line and the corner can be extracted in step S226 (S226b).

That is, in step S226b, first, the edge of the center line and the right part of the left part can be extracted using the above-described line fitting and pattern matching.

At this time, since only one side of the book is corrected in step S226b, only one center line and one corner can be extracted.

In addition, the line edge can be extracted in step S226 (S226c).

In other words, step S226c can extract the line edge on the center line and the line edge on the corner, respectively.

At this time, since the erroneous edge may be extracted as shown in FIG. 5, the edge may be newly extracted based on the amount of change of the line edge in step S226c.

In addition, the control point can be inserted in the step S226 (S226d).

As shown in Fig. 6, in step S226d, the control point can be inserted narrowly because the line edge on the center line is a portion where the book is bent much.

At this time, in step S226d, since the corner edge line is a portion where the book is slightly curved, the control point can be inserted widely.

At this time, in order to improve the speed, the step S226d may set the area for inserting the control points of the center line side and the edge side area separation to the same width.

In addition, the step S226 can separate the regions (S226e).

That is, in step S226e, the area where the center line control point is inserted and the area where the corner side control point is inserted may be separately input.

12 is a block diagram illustrating a computer system in accordance with an embodiment of the present invention.

Referring to FIG. 12, embodiments of the present invention may be implemented in a computer system 1100, such as a computer-readable recording medium. 12, the computer system 1100 includes one or more processors 1110, a memory 1130, a user interface input device 1140, a user interface output device 1150, And storage 1160. In addition, the computer system 1100 may further include a network interface 1170 connected to the network 1180. The processor 1110 may be a central processing unit or a semiconductor device that executes the processing instructions stored in the memory 1130 or the storage 1160. Memory 1130 and storage 1160 can be various types of volatile or non-volatile storage media. For example, the memory may include ROM 1131 or RAM 1132.

As described above, the image correcting apparatus and method according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments can be applied to all of the embodiments Or some of them may be selectively combined.

110: feature point extracting unit 120: geometry changing unit
130: Color changing section
1100: Computer system 1110: Processor
1120: bus 1130: memory
1131: ROM 1132: RAM
1140: User interface input device
1150: User interface output device
1160: Storage 1170: Network Interface
1180: Network

Claims (1)

A feature point extracting unit for acquiring an image of a subject photographed from a camera and selecting a mode according to a target object type to extract a plurality of feature points forming a quadrangle;
A geometry changing unit for performing a geometry change based on the plurality of feature points on the image of the object to obtain a rectangular image; And
A color changing unit for changing a color of the quadrangle based on a preset parameter and outputting a corrected image;
And an image correction unit for correcting the image.

Images