JP5306060B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a storage medium, and particularly to a technique suitable for use in converting an image into another shape.

  Conventionally, when a plurality of windows are displayed on the screen, the inside of the screen becomes complicated, and therefore, a technique has been proposed in which each window is trapezoidally converted three-dimensionally and the inside of the screen is effectively handled (for example, Patent Document 1).

Japanese Patent Laid-Open No. 11-65806 JP 2007-166209 A

  However, in the method described in Patent Document 1, for example, when an image is trapezoidally converted, even characters included in the image are trapezoidally converted. In particular, when the angle for trapezoidal conversion increases, there is a problem that the characters are greatly crushed in the horizontal direction or the vertical direction, making it difficult to read the characters, for example, as shown in FIG.

  An object of the present invention is to make it easy to recognize characters included in an image when the image is deformed.

The image processing apparatus of the present invention includes a character extraction unit that extracts characters from an image, a correction unit that corrects a portion of the image from which characters have been extracted by the character extraction unit based on a background around the portion, an image deformation unit which deforms the image in which the character is extracted by said character extraction means, and the character position determining means for determining the position and size of the character extracted by said character extraction means, the character position and the character of the placement position and size are determined by the determining means, and an image synthesizing means for synthesizing the image in which the portion is deformed by the image deforming means a corrected image by said correction means It is characterized by having.

  According to the present invention, it is possible to easily recognize characters included in an image even when the image is deformed.

It is a block diagram which shows the structural example of the video processing apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structural example of the video processing apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the video processing apparatus which concerns on 3rd Embodiment. It is a figure which shows the outline | summary of the process which determines the arrangement position of a character using a new character coordinate parameter. It is a figure explaining an example of the process which deform | transforms an image. It is a figure which shows the outline | summary of the process which performs direction deformation | transformation about each character of a character string. It is a figure explaining an example of the process which restores | restores the direction of an image. It is a figure which shows an example of the character string at the time of carrying out the trapezoid conversion of the image conventionally. It is a figure which shows an example of the character string at the time of carrying out the trapezoid conversion of the image. 6 is a flowchart illustrating an example of an image processing procedure in the first embodiment. It is a flowchart which shows an example of the image processing procedure in 2nd Embodiment. 14 is a flowchart illustrating an example of an image processing procedure according to the third embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of a video processing apparatus (image processing apparatus) 100 according to the present embodiment.
In FIG. 1, a video input unit 101 is for inputting two-dimensional image data. The character extraction unit 102 extracts character data from the input image data. Then, the extracted character data 108 is output to the character arrangement position determining unit 105 described later, and the image data 109 from which the character data is extracted is output to the image correcting unit 103.

  The image correcting unit 103 is for correcting a blank portion generated by character extraction in the image data from which characters are extracted by the character extracting unit 102. The image deformation unit 104 is for deforming the shape of the image. The character arrangement position determination unit 105 is for determining the coordinates and size of each character in the extracted character string. The image composition unit 106 synthesizes the image data whose shape has been transformed by the image transformation unit 104 and the character data whose coordinates and size have been determined by the character arrangement position determination unit 105. It is. The video output unit 107 is for outputting the synthesized image data.

Next, the flow of processing will be described with reference to FIG. FIG. 10 is a flowchart illustrating an example of an image processing procedure in the present embodiment.
First, in step S1001, the character extraction unit 102 inputs image data via the video input unit 101, and separates the image data 109 and character data 108 that do not include character data. At this time, the extracted character coordinate information is stored in a memory (not shown) as a character coordinate parameter.

  Next, in step S1002, the image data 109 from which the character data has been extracted has a blank in the portion from which the character has been extracted, so the image correction unit 103 corrects the blank portion. In the present embodiment, as an example of correction, a method is used in which an estimation is performed using a surrounding background image and the image is complemented.

  Next, in step S <b> 1003, the image deforming unit 104 performs conversion processing for deforming the shape of the image on the image data in which the blank portion is corrected by the image correcting unit 103. Here, the image deformation process is a process of deforming into a shape such as enlargement, reduction, rotation, trapezoid, or quadrilateral. For example, as a method of processing to transform into a trapezoid, there is a method of performing coordinate conversion by multiplying and adding an expansion count to each input pixel (see, for example, Patent Document 2).

FIG. 5 is a diagram illustrating an example of processing for deforming an image in the present embodiment.
5A and 5C, the normal axis to the back of the image is the z axis, the horizontal direction of the image is the x axis, and the vertical direction of the image is the y axis. As a deformation procedure, a two-dimensional image is rotated by an angle θ in the depth direction of the screen about the left side of the rectangle, and then trapezoidal deformation is performed by using a perspective projection method.

In FIG. 5A, the coordinates of a point a, b, and c in the image 501 before deformation of d ₁ × 2d ₂ are a = (x ₁ , y ₁ ) and b = (x ₂ , y _{2, respectively.} ), A pixel existing in c = (x ₃ , y ₃ ). As shown in FIG. 5D, by rotating the image 501 by an angle θ, the points a, b, and c move to points a ′, b ′, and c ′ in the rotated image 504, respectively. Here, by using the perspective projection method, the points a ′, b ′, and c ′ are moved to the points a ″, b ″, and c ″ on the screen 505 as shown in FIG. 5B, respectively.

Using this method, for point a, when a ′ = (x ₁ ′, y ₁ ′) and a ″ = (x ₁ ″, y ₁ ″),
x ₁ '= x ₁ cosθ
y ₁ '= y _{1
x 1 "= s- (s-} x 1 ') * L 1 / (L 1 + L 2)
y ₁ "= y ₁ '* L ₁ / (L ₁ + L ₂ )
It becomes. The above calculation is performed for all pixels, and a trapezoidal deformation is performed.

Here, the above calculation is also performed on the character coordinate parameters stored when the character extraction processing is performed, and new character coordinate parameters are obtained. Furthermore, when the coordinates of a certain character are (m, n),
A = (y coordinate of the upper side when x = m) − (y coordinate of the lower side when x = m)
And the ratio with the rotated axis (in this case, the y-axis) is used as the enlargement / reduction parameter.

  In step S1004, the character arrangement position determination unit 105 determines the character arrangement position and the character size using the new character coordinate parameter and the enlargement / reduction parameter calculated by the image transformation unit 104. Specifically, for example, as shown in FIG. 4, an enlargement / reduction parameter is applied to each extracted character to change the size of each character. Thereafter, the position of the character is determined using the new character coordinate parameter.

  Next, in step S <b> 1005, the image composition unit 106 synthesizes the character data determined by the character placement position determination unit 105 and the location and the enlargement / reduction ratio with the image data transformed by the image transformation unit 104. Then, the combined image data is output via the video output unit 107, and the process is terminated.

  As described above, according to the present embodiment, as shown in FIG. 9, even if the image is deformed, the direction and shape of the characters are not changed, so that the visibility of the characters can be improved, and the viewer can Correct information can be obtained from the deformed image.

(Second Embodiment)
The configuration of the video processing apparatus of this embodiment will be described with reference to the block diagram of FIG. FIG. 2 is a block diagram illustrating a configuration example of a video processing apparatus (image processing apparatus) 200 according to the present embodiment.
In FIG. 2, a video input unit 201 is for inputting two-dimensional image data. The image deformation unit 202 is for deforming input image data.
The character extraction unit 203 is for extracting character data from the image data transformed by the image transformation unit 202.

  The image orientation restoring unit 204 is for transforming characters in the character string in the image deformed by the image deforming unit 202 by changing the direction of the character without changing the size one character at a time using the parameters used for the deformation. It is. The image correcting unit 205 is for correcting a blank portion generated by character extraction in the image data 209 from which character data has been extracted by the character extracting unit 203. The image synthesis unit 206 is for synthesizing the character data in which the direction of the character is deformed by the image direction restoration unit 204 and the image data in which the blank portion is corrected by the image correction unit 205. The video output unit 207 is for outputting the synthesized image data.

Next, the flow of processing will be described with reference to FIG. FIG. 11 is a flowchart illustrating an example of an image processing procedure in the present embodiment.
First, in step S1101, the image deformation unit 202 inputs image data via the video input unit 201, and performs a conversion process for deforming the shape of the image on the input image data. Here, the image deformation process is a process of deforming into a shape such as enlargement, reduction, rotation, trapezoid, or quadrilateral.

  As in the first embodiment, in FIGS. 5A and 5C, the normal axis to the back of the image is the z axis, the horizontal direction of the image is the x axis, and the vertical direction of the image is the y axis. To do. As a deformation procedure, a two-dimensional image is rotated by an angle θ in the depth direction of the screen about the left side of the rectangle, and then trapezoidal deformation is performed by using a perspective projection method.

In FIG. 5A, the coordinates of a point a, b, and c in the image 501 before deformation of d ₁ × 2d ₂ are a = (x ₁ , y ₁ ) and b = (x ₂ , y _{2, respectively.} ), A pixel existing in c = (x ₃ , y ₃ ). As shown in FIG. 5D, by rotating the image 501 by an angle θ, the points a, b, and c move to points a ′, b ′, and c ′ in the rotated image 504, respectively. Here, by using the perspective projection method, the points a ′, b ′, and c ′ are moved to the points a ″, b ″, and c ″ on the screen 505 as shown in FIG. 5B, respectively.

Using this method, for point a, when a ′ = (x ₁ ′, y ₁ ′) and a ″ = (x ₁ ″, y ₁ ″),
x ₁ '= x ₁ cosθ
y ₁ '= y _{1
x 1 "= s- (s-} x 1 ') * L 1 / (L 1 + L 2)
y ₁ "= y ₁ '* L ₁ / (L ₁ + L ₂ )
It becomes. The above calculation is performed for all pixels, and a trapezoidal deformation is performed.

  In step S1102, the character extraction unit 203 separates the transformed image data into image data 209 that does not include character data and character data 208. In step S1103, the image data 209 from which the character data is extracted has a blank in the portion from which the character is extracted. Therefore, the image correction unit 205 corrects the blank portion. In the present embodiment, as an example of correction, a method is used in which an estimation is performed using a surrounding background image and the image is complemented.

  Next, in step S1104, the image direction restoration unit 204 returns the character data 208 extracted by the character extraction unit 203 to the original orientation without changing the size of the characters. Specifically, for example, as shown in FIG. 6, the direction of each character in the character string is changed.

FIG. 7 is a diagram for explaining an example of processing for restoring the direction of an image in the present embodiment.
In the image 601 shown in FIG. 7A, a point d ″ and a point e ″ are y = 0, respectively, and the x coordinate of the deformed character is the left end point and the right end point. Here, it is assumed that the image 601 is rotated by θ as shown in FIG. As shown in FIG. 7 (c) as seen from above, a character figure 604 deformed by rotation is generated. This is done by rotating the figure 605 around the point d ″ and using a perspective projection method. That is, the graphic 605 is the original graphic of the graphic 604.

Further, the x and y coordinates of the point d ′ are x ₄ ′ and y ₄ ′, respectively, and the x and y coordinates of the point e ′ are x ₅ ′ and y ₅ ′, respectively. Further, the x and y coordinates of the point d "are x ₄ " and y ₄ ", respectively, and the x and y coordinates of the point e" are x ₅ "and y ₅ ", respectively. Assuming that the length of the figure 605 is L, the following expression is established from the similar relationship.
_{x 5 '/ cosθ: L =} x 5 ": (x 5" -x 4 ")
By calculating L from this equation, the figure 604 can be returned to the shape of the figure 605.

  Next, the restoration of the y coordinate will be described. For the y-coordinate, the length of the axis for rotating each character (the axis passing through d ″ at the right end of the character) is L, and the y-coordinate of the upper and lower sides of the deformed character at x = a A can be restored by multiplying x = a by L / A, so that only the direction can be restored without changing the size.

  In this embodiment, the example in which the character portion is rotated around one side (the axis through which the point d ″ passes) has been described. However, as shown in FIG. 6, the character may be rotated around the center of the character. In this case, restoration is possible by adjusting the length to the difference between the y coordinate of the upper side and the y coordinate of the lower side of the central portion of the character.

  In step S <b> 1105, the image composition unit 206 combines the character data whose characters have been restored by the image direction restoration unit 204 and the image data corrected by the image correction unit 205. Then, the combined image data is output via the video output unit 207, and the process is terminated.

  As described above, according to the present embodiment, as shown in FIG. 9, even if the image is deformed, the direction and shape of the characters are not changed, so that the visibility of the characters can be improved, and the viewer can Correct information can be obtained from the deformed image.

(Third embodiment)
The configuration of the video processing apparatus of this embodiment will be described with reference to the block diagram of FIG. FIG. 3 is a block diagram illustrating a configuration example of a video processing apparatus (image processing apparatus) 300 according to the present embodiment.
In FIG. 3, a video input unit 301 is for inputting two-dimensional image data. The character extraction unit 302 extracts character data from the input image data. Then, the extracted character data 308 and the image data 309 from which the character data 308 is extracted are output to the image transformation unit 303.

  The image transformation unit 303 transforms the image data 309 from which characters are extracted by the character extraction unit 302 and the character data 308. The deformed character data 310 is output to the image direction restoring unit 304 and the deformed image data 311 is output to the image correcting unit 305.

  The image orientation restoring unit 304 transforms the character in the direction of the character string without changing the size one character at a time, using a part of the parameters used in the transformation in the character string transformed in the image by the image transformation unit 303. Is to do. The image correction unit 305 is for correcting a blank portion generated by character extraction in the image deformed by the image deformation unit 303. The image composition unit 306 is configured to synthesize the character data in which the direction of the character is deformed by the image direction restoration unit 304 and the image data in which the blank portion is corrected by the image correction unit 305. The video output unit 307 is for outputting the synthesized image data.

Next, the flow of processing will be described with reference to FIG. FIG. 12 is a flowchart illustrating an example of an image processing procedure in the present embodiment.
First, in step S <b> 1201, the character extraction unit 302 inputs image data via the video input unit 301 and separates it into image data 309 not including character data and character data 308.

  In step S <b> 1202, the image deformation unit 303 performs a conversion process for deforming the shape of the image and characters on the image data 309 and the character data 308 from which character data has been extracted. Note that the same parameter is used for both the image and the character when the image is deformed. Here, the deformation process of an image and characters is a process of deforming into a shape such as enlargement, reduction, rotation, trapezoid, or quadrilateral.

  As in the first embodiment, in FIGS. 5A and 5C, the normal axis to the back of the image is the z axis, the horizontal direction of the image is the x axis, and the vertical direction of the image is the y axis. To do. As a deformation procedure, a two-dimensional image is rotated by an angle θ in the depth direction of the screen about the left side of the rectangle, and then trapezoidal deformation is performed by using a perspective projection method.

In FIG. 5A, the coordinates of a point a, b, and c in the image 501 before deformation of d ₁ × 2d ₂ are a = (x ₁ , y ₁ ) and b = (x ₂ , y _{2, respectively.} ), A pixel existing in c = (x ₃ , y ₃ ). As shown in FIG. 5D, by rotating the image 501 by an angle θ, the points a, b, and c move to points a ′, b ′, and c ′ in the rotated image 504, respectively. Here, by using the perspective projection method, the points a ′, b ′, and c ′ are moved to the points a ″, b ″, and c ″ on the screen 505 as shown in FIG. 5B, respectively.

Using this method, for point a, when a ′ = (x ₁ ′, y ₁ ′) and a ″ = (x ₁ ″, y ₁ ″),
x ₁ '= x ₁ cosθ
y ₁ '= y _{1
x 1 "= s- (s-} x 1 ') * L 1 / (L 1 + L 2)
y ₁ "= y ₁ '* L ₁ / (L ₁ + L ₂ )
It becomes. The above calculation is performed for all pixels, and a trapezoidal deformation is performed.

  In step S <b> 1203, the image direction restoration unit 304 returns the character to the original shape with respect to the character data 310 deformed by the image deformation unit 303. Specifically, for example, as shown in FIG. 6, the direction of each character of the character string is changed.

  Similarly to the second embodiment, in the image 601 shown in FIG. 7A, the point d ″ and the point e ″ are y = 0, respectively, and the x coordinate of the deformed character is the left end point and the right end point. It is. Here, it is assumed that the image 601 is rotated by θ as shown in FIG. As shown in FIG. 7 (c) as seen from above, a character figure 604 deformed by rotation is generated. This is done by rotating the figure 605 around the point d ″ and using a perspective projection method. That is, the graphic 605 is the original graphic of the graphic 604.

Further, the x and y coordinates of the point d ′ are x ₄ ′ and y ₄ ′, respectively, and the x and y coordinates of the point e ′ are x ₅ ′ and y ₅ ′, respectively. Further, the x and y coordinates of the point d "are x ₄ " and y ₄ ", respectively, and the x and y coordinates of the point e" are x ₅ "and y ₅ ", respectively. Assuming that the length of the figure 605 is L, the following expression is established from the similar relationship.
_{x 5 '/ cosθ: L =} x 5 ": (x 5" -x 4 ")
By calculating L from this equation, the figure 604 can be returned to the shape of the figure 605.

  Next, the restoration of the y coordinate will be described. For the y-coordinate, the length of the axis for rotating each character (the axis passing through d ″ at the right end of the character) is L, and the y-coordinate of the upper and lower sides of the deformed character at x = a A can be restored by multiplying x = a by L / A, so that only the direction can be restored without changing the size.

  In this embodiment, the example in which the character portion is rotated around one side (the axis through which the point d ″ passes) has been described. However, as shown in FIG. 6, the character may be rotated around the center of the character. In this case, restoration is possible by adjusting the length to the difference between the y coordinate of the upper side and the y coordinate of the lower side of the central portion of the character.

  Next, in step S1204, since the image data 311 deformed by the image deforming unit 303 has a blank in a portion from which characters are extracted, the image correcting unit 305 corrects the blank portion. In the present embodiment, as an example of correction, a method is used in which an estimation is performed using a surrounding background image and the image is complemented.

  In step S <b> 1205, the image composition unit 306 combines the character data whose characters have been restored by the image direction restoration unit 304 and the image data corrected by the image correction unit 305. Then, the combined image data is output via the video output unit 307, and the process is terminated.

  As described above, according to the present embodiment, as shown in FIG. 9, even if the image is deformed, the direction and shape of the characters are not changed, so that the visibility of the characters can be improved, and the viewer can Correct information can be obtained from the deformed image.

(Other embodiments according to the present invention)
Each means constituting the image processing apparatus and each step of the image processing method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable storage medium storing the program are included in the present invention.

  In addition, the present invention can be implemented as, for example, a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  In the present invention, a software program for realizing the functions of the above-described embodiments (in the embodiment, a program corresponding to the flowcharts shown in FIGS. 10 to 12) may be supplied directly or remotely to the system or apparatus. Including. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

102 character extraction unit, 104 image transformation unit, 105 character arrangement position determination unit, 106 image composition unit

Claims (12)

Character extraction means for extracting characters from the image;
Correction means for correcting the portion of the image from which characters have been extracted by the character extraction means based on the background around the portion;
Image deformation means for deforming the image from which the characters have been extracted by the character extraction means;
A character position determining means for determining the position and size of the character extracted by said character extraction means,
Synthesizing said character and placement position and size are determined by the character arrangement position determination means, an image in which the portion is deformed by the image deforming means a corrected image by said correction means And an image synthesizing unit. Image deformation means for deforming an image including characters;
Character extraction means for extracting characters from the image transformed by the image transformation means;
Image direction restoring means for returning the direction of the character extracted by the character extracting means to a direction before being deformed by the image deforming means;
An image processing apparatus comprising: an image compositing unit that combines the character whose direction has been returned by the image direction restoring unit and the image deformed by the image deforming unit. Character extraction means for extracting characters from the image;
Image deformation means for deforming the image from which the character has been extracted by the character extraction means and the character extracted by the character extraction means;
Image orientation restoring means for returning the orientation of the character transformed by the image transforming means to the orientation before the transformation;
An image processing apparatus comprising: an image compositing unit that combines the character whose direction has been returned by the image direction restoring unit and the image deformed by the image deforming unit. Image correction means for correcting the portion of the image from which the characters have been extracted by the character extraction means based on the background around the portion ;
The image synthesizing means includes a character orientation returned by said image direction restoration means, according to claim 2 or 3 before Symbol unit content characterized by combining the corrected image by the image correcting unit Image processing device. Wherein the image deforming means is an image processing apparatus according to any one of claim 1 to 4, characterized in that the deformation of the image by using a perspective projection. A character extraction process for extracting characters from the image;
A correction step of correcting the portion of the image from which characters have been extracted in the character extraction step based on the background around the portion;
An image deformation step of deforming the image from which the characters have been extracted in the character extraction step;
A character placement position determination step for determining the placement position and size of the character extracted in the character extraction step;
Image to be combined with the character that the character position determination process smell Te and the placement position and size are determined, and an image, wherein the portions in the correction process is modified in the image deforming step a corrected image And an image processing method. An image transformation process for transforming an image including characters;
A character extraction step of extracting characters from the image transformed in the image transformation step;
An image orientation restoration step for returning the orientation of the characters extracted in the character extraction step to the orientation before being transformed in the image transformation step;
An image processing method comprising: an image composition step of combining the character whose direction has been returned in the image direction restoration step and the image deformed in the image deformation step. A character extraction process for extracting characters from the image;
An image transformation step for transforming the image from which the character has been extracted in the character extraction step and the character extracted in the character extraction step;
An image orientation restoring step for returning the orientation of the character transformed in the image transformation step to the orientation before the transformation;
An image processing method comprising: an image composition step of combining the character whose direction has been returned in the image direction restoration step and the image deformed in the image deformation step. A character extraction process for extracting characters from the image;
A correction step of correcting the portion of the image from which characters have been extracted in the character extraction step based on the background around the portion;
An image deformation step of deforming the image from which the characters have been extracted in the character extraction step;
A character placement position determination step for determining the placement position and size of the character extracted in the character extraction step;
Image to be combined with the character that the character position determination process smell Te and the placement position and size are determined, and an image, wherein the portions in the correction process is modified in the image deforming step a corrected image A program for causing a computer to execute a synthesis step. An image transformation process for transforming an image including characters;
A character extraction step of extracting characters from the image transformed in the image transformation step;
An image orientation restoration step for returning the orientation of the characters extracted in the character extraction step to the orientation before being transformed in the image transformation step;
A program that causes a computer to execute an image composition step of combining a character whose direction has been returned in the image direction restoration step and an image deformed in the image deformation step. A character extraction process for extracting characters from the image;
An image transformation step for transforming the image from which the character has been extracted in the character extraction step and the character extracted in the character extraction step;
An image orientation restoring step for returning the orientation of the character transformed in the image transformation step to the orientation before the transformation;
A program that causes a computer to execute an image composition step of combining a character whose direction has been returned in the image direction restoration step and an image deformed in the image deformation step. A computer-readable storage medium storing the program according to any one of claims 9 to 11 .

Images