KR101887929B1 - Image Processing Apparatus, Image Processing Method, Computer Readable Recording Medium and Image Forming Apparatus - Google Patents

Abstract

The present invention relates to an image processing apparatus, an image processing method, a computer-readable recording medium, and an image forming apparatus, wherein an image processing apparatus according to an embodiment of the present invention includes: A color evaluation unit for estimating the color of the background except for the text and the text in the divided part of the text, a vectorizing unit for vectorizing the outline of the text in the divided text part, and a text part for dividing the color of the estimated background into the text A text retouching unit for correcting the screen shot, and a metafile generating unit for generating a metafile using the color of the estimated text, the vector information of the vectorized outline, and the modified screen shot.

Description

TECHNICAL FIELD The present invention relates to an image processing apparatus, an image processing method, a computer readable recording medium, and an image forming apparatus,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, a computer readable recording medium, and an image forming apparatus, and more particularly to a method and apparatus for processing a raster image of a screen shot, To an image processing apparatus, an image processing method, a computer readable recording medium, and an image forming apparatus which are to be converted into a metafile.

The vectorization of the text information for the raster image of the screen shot essentially increases the text quality, such as printing and visualization. Partial vectorization of screen shots is a balanced approach to solving troublesome tasks, in that the detected figures and text areas are processed in different ways. The text portion is converted into a series of straight lines and curves with information about the symbols and the color of the surrounding background. The rest of the initial screen shot information is stored as a raster image.

The method disclosed in US Pat. No. 8,270,722 processes images with vectorization of selected symbols and graphics regions. The object of the invention is to compress an image including two areas of a text area and a graphic area to the best quality in the best quality state. If the symbol pixels in the region of the text are overlapped by the graphics region, the graphics region is first vectorized, and if there is no overlap, the text region is first vectorized.

Also, the method disclosed in US patent 7,873,218 is roughly composed of symbols for transformation of a binary image in contour data using approximate Bezier curves. The method includes extracting symbol regions and dividing the raster image to extract symbols in the extracted symbol regions. The vectorization process is performed independently in the non-text parts of the image.

The above-mentioned methods improve the quality of the visual picture by the vectorization method, but are directly applicable to the raster image vectorization of the screen shot, which handles a number of properties, such as low resolution of symbols, antialiasing effect application of text symbols, Can not be applied.

The method disclosed in US Patent 7,079,686 describes an approach based on classifying pixels in an image of a document to extend the image. A vector for the properties is generated for each pixel of the images. Each pixel is categorized as a text or picture depending on the appropriate vector for the properties. Further, the processing may include an improvement at the pixel level consisting of a filter that increases the sharpness of the boundaries in the pixels of the text, and an anti-aliasing filter for the pixels of the picture.

Furthermore, the method disclosed in US patent 7,177,049 represents the processing of digital images, wherein the process of improving the text quality is such that the contrast and redistribution between dark and light pixels in a predetermined mask increases the sharpness and contrast ratio of the text, Lt; / RTI >

The methods described above only focus on image enhancement, especially text areas, before printing. However, these conventional methods are not suitable for screen shots for developing scan images.

Recently, a method for generating smooth text (or anti-aliased text) in a screen shot is urgently required.

An embodiment of the present invention provides an image processing apparatus, an image processing method, a computer readable recording medium, and an image forming apparatus for converting a raster image of a screen shot into a metafile through screen shot division and vectorization, for example, in the field of digital signal processing It has its purpose.

An image processing apparatus according to an embodiment of the present invention includes a division unit for dividing a text portion in a screen shot including a graphic and a text, a color estimating unit for estimating a color of the background excluding the text and the text in the divided text portion A vectorizing unit for vectorizing the outline of the text in the divided text portion, a text retouching unit for modifying the screen shot by filling the divided text portion with the color of the estimated background, , Vector information of the vectorized outline, and a metafile generator for generating a metafile using the modified screen shot.

The dividing unit generates a map for a plurality of text parts and provides map data for the generated map to the vectorizing unit, and the plurality of text parts may be the same size raster image.

Wherein the dividing unit detects a plurality of pixels having a horizontal gradient value that is brighter and darker than an adjacent pixel with respect to an arbitrary pixel in the screen shot than the surrounding pixels and combines the detected pixels into respective parts, The text portion can be detected according to the analysis result of analyzing the histogram of each portion.

The plurality of pixels may be detected by selecting the endpoints of non-zero associated pixels where the gradient of the gradient for the screen shot is present.

As a result of the analysis, the dividing unit may determine the text portion if the number of columns having a tone value of 0 is greater than the threshold value, and may determine the text portion if the number is smaller than the threshold value.

The color evaluation unit may estimate a column located at the leftmost or rightmost position on the histogram in the color of the background and the text.

Wherein the vectorization unit converts the text portion into a halftone image to vectorize the outline of the text, binarizes the converted halftone image, tracks the outline of the symbols in the binarized image, And polygon approximation to a series of refinement portions of curves.

According to another aspect of the present invention, there is provided an image processing method including dividing a text portion in a screen shot including graphics and text, estimating a color of a background excluding the text and the text in the divided text portion, The method comprising the steps of: vectorizing an outline of the text in the segmented text portion; modifying the screen shot by populating the segmented text portion with the color of the estimated background; And generating the meta file using the vector information and the modified screen shot.

Wherein the dividing step includes generating a map for a plurality of text parts and providing map data for the generated map for the vectorization, wherein the plurality of text parts can be raster images of the same size have.

Wherein the dividing step comprises the steps of: detecting a plurality of pixels having a horizontal gradient value that is brighter and darker than an adjacent pixel with respect to an arbitrary pixel in the screen shot than the surrounding pixels; And dividing the text portion according to an analysis result of analyzing the histogram of each portion.

The plurality of pixels may be detected by selecting the endpoints of non-zero associated pixels where the gradient of the gradient for the screen shot is present.

If the number of columns having a gray value of 0 is greater than the threshold value, the dividing may be determined as the text portion, and if the number of columns is less than the threshold value, the dividing may be determined as a graphic portion.

The color estimating step may estimate a column located at the leftmost or rightmost position on the histogram in the color of the background and the text.

Wherein the step of vectorizing comprises the steps of converting the text portion into a halftone image, binarizing the transformed halftone image, tracking the outline of the symbols in the binarized image, And performing a polygon approximation with a series of refinement portions of curves.

A computer-readable medium according to an embodiment of the present invention is a computer-readable medium for executing an image processing method, the method comprising: dividing a text portion in a screen shot including graphics and text; Estimating a color of the background excluding the text and the text, vectorizing an outline of the text in the divided text part, filling the divided text part with the color of the estimated background, And generating a metafile using the color of the estimated text, vector information of the vectorized outline, and the modified screen shot.

The image forming apparatus according to an embodiment of the present invention may further include a communication interface unit for receiving a screen shot including graphics and text from the terminal apparatus, Generating vector information by vectorizing text of a text portion in a screen shot, modifying the screen shot in which the text portion is divided using a background color of the text portion, and generating the generated vector information, An image generation unit that generates a metafile using a text color of a text portion, and a printing unit that prints the generated metafile according to a user's request.

The image generation unit may modify the screen shot by dividing the text portion in the screen shot and filling the divided text portion with the background color.

Wherein the image generation unit estimates the background color and the text color by analyzing a histogram of the text and a background other than the text in the text part, and corrects the screen shot and the estimated background color And the text color.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a comparison between a screen shot conversion method and a conventional method according to an embodiment of the present invention;
2 is a flowchart showing a method of converting a screen shot into a metafile,
3 is a block diagram illustrating a system for converting a screen shot into a metafile,
4 is a diagram illustrating a method of detecting portions having smooth text,
5 is a diagram for explaining a calculation process for smooth text features,
6 is a flowchart showing a vectorization process of symbols,
7 is a diagram for explaining vectorization of symbols,
8 is a diagram showing a process of generating a map of divided texts,
9 is a diagram showing a result of a screen shot conversion according to an embodiment of the present invention,
Figure 10 illustrates an image conversion system according to an embodiment of the present invention;
11 is a block diagram showing the detailed structure of the terminal device of Fig. 10, and Fig.
12 is a block diagram showing a detailed structure of the image forming apparatus of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram illustrating a screen shot conversion method according to an embodiment of the present invention in comparison with a conventional method.

The screen shot (or captured image) 103 is captured (or stored) in the intermediate buffer by the display screen via the video memory. The result of such copying is in the form of a raster image representing a frame that is currently visible on the display.

Here, the display includes a unit of visualization of a digital computing device, such as a PC 101, a laptop, a smartphone 102, a tablet PC, and the like. The feature of the screenshot is that the textual information shown in the screenshot is normally visualized by the anti-aliasing technique. Here, the anti-aliasing technique is used to remove the jags indicated by the rasterization at the edges of objects to visually smooth the boundaries of the symbols. In this case, pixels adjacent to the boundary pixels of the image will have an intermediate value between the image color and the background color to visually blur the boundary.

The anti-aliasing technique used to visualize the text on the display is directly dependent on the size of the displayed symbols and the representation resolution, so that the pixel representation of the symbols is reconstructed with scale up / down. In addition, pseudo-anti-aliasing is fundamentally different from similar techniques used for text printing. Screenshots in the form of raster images are stored in certain files / formats 104, making it impossible to apply anti-aliasing techniques to display or print. In other words, applying anti-aliasing techniques causes visual unwanted distortions in the text's appearance. Therefore, the existing approach has the following drawbacks.

When a screen shot is saved as a file in the form of a raster image, text information is not normally displayed on the display because the file size is ignored and the display of the symbol outline is scaled or smoothed. That is, the scaling of the raster image of the text results in chaotic parameters of jagged edges and fuzzy boundaries.

There is also a negative effect of anti-aliasing of the symbols corresponding to the text visualized on the display. In particular, distortion in the shape of the original color and symbols occurs during printing or visualization re-scaling.

The image processing method 105 according to the embodiment of the present invention is based on a screen shot division (or subdivision) into a text and a non-text area and a vectorization of a text area. This approach makes it possible to display and print metafiles on a modified scale without compromising the quality of the text information. Here, the meta file means a file in which an image is defined in a vector format. In other words, the metafile defines the line and plane as vector coordinates. This makes the image unbreakable even when zoomed in or out.

2 is a flowchart showing a process of converting a screen shot into a metafile.

For example, the image processing apparatus according to the embodiment of the present invention may perform a process of converting a screen shot into a metafile as shown in FIG.

The image processing apparatus according to the embodiment of the present invention first divides (or detects) text portions in a screen shot, and generates a map for the divided text portions (S201). In an embodiment of the present invention, given text portions include a sequence of symbols corresponding to one or several characters and a portion of the surrounding background, the map is a raster image of the same size, Each pixel is an encoded initial screen shot, whether related to the text area or not.

The image processing apparatus then estimates (or evaluates) the symbols of the embodied text portion and the background color (S202). In an embodiment of the present invention, the screen shots are represented by color space RGB, so that each pixel is represented by three color components of R, G, and dark B color. Each estimate of the three color components is separated by histograms of these components. The rightmost or leftmost column (group of columns) of the histogram with the largest value (total value) corresponds to the background color opposite to the color of the symbol.

Further, the image processing apparatus performs vectorization of text (S203). Contour vectorization of symbols in a map of text is accomplished by transforming into a series of segmentation segments of straight lines and curves. A related process will be described later with reference to FIGS. 6 and 7. FIG.

The image processing apparatus modifies the text on the screen shot (S204). The screenshot is modified by populating the split text areas with the appropriate background color. Because the background is complex, a more complete method can be used to fill the text area. For example, " inpainting " technique, i.e. (Bertalmio M., Sapiro G., V., Ballester C. Image inpainting // In Proc. ACM Conf. Comp. Graphics (SIGGRAPH), pages 417-424, 2000).

Thereafter, the image processing apparatus generates a meta file (S205). A metafile is created (or based on) the appropriate records of a metafile stored in a detailed (or specific) order. The records may include vector information, coordinate values, and the like. Metafiles can be represented in Portable Document Format (PDF), X-ray Photoelectron Spectroscopy (XPS), PS and Enhanced Metafile (EMF) formats.

The modified image of the screenshot is stored in the metafile as a raster image. The symbols are stored in a vector view, where the vector view includes parameters for a series of closed (or closed) subdivision segments of straight lines and curves, and a meta It is by appropriate graphic commands of the file. In order to optimize the size of the metafile, a lossless compression algorithm can be applied to record the vector of the metafile and a lossy or lossless compression algorithm can be applied to the raster images. The image processing method according to the embodiment of the present invention effectively stores the metafile at the view point " visual quality / size " when each type of visual information is stored in an optimal manner. Further, the image processing method according to the embodiment of the present invention prevents smooth text from being distorted in the screen shot when printing or visualizing on an extended scale. This is possible because the smooth symbols in the screenshot are replaced by vectors and the like.

FIG. 3 is a block diagram illustrating a system for converting a screen shot into a metafile, FIG. 4 is a diagram illustrating a process of detecting portions having smooth text, and FIG. 5 is a flowchart illustrating a process of calculating a smooth text FIG.

Here, the conversion system may correspond to the image processing apparatus of the present invention. In addition, the image processing apparatus according to an embodiment of the present invention may be configured not only as an independent apparatus but also as an image display apparatus such as a laptop and a smart phone.

3, an image processing apparatus 300 according to an exemplary embodiment of the present invention includes a division unit 301, a color evaluation unit 302, a text retouching unit 303, a vectorization unit 304, (305). ≪ / RTI >

Here, the dividing unit 301 may be called a detecting unit or a dividing unit, and the part or all of the dividing unit 301 may include some components such as the dividing unit 301 may be omitted or some components such as the text retaking unit 303 may be omitted It is to be understood that the elements may be incorporated into other elements and the like, and that they are all included in order to facilitate a sufficient understanding of the invention.

The division unit 301 divides the text, generates a labeled map of the divided text, and determines the coordinates of the portions including the text. The initial raster image of the screen shot is input to the division unit 301. [ The map of the divided text is sent to the text retouching unit 303 and the vectorizing unit 304, and the coordinates of the text parts, or information thereon, is transmitted to the color evaluating unit 302. [

The color evaluation unit 302 determines the background color of each part and the color of the text. Of course, the coordinates of the text portions and the initial screen shot are input to the color evaluation unit 302. [ The information on the background color is transmitted to the text retouching unit 303, and the text color information is transmitted to the metafile generating unit 305.

The vectorization unit 304 performs an approximation operation that approximates the outline of symbols on the map of the subdivided text, with the subdivided portions of the closed series of straight lines and curves.

 The map from the division unit 301 is input to the text retouching unit 303 and the vectorizing unit 304. [ A series of refinement parts consisting of the coordinates of the text parts, straight lines and curves are transmitted to the metafile generating part 305.

The text retouching unit 303 modifies the initial raster image of the screen shot by coloring the area of the text with the appropriate background color. To this end, information on the background color of each text portion is input to the text retouching unit 303. The modified screen shot is transmitted to the metafile generation unit 305 where the metafile generation unit 305 generates a modified screen shot, such as a closed series of refinement portions of straight lines and curves filled with the appropriate symbol color The metafile is generated based on the records.

All of the components of the conversion system described above, that is, the division unit 301, the color evaluation unit 302, the text retouching unit 303, the vectorizing unit 304, and the metafile generating unit 305, It can be executed in the form of ASIC. The function of each component can be fully understood from the description of the above description and the existing methods. However, the computing system may have additional features, although not separately shown.

Two approaches for text detection may be possible. The first method is a smoother text detection method typical of displays, in a faster way. The second is a more common way to detect both smooth text and non-anti-aliased text. The first method related to the detection of text portions includes the following steps.

- Detect adjacent pixels of a large derivative in the horizontal direction

- combining adjacent pixels in parts

- Histogram analysis of parts to detect smooth text

- Select the parts with the characteristic of smooth text

)(403)의 일반적 의미는 어떤 지점(point)의 픽셀이 인접 픽셀들보다 밝고 어두운 정도(혹은 밝고 /어두운 인접 픽셀들)인 양(

) 및 음(

) 그라디언트들(gradients)의 결합이다. 이를 수식으로 나타내면 <수학식 1>과 같다.Figure 4 illustrates a first approach to the detection of smooth text portions in a screen shot for subsequent analysis. Referring to FIG. 4, a small portion 401 of one of the screen shots shows the detection result of the horizontal gradient. Calculated Horizontal Gradient (

) 403 generally means that a pixel at a point is less bright and dark (or bright / dark adjacent pixels) than neighboring pixels

) And negative

) &Lt; / RTI > gradients. This can be expressed by Equation (1).

Where Ii, j is the image contrast at point i, j, and Δ is the current threshold.

The binary image 402 is the result of the detection of the gradient, where non-zero elements show the presence of a gradient. An extension (or dilatation) procedure with the horizontal structural element 403 is performed because the connection of adjacent elements is weak on the binary image 402. [ At this time, the horizontal structural element 403 may be made by a method of determining a dominant box. Here, it is to be understood that the dominant means selecting the association box according to a predetermined method for any box. The separated portions 404 are determined by selecting the endpoints of non-zero associated pixels.

Further, the histogram of each selected portion is analyzed for smooth character detection of text and selection of portions including symbols.

FIG. 5 shows a histogram analysis of all three color components (red, green, and dark blue) of the color space RGB to detect the characteristics of smooth text, i.e., the non-zero b + column of the histogram and the number of bo columns of zero.

The histogram of the portion 504 including the non-text data has a completely abrupt distribution 501 to 503 of each component of the color as shown in FIG. 5 (a). The histogram of the portion 508 including the smooth text is composed of a plurality of b + values uniformly distributed (505 to 507) between the background and the color of the text as shown in (b) of FIG. 5 (B + = 7 for experiments). It consists of non-zero columns only. Referring to FIG. 5C, in an embodiment of the present invention, if bo is greater than 6, the portion is classified as text, otherwise classified as an image (e.g., an image) (509) .

A second approach for detecting portions with text includes the following steps.

- Separate images into overlapping rectangular blocks.

- Calculate the characteristics of each block

- Classify each block

Combine (or combine) adjacent blocks in text parts,

According to a preferred embodiment of the present invention, the block size has 7x7 or 9x9 pixels. The overlap of the blocks is 1, 2 or 3 pixels. Each block has the following characteristics.

- the number of columns b + and zero columns bo with non-zero histograms in each channel

, 여기서, Ii(r,c)는 라인 r과 칼럼 c에서 픽셀의 명암(luminance), N은 블록의 픽셀들의 수를 나타낸다.- average luminance of the block:

, Where Ii (r, c) is the luminance of the pixel in line r and column c, and N is the number of pixels in the block.

- the average difference of the average contrast of the blocks

- Mean difference of average contrast of adjacent blocks Ik connected in 4 directions (in quadryply) with block Ii:

와 수평

도함수(derivatives)들의 평균값:

- vertical by blocks

And horizontal

Mean value of derivatives:

, 여기서 Nd는 정규화된 접근 행렬(normalized access matrix), d는 공간 접속을 결정한다.- Block uniformity:

, Where Nd is the normalized access matrix and d is the spatial connection.

, 여기서

는 제곱된 수평 또는 수직 도함수들의 합의 제곱 루트로 계산된다.- Percent of pixels with a gradient greater than the threshold:

, here

Is calculated as the root of the sum of the squared horizontal or vertical derivatives.

에 형태학적 개방(morphological opening) 동작을 적용한 결과로서 얻어진 이미지

에서 픽셀들의 명암 변화 퍼센트;

- Binary image obtained by binarizing to threshold 128

The image obtained as a result of applying a morphological opening operation to the image

The percentage change in the contrast of pixels in pixels;

One of the following methods may be used for the properties listed above for the classification of two subsets: text and pictures (or graphics).

(1997) A decision-theoretic generalization of on-line learning and an application to boosting, J. Comput. Syst. Sci. 55, 1 (August 1997), 119 -139.),

random forest (Leo Breiman, 2001. Random Forests, Mach., 45, 1 (October 2001), 5-32.),

Machine of reference vectors (Corinna Cortes and Vladimir Vapnik, 1995. Support-Vector Networks, Mach., 20, 3 (September 1995), 273-297)

method of K-nearest neighbors (D. Coomans, DL Massart, Alternative k-nearest neighbor rules in supervised pattern recognition: Part 1. k- Nearest neighbor classification by using alternative voting rules, Analytica Chimica Acta, Volume 136, 1982, Pages 15 -27).

FIG. 6 is a flowchart illustrating vectorization of symbols, FIG. 7 is a diagram illustrating vectorization of symbols, and FIG. 8 is a diagram illustrating a process of generating a divided text map.

6 to 8 can be actually understood as a detailed process of the step S203 of FIG. Such an operation may correspond to the operation of the image processing apparatus, but may correspond to the operation of the vectorization unit 304 of FIG. They may also operate as separate devices.

First, after detecting the portions including the text in the screen shot, the color portions are converted into halftone images (S601) and increased by a coefficient k.

According to a preferred embodiment of the present invention, if the initial resolution of the symbol in the screen shot is low, bilinear interpolation is applied (S602). For example, assuming a screen scale of 1920 x 80 pixels, assuming a 100% scale, the height of the line symbol of the 12th sized Times New Roman font is 7 to 8 pixels. In addition, halftone portions of enhanced resolution may be binarized (S603).

According to an embodiment of the present invention, preferably the method of Otsu can be used to determine the threshold of parts (N. Otsu, " A threshold selection method from gray level histogram ", IEEE Transactions on System Man Cybernetics, vol 9 no 1, 1979, pp. 62-66.). All of the binarized parts are combined according to the coordinates of the screen shot for map generation of the refined text. In the embodiment of the present invention, 3 may be used as the coefficient k.

The inner and outer contours of each text portion are also traced to obtain a trajectory for the contours of that portion (S604). In step S604, the contour represents a closed series of dots (or tops) connected by linear portions of minimum length, such as one pixel. The tracking procedure starts from the initial upper side along the contour in the predetermined direction, but will not return to the initial upper side.

Then, the number of the upper sides is reduced due to the approximation of the contour trajectory by the polygon (S605). For example, the number of vertices in the contour is reduced. The polygon is transformed into a smooth outline in step S606 due to the approximation by a series of closed subdivisions of straight lines and curves. Cubic Bezier curves may be preferred in embodiments of the present invention (Piegl, L. Fundamental Developments of Computer Aided Geometric Design, San Diego, CA: Academic Press, 1993). In general, the approximation by linear segmentation (or parts) is based on the coordinate determination of the ends for the parts. The approximation by curved parts requires the determination of the end of the parts and appropriate reference (or reference) points. 7, an angle 703 between the edges 701-706 of the polynomials may be approximated by a constrained cubic Bezier curve 705 by points 702 and 704. For example, An approximate example of symbol 707 is shown at 708.

The text conversion of the screen shot from the initial raster view to the vector view according to S201, S203, and S204 in Fig. 2 is shown in Fig. The smooth text portion S801 is increased in resolution (S802) and converted into a binary image (S803). The vectorized result portion is shown in S804.

FIG. 9 is a diagram showing a screen shot conversion result according to an embodiment of the present invention.

9 (a) corresponds to the initial screen shot, and FIG. 9 (b) corresponds to the printed screen shot portion. 9 (c) is obtained by printing a screen shot of the resolution enhanced by the double interpolation method, and FIG. 9 (d) shows the printed screen shot portion after being processed by the embodiment of the present invention.

10 is a diagram showing an image conversion system according to an embodiment of the present invention.

10, the image conversion system 990 according to the embodiment of the present invention may include some or all of the terminal apparatus 1000 and the image forming apparatus 1100. [

Here, to include some or all of them means that some components such as the image forming apparatus 1100 may be omitted, and the like will be described as including all in order to facilitate a sufficient understanding of the invention.

The terminal apparatus 1000 may include a desktop and a laptop computer, a mobile phone, and the like. Here, the terminal device 1000 can capture an image displayed on the display and convert the captured image, i.e., a screen shot, from the inside into a metafile, and store the converted image. Then, the stored metafile can be called up at the request of the user and output from the image forming apparatus 1100. Therefore, the data transmitted to the image forming apparatus 1100 is preferably metafile-related data.

On the other hand, when the terminal apparatus 1000 transmits the screen shot to the image forming apparatus 1100 in the form of a raster image, the image forming apparatus 1100 converts the received screen shot to generate a meta file, Can be stored. Concerning the conversion of the screen shot to the metafile, the description has been omitted since it has been explained sufficiently before.

11 is a block diagram showing a detailed structure of the terminal device of FIG.

10, a terminal apparatus 1000 according to an embodiment of the present invention includes a communication interface unit 1010, a user interface unit 1020, a control unit 1030, a storage unit 1040, Part 1050 of FIG.

The image generating unit 1050 includes part or all of the image forming apparatuses 1100 and 1150 in the case where the image forming apparatus 1100 includes the image generating unit 1160 in FIG. May be omitted, meaning that some elements are omitted or some elements are integrated into another element, and the description is made to cover all of them in order to facilitate a sufficient understanding of the invention.

The communication interface unit 1010 performs communication with the image forming apparatus 1100. For example, the communication interface unit 1010 may transmit the metafile of the screen shot generated by the image generation unit 1050 to the image forming apparatus 1100 under the control of the control unit 1030.

The user interface unit 1020 includes a button unit or a display unit. The user interface unit 1020 receives a user command for generating a screen shot. At this time, the user command can be inputted through the touch screen type display unit. In addition, the user interface unit 1020 can invoke a metafile of the screen shot stored in the storage unit 1040 and display the image on a screen or receive a command to transmit the metafile to the image forming apparatus 1100. [

The control unit 1030 controls the communication interface unit 1010, the user interface unit 1020, the storage unit 1040 and the image generation unit 1050 in the terminal device 1000 as a whole. For example, when there is a user command for generating a screen shot, the control unit 1030 converts a screen shot of the image displayed on the screen into a metafile, and stores the converted metafile in the storage unit 1040 do. If there is another user command, the stored metafile can be called up and displayed on the screen or transferred to the image forming apparatus 1100.

The storage unit 1040 stores the metafile for the screen shot generated by the image generation unit 1050 under the control of the control unit 1030. The controller 1030 outputs the stored metafile at the request of the controller 1030.

The image generation unit 1050 generates a metafile of the screen shot of the image displayed on the screen. This has been described in sufficient detail in the foregoing, so further explanation is omitted. In this case, the image generating unit 1050 may have the detailed configuration as shown in FIG. 3, for example, and may store and execute a program or algorithm for performing the same function as the functional blocks of FIG.

12 is a block diagram showing a detailed structure of the image forming apparatus of FIG.

12, an image forming apparatus 1100 according to an embodiment of the present invention includes a communication interface unit 1110, a user interface unit 1120, a storage unit 1130, a control unit 1140, Part 1150 and part or all of the image generation part 1160. [

Here, the image forming apparatus 1100 includes some or all of the image forming apparatuses 1100 in cooperation with the terminal apparatus 1000, and the terminal apparatus 1000 in FIG. 11 includes the image generating unit 1050, In the case where only the printing function is performed, the image generating unit 1160 may be omitted, and some of the components may be omitted, or some components such as the storage unit 1130 may be omitted from the image generating unit 1160 And the like, which are incorporated into the constituent elements, and the like, and the description is made to cover all of them in order to facilitate a sufficient understanding of the invention.

The communication interface unit 1110 performs communication with the terminal device 1000. [ For example, the communication interface unit 1110 may receive a metafile for a screen shot for delivery to the printing unit 1150 under the control of the control unit 1140. [ Or the communication interface unit 1110 may receive a raster image of a screen shot for generating a meta file in the image generating unit 1160. [ In this process, the communication interface unit 1110 may further perform an operation of decoding the encoded image data, for example.

The printing unit 1150 prints the image at the request of the user. For example, the printing unit 1150 preferably prints the meta file transferred under the control of the control unit 1140. [ At this time, the meta file may be provided in the terminal device 1000 or may be provided in the image generating unit 1160.

The user interface unit 1120, the storage unit 1130, the control unit 1140 and the image generation unit 1160 may be implemented by the user interface unit 1020, the control unit 1030, the storage unit 1040, (1050), it is intended to substitute the contents thereof.

The image generating method (or the driving method) of the terminal apparatus 1000 and the image forming apparatus 1100 according to the embodiment of the present invention is not shown in a separate drawing, So that further explanation will be omitted.

Further, the image generating method and system according to the embodiment of the present invention can be executed in a printer driver or support software for a black-and-white color printer and an MFP. Further, the method according to an embodiment of the present invention may be executed in the form of a software application for printing.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art. Such a computer program may be stored in a non-transitory computer readable medium readable by a computer, readable and executed by a computer, thereby implementing an embodiment of the present invention.

Here, the non-transitory readable recording medium is not a medium for storing data for a short time such as a register, a cache, a memory, etc., but means a medium that semi-permanently stores data and can be read by a device. Specifically, the above-described programs can be stored in non-volatile readable recording media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

While the invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

301: division unit 302: color evaluation unit
303: Text retaking unit 304: Vectorizing unit
305: Metafile generation unit 1000:
1100: Image forming apparatuses 1010 and 1110: Communication interface unit
1020, 1120: user interface unit 1030, 1140:
1040, and 1130: storage units 1050 and 1160:

Claims (18)

A partition that divides the text portion of a screen shot that contains graphics and text;
A color evaluation unit for estimating a color of the background excluding the text and the text in the divided text part;
A vectorization unit for vectorizing an outline of the text in the divided text part;
A text retouching unit for retouching the screen shot by filling the divided text with a color of the estimated background; And
And a metafile generator for generating a metafile using the estimated color of the text, vector information of the vectorized outline, and the modified screen shot,
The color evaluating unit may include:
And estimates a column located at the leftmost or rightmost column of the column having the largest value on the histogram of the RGB color components of each pixel as the color of the background. The method according to claim 1,
Wherein the dividing unit generates a map for a plurality of text parts, supplies map data for the generated map to the vectorizing unit,
Wherein the map for the plurality of text portions is a raster image of the same size. The method according to claim 1,
Wherein the dividing unit detects a plurality of pixels having a horizontal gradient value that is brighter and darker than an adjacent pixel with respect to an arbitrary pixel in the screen shot than the surrounding pixels and combines the detected pixels into respective parts, And detects the text portion according to an analysis result of analyzing a histogram of each portion. The method of claim 3,
Wherein the plurality of pixels are detected by selecting the end points of non-zero associated pixels in which the gradient of the gradient for the screen shot is present. The method of claim 3,
Wherein the dividing unit determines that the text portion is larger than the threshold value if the number of columns having a gray value of 0 is greater than the threshold value and determines that the portion is a graphic portion if the number is smaller than the threshold value. delete The method according to claim 1,
Wherein the vectorization unit converts the text portion into a halftone image to vectorize the outline of the text, binarizes the converted halftone image, tracks the outline of the symbols in the binarized image, And a polygon approximation to a series of refinement parts made up of curves. Splitting the text portion in a screen shot that includes graphics and text;
Estimating a color of the background excluding the text and the text in the divided text portion;
Vectorizing an outline of the text in the segmented text portion;
Correcting the screen shot by populating the divided text portion with the color of the estimated background; And
Generating a metafile using the estimated color of the text, vector information of the vectorized outline, and the modified screen shot,
Wherein the color estimating step comprises:
And estimates a column located at the leftmost or rightmost column of the column having the largest value on the histogram of the RGB color components of each pixel as the color of the background. 9. The method of claim 8,
Wherein the dividing step comprises:
Generating a map for a plurality of text parts and providing map data for the generated map for the vectorization,
Wherein the map for the plurality of text portions is a raster image of the same size. 9. The method of claim 8,
Wherein the dividing step comprises:
Detecting a plurality of pixels having a horizontal gradient value that is brighter and darker than an adjacent pixel with respect to an arbitrary pixel in the screen shot;
Combining the plurality of detected pixels into respective portions; And
Dividing the text portion according to an analysis result of analyzing a histogram of each portion;
The image processing method comprising the steps of: 11. The method of claim 10,
Wherein the plurality of pixels are detected by selecting the endpoints of non-zero associated pixels in which the gradient of the gradient for the screen shot is present. 11. The method of claim 10,
Wherein the dividing step determines that the text portion is larger than the threshold value if the number of columns having a tone value of 0 is greater than the threshold value and determines the portion as a graphic portion if the number is smaller than the threshold value. delete 9. The method of claim 8,
Wherein the vectorizing comprises:
Converting the text portion into a halftone image;
Binarizing the transformed halftone image;
Tracking an outline of symbols in the binarized image; And
Performing a polygon approximation on the traced contour as a series of refinement portions of straight lines and curves;
Including an image processing method. A computer-readable recording medium for executing an image processing method,
Splitting the text portion in a screen shot that includes graphics and text;
Estimating a color of the background excluding the text and the text in the divided text portion;
Vectorizing an outline of the text in the segmented text portion;
Correcting the screen shot by populating the divided text portion with the color of the estimated background; And
Generating a metafile using the color of the estimated text, vector information of the vectorized outline, and the modified screen shot,
Wherein the color estimating step comprises:
And estimates the color of the background located at the leftmost or rightmost column of the column having the largest value on the histogram of the RGB color components of each pixel. Computer readable recording medium. 1. An image forming apparatus connected to a peripheral terminal apparatus,
A communication interface for receiving a screen shot including graphics and text from the terminal device;
Estimating a color of a background excluding the text and the text in the divided text part, vectorizing the outline of the text in the divided text part to generate vector information An image generating unit for modifying the screen shot by filling the divided text portion with the estimated background color, and generating a metafile using the modified screen shot and the text color of the text portion; And
And a printing unit printing the generated metafile according to a user's request
The image generation unit may include:
And estimates a column located at the leftmost or rightmost column of the column having the largest value on the histogram of the RGB color components of each pixel as the color of the background. delete delete

Images