JP2011054123A - Image processing apparatus, image processing method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problems: when two or more kinds of image processing are performed on an input image, it is preferable to generate an image suitable for each image processing, while a large memory capacity is needed to simultaneously store the images. <P>SOLUTION: When two or more kinds of image processing (area analysis, character section vectorization, pattern section vectorization, etc.) are performed on the input image, an image suitable for each image processing is generated prior to execution of the image processing. The prossing result of the image processing is stored in area information, and the next image processing is performed after the work memory used in each image processing is released. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method for generating an electronic file by performing region analysis processing and vectorization processing on an input image.

  In recent years, the digitization of information has progressed, and systems that do not store paper documents as they are, scan them with a scanner or the like, store them electronically, and send the electronic data to other devices have become widespread. In order to reduce the amount of data when electronic data is transmitted, high compressibility is required for digitized documents. On the other hand, high image quality of electronic data is also required.

  Therefore, in Patent Document 1, first, a character region, a line drawing region, a photographic region, and the like are extracted by performing region analysis processing (region identification processing) on the scanned image. Then, the extracted character area or line drawing area is converted into vector data, and other areas not suitable for vectorization (such as a photographic area (natural image)) are compressed by JPEG. A method for improving the high compressibility, reusability, and high image quality of a generated file by combining the compression results (vector data, JPEG compressed data, etc.) of each area into a single file and outputting it. Proposed.

JP 2007-272601 A

  It is conceivable to perform vectorization processing (vector conversion processing) for both the character area and the drawing area (such as an area such as an illustration with a limited number of appearing colors or a line drawing area). The resolution of images suitable for vectorization processing is different. That is, since characters are often more complex and smaller than a graphic area, a high resolution image is required for an image when performing vector processing of the character area. On the other hand, in the case of a drawing area, color information is often more important than resolution. Therefore, when performing the vectorization process of the graphic area, an image with a lower resolution (for example, an image with a medium resolution) than that when the character area is vectorized may be used, but a color image is required. In addition, since it is desired that the area analysis process performed before the vectorization process be performed at high speed, an image having a multi-value medium resolution or less reduced to N values may be used.

  However, if three types of images suitable for the image processing of the region analysis processing, the character region vectorization processing, and the drawing region vectorization processing are simultaneously held, a large amount of memory is required. .

  A specific example is given. For example, if a 600 dpi multi-valued image is input, the YCbCr image is 300 dpi for analysis processing, the 600 dpi grayscale image is used for vectorization processing of character areas, and the pixel ratio is 300 dpi for drawing vectors and backgrounds. It is assumed that Y: Cb: Cr = 4: 1: 1 images are converted and held. In this case, all images cannot be held unless a memory twice as large as the original image is secured.

  Furthermore, when performing vectorization processing and hole filling processing suitable for each area in the page, it is necessary to secure work memory necessary for each processing. It is also necessary to store the results of individual conversion processes in the memory until the electronic file that is the final output product is generated.

  If an attempt is made to individually hold these work memories and memories for holding various data, a large amount of memory is required.

  In the present invention, fewer images are generated when an electronic file is generated by generating an image suitable for each of the area analysis processing, the vectorization processing of the character portion, and the vectorization processing of the drawing portion and executing each processing. Executable with memory capacity.

  In order to solve the above problems, an image processing apparatus of the present invention generates a memory for storing an input image, a region analysis image suitable for region analysis processing from the input image, and generates the generated region analysis image. , Area analysis image generating means for storing in a first memory area secured on the memory, area analysis processing work memory securing means for securing area analysis processing work memory on the memory, and the secured area A processing result holding memory area for holding processing results secured on the memory by executing area analysis processing on the area analysis image stored in the first memory area using the analysis processing work memory A region analysis unit for storing a result of the executed region analysis process, and the first memory region secured in the memory at the end of the process by the region analysis unit First memory releasing means for releasing the area analysis processing work memory, a character part vectorization processing image suitable for character part vectorization processing is generated from the input image, and the generated character part vectorization processing The character portion vectorization processing image generation means for storing the image for use in the second memory area secured on the memory after being released by the first memory releasing means, and the first memory releasing means being released Using the character portion vectorization processing work memory securing means for securing the character portion vectorization processing work memory on the later memory and the secured character portion vectorization processing work memory, the processing result holding memory Executing character part vectorization processing on the character part vectorization processing image stored in the second memory area based on the result of the area analysis processing held; Character part vectorization processing means for storing the result of the executed character part vectorization processing in the processing result holding memory, and the memory secured at the end of the processing by the character part vectorization processing means A second memory releasing unit for releasing a second memory area and the character part vectorization processing work memory; and a graphic part vectorization processing image suitable for the vectorization process of the graphic part from the input image. The graphic part vectorization processing image generation means for storing the generated graphic part vectorization processing image in a third memory area secured on the memory after being released by the second memory releasing means; and Character memory portion filling processing work memory securing means for securing the character portion filling processing work memory on the memory released by the second memory releasing means, and the secured character portion Using the work memory for hole filling processing, the character portion is filled in the graphic part vectorization processing image stored in the third memory area based on the result of the area analysis processing held in the processing result holding memory. Character portion filling processing means for executing processing, and third memory releasing means for releasing the character portion filling processing work memory secured on the memory at the end of processing by the character portion filling processing means; Work memory securing means for drawing part vectorization processing for securing the drawing part vectorization and filling process work memory on the memory released by the third memory release means, and the secured drawing part vectorization and filling process And the third memory area based on the result of the area analysis process held in the process result holding memory. Graphic part vectorization means for executing a graphic part vectorization process on the graphic part vectorization process image stored in the image data, and storing the result of the graphic part vectorization process executed in the processing result holding memory; For the graphic part vectorization processing image after the character part filling process stored in the third memory area using the secured graphic part vectorization and filling process work memory, the graphic part The graphic part hole filling processing means for executing the hole filling process, and a fourth memory for releasing the graphic part vectorization and hole filling processing work memory secured on the memory at the end of the processing by the graphic part hole filling processing means Using the release means and the memory released by the fourth memory release means, the character part vectorization held in the processing result holding memory An electronic file including the result of the image processing, the result of the graphic part vectorization processing, and the graphic part vectorization processing image after the character part and the graphic part held in the third memory area are subjected to the filling process. Electronic file generating means for generating.

  In order to solve the above problems, an image processing apparatus according to the present invention generates an image suitable for image processing from a memory for storing an input image and the input image, and secures the generated image on the memory. Image generation means for storing in the memory area, work memory securing means for securing the work memory for image processing on the memory, and image processing stored in the memory area using the secured work memory Image processing means for executing image processing on a suitable image and storing the result of the executed image processing in a memory area for retaining the processing result secured in the memory; and image processing of the image processing means A memory release means for releasing the memory area secured on the memory and the work memory for image processing at the end of For each type of image processing, processing by the image generation unit, the work memory securing unit, the image processing unit, and the memory release unit is executed, and the plurality of types of images are processed from the processing result holding memory area. Electronic file generation means for generating an electronic file including the processing results of each process is further provided.

  According to the present invention, an electronic file can be generated by executing a plurality of processes with less memory.

System configuration example Block diagram for explaining the processing configuration (A) Flow chart of region analysis processing, (B) Flow chart of attribute determination Explanatory drawing when detecting corners of contour line Explanatory drawing when generating one contour line from the outer contour and inner contour Flowchart of graphic part vectorization processing Block diagram for explaining the clip area vectorization process Flow chart of clip art filling process Example of page drawing information Electronic file example The figure which shows the processing procedure of each processing section, and the internal state of the memory at that time Example of a document image that includes a clip art area

Example 1
FIG. 1 is an example of a diagram showing a system configuration using the first embodiment.

  The image processing apparatus 100 is a configuration example of an apparatus for realizing the present invention, and includes a scanner 101, a CPU 102, a memory 103, a hard disk 104, and a network I / F 105. The scanner 101 reads the page information of the document and converts it into image data. The CPU 102 functions as a device that performs image processing according to the present invention on image data by executing a computer program. The memory 103 is used for temporary storage of work memory and data when executing the program. The hard disk 104 stores the program and data. The network I / F 105 performs data input / output with an external device.

  A personal computer (PC) 120 is connected to the image processing apparatus 100 via a network such as the LAN 110 and receives data transmitted from the image processing apparatus 100.

  FIG. 2 is a block diagram illustrating a configuration of processing executed in the first embodiment. In FIG. 2, processing units that execute each process are indicated by S <b> 10 to S <b> 100, and data to be processed and processing result data are indicated by 201-211. In the present embodiment, the computer program is executed by the CPU 102 to cause the computer to function as the processing units (processing units) S10 to S100 in FIG. 2, but part or all of the processing units S10 to S100 may be used. You may make it comprise with an electric circuit. Note that the computer program is stored in a computer-readable storage medium. In addition, a work memory necessary for holding the data 201 to 211 and a work memory necessary for executing the processing units S <b> 10 to S <b> 100 are secured in the memory 103. The present invention is characterized by the execution order of each image processing performed in each processing unit and the timing of securing and releasing the memory in each image processing, the details of which will be described later.

  First, details of processing executed by each processing unit in FIG. 2 will be described.

  The area analysis image generation unit S10, the character part vectorization processing image generation unit S20, and the graphic part vectorization processing image generation unit S30 perform resolution and color space conversion of the input image 201 input from the scanner, and perform subsequent stages. Each image 202, 203, 204 suitable for each process is generated. A known method is used for resolution conversion and color space conversion.

  In the region analysis image generation unit S10, the input image 201 is converted into an image suitable for processing in the subsequent region analysis unit S40 (region analysis image 202). A multi-value medium resolution image is suitable for the processing of the region analysis unit S40. Therefore, the area analysis image generation unit S10 converts the input image 201 (for example, color space RGB, resolution 600 dpi) into an image of the color space YCbCr / resolution 300 dpi, and generates the area analysis image 202.

  In the character part vectorization processing image generation unit S20, the input image 201 is converted into an image (character part vectorization processing image 203) suitable for the processing of the subsequent character part vectorization processing unit S50. Since the character size is generally smaller than the page size, a high-resolution image is suitable for the processing of the character vectorization processing unit S50. Therefore, the character part vectorization processing image generation unit S20 converts the luminance information Y from the input image 201 into a gray scale image having a resolution of 600 dpi, and generates a character vectorization processing image 203.

  The graphic part vectorization processing image generation unit S30 converts the input image 201 into an image (graphic part vectorization processing image 204) suitable for the processing of the graphic part vectorization processing part S70 in the subsequent stage. When vectorizing clip art images such as illustrations with a small number of colors and line art images such as line drawings, the color information is important because it is divided into areas for each color. Color and medium-resolution images are suitable for the processing of the part S70. Therefore, in the graphic part vectorization processing image generation unit S30, the input image 201 is converted into an image having a resolution of 300 dpi in a pixel ratio of Y: Cb: Cr = 4: 1: 1, and graphic part vectorization processing is performed. A work image 204 is generated. That is, an image is generated in which the resolution of the color difference information Cb and Cr is half that of the luminance information Y (an image in which the luminance information and the color difference information have different resolutions).

  The area analysis processing unit S40 analyzes the area analysis image 202, determines areas such as a character part and a graphic part, and divides them into respective areas. Here, it is possible to use a known color image dividing technique. In this embodiment, the technique described in Patent Document WO2006 / 066325 is used. FIG. 3A is a flowchart of the area analysis processing executed by the area analysis processing unit S40.

  In step S401, the region analysis image 202 to be processed is color quantized, and regions having similar colors are integrated.

  In step S402, the attribute of each area obtained by color quantization is determined. In this embodiment, attributes such as characters, line drawings, photographs, tables, frames, and page backgrounds are determined. FIG. 3B is a flow showing details of attribute determination in step S402.

  In step S4021, each region obtained by color quantization is converted into a pixel connected component (color connected component) for each color, and a statistical value of each connected component is obtained. The pixel connection component is a group that is created for each pixel group that has the same color (similar color) and is connected (contacted). For the statistical value, the position and size of the circumscribed rectangle of each connected component, the number of pixels of the connected component, the length of the boundary line, the average color, and the like are used.

  In step S4022, the color connected components are classified based on the respective statistical values, and attribute determination is performed.

  In step S4023, based on the position information of the color connected components, the inclusive relation is examined to create a tree structure, and grouping is performed based on the tree structure. For example, when there is a character attribute color connected component and there is a similar character attribute connected component in the vicinity, it is determined that a character string or paragraph is to be formed and grouped into one character region. When grouped, the tree structure is updated.

  In step S4024, information on the color connected components after grouping is generated, a memory for holding the region information 205 is secured, and information on the color connected components is stored as part of the region information 205. The information related to the color connected component stored in the area information 205 includes attribute information such as characters, line drawings, photographs, tables, frames, and page backgrounds, and positional information of each connected component. It is represented by a tree structure. The processing result of each area performed after the vectorization process or the like is recorded in the area information 205.

  In the character part vectorization processing unit S50, the character part (character image in the text or character image in the table) is vector-converted by performing character vectorization processing using the character part vectorization processing image 203. To do. Details of the character vector processing will be described below.

  Based on the color connected component having the character attribute included in the region information 205, the position information of the character region is acquired, and in the character part vectorization processing image 203, the grayscale connected component in the corresponding region is analyzed, Get connected components of high resolution characters. Then, the outline of the connected component of each character is converted into vector data. For each character region, the grayscale character part vectorization processing image 203 may be binarized, and then a connected component of the character may be extracted and the outline of the connected component may be converted into vector data.

  For example, the following method can be used as the method for converting the outline of the connected component of characters into vector data. In order to express the outline of the connected component of characters as a combination of straight lines and / or curves, “corners” serving as reference positions for dividing the outline into a plurality of sections (pixel columns) are detected. A corner is a point at which the curvature is maximized. For example, whether or not the pixel Pi on the contour line in FIG. 4 is a corner is determined as follows. That is, a line segment L connects pixels Pi−k and Pi + k that are separated from Pi by a predetermined number of pixels (here, k pixels) in both directions along the contour line. A distance between the pixels Pi-k and Pi + k is d1, a distance between the line segment L and the pixel Pi is d2, and an arc length between the pixels Pi-k and Pi + k is A. When d2 reaches a maximum or when the ratio (d1 / A) is equal to or less than the threshold, the pixel Pi is determined to be a corner. Then, the contour pixel row is divided at the determined corner position, and each divided pixel row is approximated by a straight line or a curve. The approximation to a straight line is executed by the least square method or the like, and the approximation to a curve uses a cubic spline function or the like. The pixel at the corner that divides the pixel row is the approximate line or the start or end of the approximate line. Further, it is determined whether or not the contour of the pixel block having a luminance different from that of the pixels constituting the contour (inner contour) exists in the vectorized contour, and if the inner contour exists, the contour is vectorized. . Further, the inner contour of the inverted pixel is recursively vectorized, such as the inner contour of the inner contour. In this way, the outline of the character portion can be vectorized by using the lane marking approximation of the contour of the connected component.

  As shown in FIG. 5, when the outer contour PRj and the inner contour PRj + 1 or another outer contour are close to each other at a certain interval in a certain attention section, two or a plurality of contour lines are grouped together and the thickness is set. You may make it express as vector data of a held line. For example, the distance PQi from each pixel Pi of the contour PRj + 1 to the pixel Qi that is the shortest distance on the contour PRj is calculated. When the variation in the distance PQi-1 to the distance PQi + 2 for the target section (for example, between Pi-1 and Pi + 2) is slight, a straight line or a curve along the point sequence of the midpoint Mi of the pixels Pi and Qi And can be expressed as a line having a thickness. Note that the thickness of the approximate line and the approximate curve is, for example, the average value of the distances PQi in the target section.

  In this manner, the character part vectorization processing unit S50 generates vector data from the outline of the character connected component, generates the character vector 206 together with the color information of the color connected component, and records it in the area information 205. .

  In the character portion filling processing unit S60, the character region in the drawing portion vectorization processing image 204 is subjected to filling processing (processing for filling character pixels with a background color). In this embodiment, a method of painting the character pixel portion with the average color of the surrounding pixels of the character pixel is used.

  First, the information of the color connected component having the character attribute is acquired from the area information 205 and developed as an image in the character hole filling work memory. The pixel position of the image developed in the character hole filling work memory and the drawing portion vectorization processing image 204 are overlapped to specify the character pixel portion in the drawing portion vectorization processing image 204. Next, in the graphic part vectorization processing image 204, the average color of the peripheral pixels of each character pixel portion (that is, the average color of the background of each character) is calculated. Finally, the character pixel portion of the graphic part vectorization processing image 204 is filled with the calculated average color. As a result, a graphic part vectorization processing image in which the character vectorized area is filled with the background color and the character image portion is erased is generated.

  In the graphic part vectorization processing unit S70, the graphic part is vector-converted by using the graphic part vectorization processing image after the character part filling process is performed. The graphic part vectorization processing executed by the graphic part vectorization processing unit S70 will be described with reference to FIG.

  In step S701, based on the area information of the drawing area (line drawing, clip art such as an illustration with a small number of colors, table ruled line, and photograph), it is determined whether the drawing vectorized image of the corresponding area can be vector-processed. If it is determined that processing is possible, the process proceeds to step S702, and if not, the process proceeds to step S703. For example, referring to the attribute of the area information, if it is a line drawing, a clip art such as an illustration with a small number of colors, or a table, it is determined that vectorization processing is possible, and a natural image such as a photograph is determined to be unsuitable for vectorization processing. Also, when the color quantization result in the region to be processed is determined to be excessively divided (when the number of color connected components is excessive), or when the area of the region information is greater than a certain value, Considering the processing speed, it may be determined that the region is not suitable for vectorization processing.

  In step S <b> 702, the drawing unit vectorization process is executed to generate a drawing vector 207. In addition, when vectorizing a line drawing or table ruled line, it is possible to approximate the contour line in the same way as the character part, and the vectorization process can be performed. In particular, the table ruled line which is a collection of lines and lines is thick. By expressing as a set of lines having, vector expression can be efficiently performed. Further, as a technique for vectorizing a line drawing (line art), a technique disclosed in Japanese Unexamined Patent Application Publication No. 2007-293829 can also be used. Further, as a technique for vectorizing a clip art region such as an illustration, a method disclosed in Japanese Patent Application Laid-Open No. 2006-334069 can be used.

  The details of the vectorization process of the clipart area in this embodiment will be described with reference to FIG. S7021 to S7024 in FIG. 7 are detailed processing units constituting the graphic part vectorization processing unit S70 that executes the vectorization processing S702. The data 212 to 215 are recorded on the memory 103 secured as a work memory for drawing part vectorization and hole filling.

  The resolution enhancement processing unit S7021 converts a YCbCr image having a coarse resolution of at least one of the plurality of color components into an RGB image that matches the resolution of the highest resolution color component. In the present embodiment, the drawing part vectorization processing image 204 is an image of Y: Cb: Cr = 4: 1: 1 (an image in which the resolution of the color components of Cb and Cr is lower than the Y component). By converting to the image 212, the resolution is returned to the same resolution as that of the Y color component.

  In the clipart area dividing unit S7022, a rectangular clipart area is extracted from the RGB image 212 based on the position information of the color connected component stored in the area information 205, and each pixel in the clipart area is color-characterized. Based on the above, it is divided into areas for each color.

  The clip art area background identification unit S7023 identifies the background area information 214 in the clip art area and the background color 213 in the clip art based on the area division result of the clip art area dividing unit S7022. For example, among the area division results of the clip art area dividing unit, an area having a large number of pixels in contact with the circumscribed rectangle boundary of the clip art area is identified as the background area in the clip art area. The identified background area information 214 and the background color 213 in the clip art are held as intermediate information 215. The intermediate information 215 holds various types of information generated during other clip art vectorization processing (not shown).

  Based on the identification result of the background region identification unit 7023, the clip art vectorization unit S7024 vectorizes each region portion other than the background region in the region division result of the clip region division unit S7022, and draws the graphic region vector of the clip region. 207 is generated.

  The background area information 214 and the background color 213 in the clip art held as the intermediate information 215 are also used in the drawing part hole filling processing part S80 described later.

  Returning to the description of FIG. In step S703, a raster image is cut out for an area such as a photograph that is not suitable for vectorization. Here, a corresponding region is cut out from the graphic portion vectorization processing image 204 after the character portion is filled, and a rectangular photograph image is generated.

  The drawing vector 207 generated in step S702 and the rectangular photograph image 208 generated in step S703 are recorded in the area information 205.

  The graphic part hole filling processing unit S80 performs a hole filling process for each graphic area to generate a base image 209. The graphic portion filling processing unit S80 switches the filling processing method for each region depending on whether a graphic portion vector or a rectangular photograph image is generated in the output result of the previous drawing portion vectorization processing portion S70. .

  In the case of a drawing area in which a rectangular photographic image is generated, the average color of surrounding pixels in the area is calculated, and the corresponding drawing area in the drawing portion vectorization processing image 204 is filled with the calculated average color.

  Hereinafter, the details of the hole filling process when the drawing area is the clip art area will be described. FIG. 12A shows an example of a document image including a clip art area. FIG. 12B shows an object circumscribing rectangle 81 that is a circumscribed rectangle extracted from a clip art region as a vectorization target. Further, in the background area information 214 in the clipart area held in the intermediate information 215, an area having a large number of pixels in contact with the circumscribed rectangle boundary of the clipart area is identified as the background area 82 in the clipart area. . FIG. 12C is a diagram in which the background region 83 is excluded from the vectorization target from the clip art region of FIG. 12B and only the region other than the background (the vectorization target object 87) is expressed. In FIG. 12C, a boundary 85 indicates a boundary portion (hereinafter referred to as “boundary portion 1”) between the object region (region other than the background) and the base 84. A boundary 86 indicates a boundary portion (hereinafter also referred to as “boundary portion 2”) between the object region (region other than the background) and the background in the clipart region. Next, the clip art hole filling process will be described in detail with reference to the flowchart of FIG.

  In step S1101, based on the background area in the clip area to be processed, information for determining the background area is converted into a binary bitmap image (background identification) that has been increased in resolution and generated with 1 bit per pixel. Image). The background area and the object area are generated in a format that can be discriminated based on whether the pixel value is 1 or 0 and held in the work memory. At that time, a series of initialization processing is performed as preparation for processing the image data corresponding to the clipart area in the raster scan order. When the series of initialization processes is completed, the hole filling process for the portion corresponding to the clipart area is started in the raster scanning order.

  In step S1102, raster scanning is performed on the Cr data (or Cb data) of the portion of the image data corresponding to the clip area in the graphic part vectorization processing image 204. It is determined whether or not the pixel at the target position in the raster scanning order at this time is a pixel in the background region that is not the boundary 1 and not the boundary 2 described above. Specifically, the pixel at the target position in the raster scanning order at this time is 2 × 2 pixels of the corresponding portion (a region of 2 × 2 pixels corresponds) on the bitmap image (background identification image) described above. It is checked whether or not all of the four pixels are background area pixels. If all four pixels are background pixels, it is determined that the background area is other than the boundary.

  As a result of the determination, if the target pixel is a pixel in the background area other than the boundary, the process proceeds to step S1107. If the target pixel is not a pixel in the background area other than the boundary, the process proceeds to step S1103.

  In step S1103, as in step S1102, it is determined whether or not the target pixel is a pixel corresponding to the boundary 2 (that is, a boundary between the background area and the object area (an area other than the background in the clipart area)). To do. In other words, the pixel at the current position in the raster scan order at that time is the background region, and any one of the corresponding 2 × 2 pixels on the bitmap image (background identification image) is the background region. It is checked whether all the remaining pixels in the four pixels are pixels in the object area. When this condition is satisfied, it is determined that the target pixel is a pixel corresponding to the boundary portion 2 (that is, the boundary between the background area and the object area).

  As a result of the determination, if the target pixel corresponds to the boundary between the background area and the object area, the process proceeds to step S1108. If the target pixel does not correspond to the boundary between the background area and the object area, the process proceeds to step S1104.

  In step S1104, as in the determination process described above, it is determined whether the target pixel is a pixel corresponding to the boundary portion 1 (that is, the boundary between the background region and the object region). That is, the pixel of interest is an object region among the 4 pixels of 2 × 2 pixels of the corresponding portion (the region of 2 × 2 pixels corresponds) on the bitmap image (background identification image) described above. Whether any of the remaining pixels in the four pixels is a pixel in the base region is checked. When this condition is satisfied, it is determined that the target pixel is a pixel corresponding to the boundary portion 1 (that is, the boundary between the background area and the object area).

  As a result of the determination, if the target pixel corresponds to the boundary between the background area and the object area, the process proceeds to step S1109. If the target pixel does not correspond to the boundary between the background area and the object area, the process proceeds to step S1105. In step S1105, since the target pixel is a pixel in the object region (the object body serving as the foreground), the background color (Y, Cb, Cr) is filled. That is, the Cb data, Cr data, and Y data of the pixel at the target position in the raster scanning order at that time are converted into the values obtained from the background color data on the image data that has been reduced in resolution and stored in the RAM 30. replace. Here, since the background color data in the clip art area is held as RGB data, it is obtained by converting it into YCbCr.

  Next, in step S1106, it is determined whether or not the processing in the raster scan order of the image data corresponding to the clipart area has advanced to the last pixel of the area. As a result of the determination, if the process has been completed up to the last pixel, the series of processes is completed, and if not completed, the process proceeds to step S1107. In step S1107, the pixel at the position of interest (target pixel) is advanced to the pixel at the next position, and the process returns to step S1102.

  On the other hand, in step S1108, since the target pixel is a pixel corresponding to the boundary portion 2 (that is, the boundary between the background region and the object region), only Cb and Cr of the target pixel are changed to the values obtained from the background color data. replace. That is, the boundary 2 is filled with the background color data. Here, since the background color data is held as RGB data, it is obtained by converting it into YCbCr. Then, when step S1108 is completed, the process proceeds to step S1106 described above.

  On the other hand, in step S1109, since the target pixel is a pixel corresponding to the boundary portion 1 (that is, the boundary between the background region and the object region), Y, Cb, and Cr of the target pixel are left as they are, and the background color is changed. Do not replace with data. That is, the boundary portion 1 is filled with the color of the base region. Then, when step S1109 is completed, the process proceeds to step S1106 described above.

  By configuring in this way, even if the object to be vectorized of the figure in the page of the color document and the background outside the circumscribed rectangular area including it are composed of data with different resolutions, a false color is displayed at the boundary part. Does not occur. The boundary portion is a rectangular boundary portion between the vectorization target object and its circumscribed rectangle area, or a boundary portion between the vectorization target object and the background region other than the vectorization target object within its circumscribed rectangle.

  In the page drawing information generation processing unit S90, the data and area information generated by each processing unit are combined into one. An example of data generated by the page drawing information generation process is shown in FIG. Reference numeral 2100 denotes an example of the overall configuration of the page drawing information 210 generated by the page drawing information generation processing unit S90. The text object 2101 holds the vector conversion result (character vector 206) of the area determined as the character by the area analysis unit S40 and the position information in the image. Similarly, the rectangular photographic image 208 and its position information are held in the photographic object 2102, and the vector conversion result (drawing vector 207) of the clip art or line art (line art) and its positional information are held in the drawing object 2103. The The background object 2104 holds an image after the hole filling process (background image 209) and its position information. The property 2105 holds bibliographic information (for example, the number of pages, page size, input device information, time stamp, etc.) of input data.

  In the electronic file generation unit S100, the page drawing information 210 is converted into an electronic file 211. In this embodiment, the data is converted into an electronic file that can be played or edited by the PC 120.

  An electronic document 2100 in FIG. 10 is an example of an electronic file 211 created in accordance with a virtual XML format created for explaining the present embodiment. In FIG. 10, 2111 is an example of description which draws the background image data of the background object 2104 of FIG. Reference numeral 2112 denotes a description for drawing a photographic image in the photographic object 2102 at a position specified by the position information. Reference numeral 2113 denotes a description for drawing a drawing vector in the drawing object 2103 at a specified position in the page. Reference numeral 2114 is a description for drawing a character vector in the character object 2101 at a specified position in the page. In the PC that has received the electronic document in FIG. 10, the information described in 2111 to 2114 is drawn in order, and the background, the photograph, the drawing vector, and the character vector are superimposed so that the page has the same appearance as the original document. The image can be played back. Further, it is also possible to use each of the descriptions 211 to 2114 as an object for editing. Note that FIG. 10 is merely an example, and may be described using other formats. For example, graphic page descriptions such as PDF, XPS, and SVG, page edit data descriptions such as Office Open XML, and Open Document Format may be used. The type of format to be converted may be specified by the user in advance, or may be specified by inquiring the user during the conversion process.

  Details of the execution order of the processes in the present invention and the timing of securing and releasing the memory in each process will be described. FIG. 11 is a schematic diagram showing the processing procedure of each processing unit in FIG. 2 and how the internal state of the memory 103 is changed for each processing order. That is, FIG. 11A shows the processing order of each processing unit in the present invention, and FIG. 11B shows the internal behavior of the memory 103 (usage status of the memory 103) when performing each processing.

  In FIG. 11B, the vertical axis indicates the time axis according to the processing order, and the horizontal axis indicates the memory address (memory usage status). Each rectangle in FIG. 11B indicates the timing and memory address at which the memory area used as various data and work memory is secured in the memory 103. The upper and lower ends of the rectangle indicate the timing for securing and releasing the memory, and this timing is described so as to match the processing timing in each processing step of FIG. 11A on the horizontal line at the upper and lower ends of the rectangle. ing. As a specific example, the memory area in which the input image 201 is stored is released in the state shown in FIG. 11A while being secured in the memory 103 and is released at the end of the process of the page drawing information generation unit S90. .

  Details of FIG. 11 will be described below. In an initial state at the start, an input image 201 input via a scanner or a network is stored in an input image holding memory area secured in the 103 memory.

  First, as shown in FIG. 11A, a region analysis image generation process is performed by the region analysis image generation unit S10. When this processing is executed, the area analysis processing work memory 216 is secured on the memory 103, and a memory area for storing the analysis processing image 202 is secured, and the analysis area generated from the input image 201 is secured. An image 202 is stored in the memory area.

  Next, the area analysis unit S40 executes the area analysis process by using the area memory for analysis processing (analysis process memory area) 216 secured as described above and using the area analysis image 202 as a processing target image. Here, a memory area (memory area for holding the processing result) of the area information 205 is secured, and information relating to the color connected component 212 (including attribute information and position information of each color connected component) as the area analysis result is area information. Saved in 205 (processing result holding memory area). At the end of the analysis processing of the region analysis unit S40, the memory region in which the region analysis image 202 is held and the analysis processing work memory 216 are released (memory release).

  Next, a character part vectorization processing image generation process is performed by the character part vectorization processing image generation unit S20. When this processing is executed, the character part vectorization processing work memory 217 is secured on the memory 103, and a memory area for storing the character part vectorization processing image 203 is secured. Then, the character part vectorization processing image 203 generated from the input image 201 is stored in the reserved memory area for image holding. In addition, as shown in FIG. 11B, these memory areas are secured in a memory area that is released after the processing of the area analysis processing unit S40.

  Next, the character part vectorization processing unit S50 executes character part vectorization processing using the character part vectorization processing work image as a processing target image using the reserved character part vectorization processing work memory. To do. This character part vectorization processing is executed for the character part vectorization processing image at a position corresponding to the character region of the region analysis result held in the region information 205. Details of the processing are as described above. The character vector 206 of each character area, which is the result of the vectorization processing of the character part, is recorded in the area information 205. At the end of the vectorization process of the character part vectorization processing unit S50, the memory area in which the character part vectorization processing image 203 is held and the character part vectorization processing work memory 217 are released.

  Next, the graphic part vectorization processing image generation process is performed by the graphic part vectorization processing image generation unit S30. When this process is executed, a memory area for storing the graphic part vectorization processing image 204 is secured on the memory 103, and the graphic part vectorization processing image 204 generated from the input image 201 is stored in the memory 103. Saved in the area. As shown in FIG. 11B, this memory area is secured in a memory area that is released after the processing by the character part vectorization processing unit S50.

  Next, the character portion filling processing unit S60 secures the character portion filling processing work memory 218 in the memory 103, and executes the character portion filling processing. Here, as described above, information relating to pixel connection components (color connection components of character attributes) in the character region is extracted from the region information 205 and developed as an image in the character hole filling work memory 218. Then, the character image developed in the character hole filling work memory 218 and the graphic part vectorization processing image 204 are overlapped to specify a character pixel portion in the graphic part vectorization processing image 204. Next, the average color of the surrounding pixels of the character pixel portion of the graphic portion vectorization processing image 204 is calculated, and the character pixel portion of the graphic portion vectorization processing image 204 is filled with the calculated average color. As described above, when the processing of the character part filling process section S60 is completed, the graphic part vectorization processing image 204 is an image composed of the graphic part area and the page background, with the character area filled. That is, the graphic part vectorization processing image 204 after the character part is filled is stored in the memory area for storing the graphic part vectorization processing image. At the end of the processing of the character portion filling processing unit S60, the character hole filling work memory 218 is released.

  Next, the graphic part vectorization processing unit S70 secures a work memory 219 for graphic part vectorization and hole filling processing in the memory 103, and executes the graphic part vectorization process. This graphic part vectorization processing is executed for the graphic part vectorization processing image at a position corresponding to the graphic area of the area analysis result held in the area information 205. Details of the processing are as described above. The graphic part vector 207 and the rectangular photographic image 208 obtained as a result of the vectorization processing of the graphic part are recorded in the area information 205. Also, the data 212 to 215 shown in FIG. 7 are recorded in the work memory 219 for the graphic part vectorization process.

  Next, the graphic part filling process is performed in the graphic part filling process part S80. Here, as described with reference to FIGS. 7 to 8, the graphic part filling process is performed on the graphic part vector processing image 204 after the character part filling process. Therefore, when the process of the graphic part hole filling processing unit S80 is completed, the graphic part vector processing image 204 becomes an image composed only of the page background, and is used as the base image 209 in FIG. Also, at the end of the process of the graphic part hole filling processing unit S80, the graphic part vectorization and hole filling work memory 219 on the memory 103 is released.

  Next, the page drawing information generation unit S90 secures a memory area for storing the page drawing information 210 in the memory 103, generates page drawing information, and stores it in the memory area. Here, the character vector 206, the drawing vector 207, and the rectangular photograph image 208 stored in the area information 205 are acquired, and the background image 209 is obtained from the drawing portion vectorization processing image 204 in which the character portion and the drawing portion are filled. Acquire and generate page drawing information using bibliographic information of input data as a property. At the end of the process of the page drawing information generation unit S90, the memory area of the graphic part vector processing image 204, the memory area of the area information 205, and the memory area of the input image 201 on the memory 103 are released.

  Next, in the electronic file generation unit S100, a memory area for storing the electronic file 211 is secured, and as described above, the page drawing information 210 is converted to generate the electronic file 211 in a predesignated format. The memory area for storing the electronic file 211 is secured in the memory area released at the end of the process of the page drawing information generation unit S90. At the end of the processing of the electronic file generation unit S100, the memory area in which the page drawing information 210 on the memory 103 is held is released.

  As described above, in the present embodiment, an image suitable for the region analysis process, an image suitable for the character part vectorization process, and an image suitable for the graphic part vectorization process are provided for each of a plurality of types of image processing. Generate immediately before. Each processing result is recorded in separately secured area information 205, so that each processing image and processing work memory that are no longer required when the processing is completed are opened. As a result, the electronic file can be generated with a smaller memory capacity.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (11)

Memory to store input images;
A region analysis image generating unit that generates a region analysis image suitable for region analysis processing from the input image, and stores the generated region analysis image in a first memory region secured on the memory;
Area analysis processing work memory securing means for securing area analysis processing work memory on the memory;
The area analysis processing is executed on the area analysis image stored in the first memory area by using the reserved area analysis processing work memory, and the processing result for holding the processing result secured on the memory is obtained. Area analysis means for storing the result of the executed area analysis processing in the holding memory area;
First memory releasing means for releasing the first memory area secured on the memory and the work memory for area analysis processing at the end of processing by the area analyzing means;
A character portion vectorization processing image suitable for character portion vectorization processing is generated from the input image, and the generated character portion vectorization processing image is released on the memory after being released by the first memory releasing means. An image generation means for character vectorization processing stored in the second memory area secured in
Character part vectorization processing work memory securing means for securing character part vectorization processing work memory on the memory released by the first memory releasing means;
Using the secured character part vectorization processing work memory, the character part vectorization process stored in the second memory area based on the result of the area analysis process held in the process result holding memory Character part vectorization processing means for executing character part vectorization processing on the image for use and storing the result of the executed character part vectorization processing in the processing result holding memory;
A second memory releasing means for releasing the second memory area secured on the memory and the work memory for character part vectorization processing at the end of the processing by the character part vectorization processing means;
On the memory after generating the graphic part vectorization processing image suitable for the vectorization processing of the graphic part from the input image, and releasing the generated graphic part vectorization processing image by the second memory releasing means An image generation means for drawing part vectorization processing stored in a third memory area secured in
A character part filling process work memory securing means for securing a character part filling process work memory on the memory released by the second memory releasing means;
For the graphic part vectorization process stored in the third memory area based on the result of the area analysis process held in the processing result holding memory, using the secured character part filling process work memory Character part filling processing means for executing character part filling processing on the image;
Third memory releasing means for releasing the work memory for character portion filling processing secured on the memory at the end of the processing by the character portion filling processing means;
Graphic part vectorization processing work memory securing means for securing graphic part vectorization and hole filling processing work memory on the memory released by the third memory release means;
The graphic part vector stored in the third memory area based on the result of the area analysis process held in the processing result holding memory using the secured graphic part vectorization and hole filling process work memory Graphic part vectorization means for executing graphic part vectorization processing on the graphic processing image, and storing the result of the graphic part vectorization processing executed in the processing result holding memory;
For the graphic part vectorization processing image after the character part filling process stored in the third memory area is performed using the reserved graphic part vectorization and filling process work memory, Drawing part filling processing means for executing the part filling process;
A fourth memory releasing means for releasing the graphic portion vectorization and filling processing work memory secured on the memory at the end of the processing by the drawing portion filling processing means;
Using the memory released by the fourth memory releasing means, the character part vectorization process result, the graphic part vectorization process result held in the processing result holding memory, and the third memory An electronic file generating means for generating an electronic file including an image for graphic part vectorization processing after the character part and graphic part held in the region have been subjected to filling processing;
An image processing apparatus.   The electronic file generating means uses the memory released by the fourth memory releasing means, and uses the result of character part vectorization processing and the result of graphic part vectorization processing held in the processing result holding memory. And page drawing information including the graphic part vectorization processing image after the character part and the graphic part held in the third memory area are subjected to the filling process, and further, the page drawing information When the generation ends, the third memory area and the processing result holding memory are released, and the electronic file is generated from the page drawing information using the released memory. The image processing apparatus according to claim 1.   The graphic part vectorization means determines whether each graphic area can be vectorized based on the result of the area analysis process held in the processing result holding memory, and determines that the graphic process is possible. A vectorization process is performed on the area to generate a drawing vector, and a drawing area that is determined not to be vectorizable is cut out as a raster image from the drawing part vectorization processing image, and the generated area is generated. The image processing apparatus according to claim 1, wherein the graphic vector and the extracted raster image are stored in the processing result holding memory as a result of the graphic vector processing.   4. The image processing apparatus according to claim 1, wherein the result of the area analysis process executed by the area analysis unit includes attribute information and position information regarding a connected component of pixels. .   The image processing apparatus according to claim 1, wherein a resolution of the character part vectorization processing image is higher than a resolution of the region analysis image.   6. The graphic part vectorization processing image generated by the graphic part vectorization processing image generation means has a resolution of color difference information lower than a resolution of luminance information. Image processing apparatus. The drawing part vectorization means extracts a rectangular clip art region from the vectorization processing image, identifies a background region in the clip art region, and object regions other than the background region identified in the clip art region. To generate a graphic part vector, and store it in the processing result holding memory as a result of the graphic part vectorization process,
The drawing portion filling processing means performs a filling processing of a portion corresponding to the object region on the drawing portion vectorization processing image, and a boundary between the object region and the outside of the clipart region. The image processing apparatus according to claim 6, wherein different hole filling processing is performed at a boundary between the object region and the background region. An image processing method executed by an image processing apparatus including a memory for storing an input image,
A region in which region analysis image generation means generates a region analysis image suitable for region analysis processing from the input image, and stores the generated region analysis image in a first memory region secured on the memory An analysis image generation process;
The area analysis processing work memory securing means secures the area analysis processing work memory on the memory, and the area analysis processing work memory securing step;
The area analysis means executes the area analysis process on the area analysis image stored in the first memory area using the secured area analysis processing work memory, and the processing result secured on the memory An area analysis step of storing the result of the executed area analysis processing in the memory area for holding the processing result for holding;
The first memory releasing means releases the first memory area secured on the memory and the area analysis processing work memory at the end of the processing in the area analyzing step. Process,
The character part vectorization processing image generation means generates a character part vectorization processing image suitable for character part vectorization processing from the input image, and the generated character part vectorization processing image is used as the first character part vectorization processing image. A character part vectorization processing image generation step of storing in a second memory area secured on the memory after being released in the memory release step of
A character part vectorization processing work memory securing means for securing a character part vectorization processing work memory on the memory released in the first memory release step;
Character part vectorization processing means stores in the second memory area based on the result of the area analysis process held in the process result holding memory, using the reserved character part vectorization process work memory A character part vectorization processing step of executing character part vectorization processing on the character part vectorization processing image being performed and storing the result of the executed character part vectorization processing in the processing result holding memory;
The second memory releasing means releases the second memory area secured on the memory and the character portion vectorization processing work memory at the end of the processing by the character portion vectorization processing step. 2 memory release steps;
The graphic part vectorization processing image generation means generates a graphic part vectorization processing image suitable for the graphic vector processing of the graphic part from the input image, and the generated graphic part vectorization processing image is used as the second graphic part vectorization processing image. An image generation process for drawing part vectorization processing stored in a third memory area secured on the memory after being released by the memory releasing means;
A character part filling process work memory securing unit for securing a character part filling process work memory on the memory released by the second memory releasing unit;
Character portion filling processing means is stored in the third memory area based on the result of the area analysis processing held in the processing result holding memory, using the secured character portion filling processing work memory. A character portion filling process step of performing character portion filling processing on the graphic portion vectorization processing image;
A third memory releasing step for releasing the work memory for character portion filling processing secured on the memory at the end of the processing by the character portion filling processing step;
The graphic part vectorization processing work memory securing means secures the graphic part vectorization and hole filling processing work memory on the memory released in the third memory releasing process. When,
The graphic part vectorization means uses the reserved graphic part vectorization and hole filling process work memory to store the third memory area based on the result of the area analysis process held in the processing result holding memory. A graphic part vectorization process for executing graphic part vectorization processing on the stored graphic part vectorization processing image, and storing the result of the graphic part vectorization process executed in the processing result holding memory;
The drawing part vectorization process after the drawing part filling process means has executed the character part filling process stored in the third memory area by using the graphic part vectorization and filling process work memory secured. A graphic part filling process for executing a graphic part filling process on the image for use;
A fourth memory releasing step for releasing the graphic part vectorization and hole filling process work memory secured on the memory at the end of the process by the graphic part filling process;
The electronic file generation means uses the memory released in the fourth memory releasing step, and the result of the character part vectorization process and the result of the graphic part vectorization process held in the processing result holding memory An electronic file generating step for generating an electronic file including an image for graphic part vectorization processing after the character part and graphic part held in the third memory area have been subjected to the filling process;
An image processing method. Computer
Region analysis image generation for generating a region analysis image suitable for region analysis processing from the input image stored in the memory and storing the generated region analysis image in the first memory region secured in the memory means,
Area analysis processing work memory securing means for securing area analysis processing work memory on the memory;
The area analysis processing is executed on the area analysis image stored in the first memory area by using the reserved area analysis processing work memory, and the processing result for holding the processing result secured on the memory is obtained. Area analysis means for storing the result of the executed area analysis processing in the holding memory area;
First memory releasing means for releasing the first memory area secured on the memory and the work memory for area analysis processing at the end of processing by the area analyzing means;
A character portion vectorization processing image suitable for character portion vectorization processing is generated from the input image, and the generated character portion vectorization processing image is released on the memory after being released by the first memory releasing means. A character part vectorization processing image generation means stored in the second memory area secured in
A character part vectorization processing work memory securing means for securing a character part vectorization processing work memory on the memory released by the first memory releasing means;
Using the secured character part vectorization processing work memory, the character part vectorization process stored in the second memory area based on the result of the area analysis process held in the process result holding memory Character part vectorization processing means for executing character part vectorization processing on the image for use and storing the result of the executed character part vectorization processing in the processing result holding memory;
Second memory releasing means for releasing the second memory area secured on the memory and the work memory for character part vectorization processing at the end of processing by the character part vectorization processing means;
On the memory after generating the graphic part vectorization processing image suitable for the vectorization processing of the graphic part from the input image, and releasing the generated graphic part vectorization processing image by the second memory releasing means Image generation means for drawing part vectorization processing stored in a third memory area secured in
A character part filling process work memory securing means for securing a character part filling process work memory on the memory released by the second memory releasing means;
For the graphic part vectorization process stored in the third memory area based on the result of the area analysis process held in the processing result holding memory, using the secured character part filling process work memory Character portion filling processing means for executing character portion filling processing on the image,
Third memory releasing means for releasing the work memory for character part filling processing secured on the memory at the end of the processing by the character part filling processing means;
Graphic part vectorization processing work memory securing means for securing graphic part vectorization and hole filling processing work memory on the memory released by the third memory releasing means;
The graphic part vector stored in the third memory area based on the result of the area analysis process held in the processing result holding memory using the secured graphic part vectorization and hole filling process work memory Graphic part vectorization means for executing graphic part vectorization processing on the graphic processing image, and storing the result of the graphic part vectorization processing executed in the processing result holding memory;
For the graphic part vectorization processing image after the character part filling process stored in the third memory area is performed using the reserved graphic part vectorization and filling process work memory, Drawing part filling processing means for executing part filling processing;
A fourth memory releasing means for releasing the graphic portion vectorization and hole filling processing work memory secured on the memory at the end of the processing by the drawing portion filling processing means;
Using the memory released by the fourth memory releasing means, the character part vectorization process result, the graphic part vectorization process result held in the processing result holding memory, and the third memory An electronic file generating means for generating an electronic file including an image for graphic part vectorization processing after the character part and graphic part held in the region have been subjected to filling processing;
Computer program to function as.   A computer-readable storage medium storing the computer program according to claim 9. Memory to store input images;
Image generating means for generating an image suitable for image processing from the input image, and storing the generated image in a memory area secured on the memory;
A work memory securing means for securing a work memory for the image processing on the memory;
Using the secured work memory, execute image processing on an image suitable for image processing stored in the memory area, and execute the process in the memory area for holding the processing result secured on the memory. Image processing means for storing the result of the processed image processing;
An image processing apparatus comprising: a memory releasing unit that releases the memory area secured on the memory and the image processing work memory at the end of the image processing of the image processing unit;
For each of a plurality of types of image processing, processing by the image generation unit, the work memory securing unit, the image processing unit, and the memory release unit is executed, and the plurality of types of image processing are performed from the processing result holding memory area. An image processing apparatus, further comprising: an electronic file generation unit configured to generate an electronic file including a processing result of each image processing.

Images