JP2003189096A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To surely extract a character area from an image data to execute a proper process for the character area in the read image data. <P>SOLUTION: After the character area is firstly extracted from a received image data (S241), a photograph area and a graphic area are separated from the data remaining after extraction (S243, S245, S246). Consequently, the character area is not extracted in accordance with the other areas under the effect of separating process for the other areas. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus for performing area separation processing for separating a photograph area, a graphic area, and a character area from image data to be processed.

[0002]

2. Description of the Related Art From image data obtained by reading an original, a photographic area, a graphic area, and a character area are discriminated, processing is performed in accordance with each area, and then image data obtained by combining the areas is output. There is known an image processing technique (see, for example, Japanese Patent Laid-Open No. 5-342408).

[0003]

However, in the above conventional technique, it may be difficult to determine the type of each area in the image data without error. For example, in the image data, when a plurality of different types of areas are arranged in a complicated layout, or when they are arranged so as to overlap each other, there is a high possibility that the type of area is erroneously determined.

For example, when a character image overlaps a photographic image or a graphic image in the image data, there is a possibility that the character region is not discriminated and the whole is discriminated as a photographic region or a graphic region and is extracted. In addition, for the area including the extracted character image, a J
If the PEG compression process is performed, the character image in the photographic area may deteriorate and may be difficult to read when restored.

As described above, when it is desired to intensively extract a character area including a character image from the image data and to execute processing suitable for the character area such as character recognition processing on the extracted area. However, since the character region cannot be reliably extracted, not only the character recognition processing for the character image cannot be sufficiently executed, but also the character image may be inappropriately processed.

The present invention has been made in view of the above problems of the prior art, and an object of the present invention is to perform appropriate processing on a character area in read image data. It is to reliably extract the character area from the image data.

[0007]

The above objects of the present invention can be achieved by the following means.

(1) From the image data to be processed,
An image processing apparatus having area separating means for separating a photograph area, a graphic area, and a character area, wherein the area separating means extracts a character area from the image data before the character area and the graphic area. An image processing apparatus comprising area first extraction means, wherein a character area is extracted from the image data and remaining data is separated into a photographic area and a graphic area.

(2) The area separating means further comprises a photographic area preceding extracting means for extracting a photographic area prior to a figure area from the data remaining after the character area is extracted from the image data. The image processing apparatus according to (1) above, which is characterized.

(3) The area separating means further comprises graphic area preceding extraction means for extracting a graphic area prior to a photographic area from the data remaining after the character area is extracted from the image data. The image processing apparatus according to (1) above, which is characterized.

(4) The photographic area preceding extraction means performs a first area division process for dividing the character area extracted from the image data and remaining data into a plurality of predetermined areas, and the image data is divided. First to extract a photo region from a region
A photograph area extracting unit and a plurality of data smaller than the predetermined area with respect to the remaining data obtained by further extracting the photograph area by the first photograph area extracting unit from the data remaining after the character area is extracted from the image data. The image processing apparatus according to (2) above, further comprising: a second photographic area extracting unit that performs a second area division process that can be divided into areas and extracts a photographic area from the divided areas.

(5) The graphic area preceding extraction means performs a first area division process capable of dividing a character area extracted from the image data and remaining data into a plurality of predetermined areas, and the divided data is divided. First to extract a graphic area from the area
A graphic area extracting unit and a plurality of data smaller than the predetermined area with respect to the data remaining after the graphic area is further extracted by the first graphic area extracting unit from the data remaining after the character area is extracted from the image data. The image processing apparatus according to (3) above, further comprising: second graphic area extracting means for performing a second area dividing process that can be divided into areas and extracting a graphic area from the divided areas.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing the overall configuration of an image processing system including an image processing apparatus according to an embodiment of the present invention. The image processing system includes an image processing device 1
, A scanner 2 and a file server 3, which are communicably connected to each other via a computer network 4. The types and the number of devices connected to the computer network are not limited to the example shown in FIG.

FIG. 2 is a block diagram showing the arrangement of the image processing apparatus 1 according to this embodiment. In FIG. 2, the image processing apparatus 1 includes a control unit 101, a storage unit 102, and an operation unit 1.
03, input interface unit 104, output interface unit 105, area separation unit 106, image processing unit 108, document file creation unit 109, and file format conversion unit 11
0, which are connected to each other via a bus 111 for exchanging signals.

The control unit 101 is a CPU, and controls various parts described above and executes various arithmetic processes according to a program. The storage unit 102 stores a ROM that stores various programs and parameters in advance, a RAM that temporarily stores programs and data as a work area, stores various programs and parameters, or stores image data obtained by image processing. It consists of a hard disk used for temporary storage.

The operation unit 103 is composed of keys, an operation panel and the like for setting various items or instructing the start of operation. As shown in FIG. 3, items that can be set include a destination of image data, an output file format, a document mode, scan conditions, post-scan processing, and the like.

The input interface unit 104 is an interface for receiving data such as image data and instructions, and the output interface unit 105 is an interface for transmitting data such as output files and instructions.

The area separating unit 106 separates the photograph area, the graphic area, and the character area from the image data. The image processing unit 108 includes a photo area processing unit 108a, a graphic area processing unit 108b, and a character area processing unit 108c. The area processing units 108a to 108c perform appropriate image processing on the character area, the graphic area, and the photograph area, which are extracted by the area separating unit 106, according to the type of the area.

The document file creation unit 109 synthesizes the respective areas including the processed images sent from the photograph area processing unit 108a, the graphic area processing unit 108b, and the character area processing unit 108c, and creates a composite file in the internal file format. Create a document file. The file format conversion unit 110
Convert the document file created by the internal file format to the set output file format. The output file formats include various document creation software document formats, PostScript (registered trademark), PDF, JPEG, and TIF.
A general-purpose format such as F is used.

The scanner 2 reads a document to acquire image data, and transmits the obtained image data to the image processing apparatus.

The file server 3 is a computer,
The file received via the computer network 4 is stored, and the stored file is transferred to another device on the computer network in response to a transfer request.

The computer network 4 is a LAN connecting computers, peripheral devices, network devices, etc. according to standards such as Ethernet (registered trademark), token ring, and FDDI, or a WA connecting LANs by dedicated lines.
It consists of N etc.

Next, the procedure of processing in the image processing apparatus 1 of this embodiment will be described with reference to FIG. The algorithm shown in the flowchart of FIG. 4 is stored as a program in the storage unit 102 of the image processing apparatus 1, and is executed by the control unit 101.

First, in step S101, various items are set. That is, the settings for the destination of image data, output file format, original mode, scan condition, and post-scan processing are performed. Here, the destination of the image data is set by inputting the IP address, the host name, the mail address, etc. of the image output destination device. The output file format is set by selecting the file format of the output file to be transmitted to the image output destination device.

The document mode is set by selecting which area of the photo area including the photographic image, the graphic area including the graphic image, and the character area including the character image is given priority. Done by. Here, the photographic image refers to an image having a gradation that continuously changes, such as a photograph or a design. The graphic image refers to an image created by, for example, a personal computer such as a line or a solid picture.

As shown in FIG. 3, in the present embodiment, the user gives the highest priority to the area (first priority area) and the second priority through the operation unit 103 according to the content of the original. It is possible to select an area (second priority area) to be processed as a result. If the user does not select the first priority area and / or the second priority area,
Area priorities are determined according to a predetermined default value.

The scan conditions are set by designating the scan area, scan resolution, color / monochrome and the like. The post-scan processing is set by specifying a character recognition process, a vector conversion process, an image compression method, a color reduction method, an output resolution, and the like.

In step S102, the process waits until there is a command to start image processing. The start command is issued by the user on the operation unit 1.
03, for example, by operating the start key.

In step S103, an original reading command is transmitted to the scanner 2 via the output interface unit 105. Here, when the scanner 2 receives an original reading command from the image processing apparatus 1, the scanner 2 reads an original set at a predetermined position to acquire image data, and transmits the obtained image data to the image processing apparatus 1.

In step S104, the scanner 2
It waits until image data is received via the input interface unit 104. Here, when the image data is received from the scanner 2, the received image data (RGB image data) is stored in the storage unit 102.

FIG. 5 is a diagram schematically showing an example of the received image data. The image data shown in FIG. 5 includes photographic images PI1 to PI4, graphic images GI1 to GI3, character images CI1 to CI6, and a base U. As shown in the figure, the graphic image GI3 and the character images CI1 and CI3 are arranged in the photographic image PI1, and the photographic images PI3 and PI4 and the character images CI5 and CI6 are arranged in the graphic image GI2. . Here, the background refers to a portion other than the photograph area, the graphic area, and the character area in the image data, for example, the portion of the image data corresponding to the portion where the color of the unprinted paper of the original document remains. Say.

The image processing start command may be input from another device on the computer network 4 or from the scanner 2.

In step S105, the area separation unit 106
Thus, from the image data received from the scanner 2, that is, the image data to be processed, the photograph area, the graphic area,
And the character area are separated. In the present embodiment, the area separation unit 106 extracts the first priority area from the image data first according to the original mode set by the selection made by the user through the operation unit 103, and first extracts the first priority area from the image data. The remaining two areas are separated from the data remaining after the areas have been extracted. Furthermore, when the second priority area is selected, the remaining two areas are separated by first extracting the second priority area from the data remaining after the first priority area is extracted from the image data. . Where 3
Each image is created for each type of area, and each area is
It is extracted as a region including each image. Details of the procedure of this area separation processing will be described later.

In step S106, step S105
The process adapted to the photo region is performed on the photo region separated in (1). That is, the photograph area processing unit 108
According to a, the photographic image in the photographic area is subjected to, for example, resolution conversion, irreversible compression processing of the color image, and is stored in the storage unit 102 together with the position information.

In step S107, step S105
The processing adapted to the graphic area is performed on the graphic area separated in (1). That is, the graphic image processing unit 108
In step b, the graphic image in the graphic area is subjected to, for example, smoothing processing, color reduction processing, and the like, and then lossless compression processing of the color image is performed, and the storage unit 102 together with position information.
Stored in.

In step S108, step S105
A process adapted to the character area is performed on the character area separated in. That is, the character image processing unit 108
By c, the character image in the character area is, for example, binarized and then subjected to lossless compression processing of 1-bit data,
The color information and the position information are stored in the storage unit 102.

In step S109, the document file creation unit 109 synthesizes the above three areas including the processed images sent from the photograph area processing unit 108a, the graphic area processing unit 108b, and the character area processing unit 108c. A document file is created.

The areas are synthesized by, for example, as shown in FIG. 6A, a photograph area 600, a graphic area 700, and
And by outputting the character area 800.
Here, as shown in FIG. 6B, a portion other than the photographic image 600a in the photographic area 600, a portion other than the graphic image 700a in the graphic area 700, and a portion other than the character image 800a in the character area 800 are respectively Mask part 600b-80
Designated as 0b. The areas 600 to 800 are given by circumscribed rectangles of the images 600a to 800a. The masking part is a part where a masking process for validating the information previously stored in the memory is performed. First, the photograph area 600 and the graphic area 700 are masked and arranged, and then the character area 800 is masked and arranged. The reason why the masking process is performed on all the areas is that not only the case where the image in each area is rectangular, but also the case where the image of the other area has a shape that goes inside. In this way, as shown in FIG. 6A, the images in the three areas are output to the memory without being lost, and the area composition is completed.

The reason for arranging the character area after the other areas is that, as will be described later, after extracting the character area during the area separation processing, the portion where the character image existed is replaced with the original image data. This is because the interpolation process of interpolating in the background portion is performed. That is, when the character area is arranged before the other areas, the character image is prevented from being hidden by the interpolated portion in the area arranged later. In addition, after the graphic area is extracted, when the interpolation process is performed on the portion where the graphic image was present, the photograph area, the graphic area, and the character area are output in this order on the memory.

In step S110, the file format conversion unit 110 converts the document file created in step S109 into the set output file format.

In step S111, step S110
The output file obtained in step 3 is the output interface unit 10
5 and the computer network 4 to the file server 3.

In this embodiment, when the file server 3 receives an output file from the image processing apparatus 1 via the computer network 4, it develops a character image and a graphic image from the received file and performs character recognition on the character image. After processing, it converts to character code data, and the graphic image undergoes vector conversion processing to convert to vector data,
The converted data are combined again with the photographic image, and the document file obtained by converting the data into a predetermined file format is stored in a predetermined directory of a storage device such as a hard disk. Then, when there is a transfer request for the file from another device on the computer network 4, the stored file is transferred to the other device via the computer network 4.

Next, the image processing apparatus 1 according to the present embodiment.
Which is the characteristic of the image processing of step S105 shown in FIG.
The procedure of the area separation process of is described in more detail.

7 to 12 are flowcharts showing the procedure of the area separation processing according to the original mode. There are a total of 6 types of document modes set in step S101 of FIG. That is, in the manuscript mode, the first priority area and the second priority area are the photographic area and the graphic area, respectively (see FIG. 7), and the second mode is the photographic area and the text area (see FIG. 8). ), A third mode that is a graphic area and a photo area (see FIG. 9), a fourth mode that is a graphic area and a text area (see FIG. 10), and a fifth mode that is a text area and a photo area, respectively. (See FIG. 11) and a sixth mode (see FIG. 12) which is a character area and a graphic area, respectively.

Each of the region separation processings in FIGS. 7 to 12 is divided into regions by binarization, first photo / graphic region extraction, edge region division, second photo / graphic region extraction, photo / graphic region extraction, and It includes a processing block called “character area extraction” that has the same contents in each mode. That is, in each area separation process, the priority area is different depending on the original mode, and the area extraction order is different according to the area priority. For example, in the case of the first mode, the priority of regions is from the highest to the photographic region, the graphic region, and the character region, and the region extraction order is also the same.

The contents of each processing block in the area separation processing will be described in detail below. Here, as an example, the content of each processing block will be specifically described in the same order as the processing order of the first mode in FIG. 7.

(Region Division by Binarization) First, the procedure of the area division by binarization will be described with reference to FIG.

For example, a lightness image, that is, a lightness image is created based on the received image data shown in FIG. 5 (step S301). Next, the background is removed from the brightness image (step S302), and smoothing processing using a smoothing filter is performed (step S303). The brightness image from which the background and noise have been removed is binarized at the brightness level of the background (hereinafter referred to as “background level”) (step S304). As a result, as shown in FIG. 14, a binary image in which the area other than the base is painted black is obtained. Edge detection is performed on this binary image by using, for example, a Laplacian filter which is a second-order differential filter (step S305). Then, by performing a closing (expansion / erosion) process, which is a type of morphological process, the edge is interpolated, and an image composed of the edge,
That is, an edge image is obtained (step S306, FIG. 1).
5). The edge image in FIG. 15 corresponds to the contour line of the binary image in FIG.

By detecting the contour line of the binary image obtained by binarizing at the background level in this way, when there are a plurality of regions in contact with each other, these regions are combined into one form. , Area division is performed. In the first to fourth modes, four large areas and character areas as shown in FIGS. 14 and 15 are obtained. However, in the fifth and sixth modes in which the character areas are prioritized, the character areas are extracted before the area division by binarization, so only four large areas are obtained.

(First Photo / Graphic Area Extraction) Next, referring to FIG.
The procedure of extracting the first photograph / graphic area will be described with reference to FIGS.

The process for extracting the first photograph / graphic area is as follows:
It is executed for each area (area surrounded by the closed curve in FIG. 15) defined by the contour line of the binary image obtained by binarizing at the background level. First, the position of the line segment between edges in the first direction, for example, the main scanning direction is detected based on the position of the edge of the edge image in FIG. 15 (step S401). Then, using the pixels of the brightness image corresponding to the position of the line segment between the target edges, for example, as shown in FIG.
A histogram (first histogram) as shown in 8 is created (step S402). Further, by performing a smoothing process using an average value filter (| 1 | 0 | 1 |) on the first histogram, a second histogram as shown in FIG. 19, for example, is created (step S403). Then, the first histogram Hist1 and the second histogram Hist2 for each gradation are calculated according to the following formula.
Is calculated, and the sum thereof is used as a feature amount (step S4).
04). Note that the brightness is represented by 8-bit data and is 25
It has 6 gradations.

[0053]

[Equation 1]

Then, the ratio R1 (= N1 / T) of the total number N1 of pixels located on the line segment between the target edges and a predetermined constant T1.
1) is calculated (step S405). Constant T1
Is a first parameter for separating the photo area and the graphic area. Subsequently, the ratio R1 which is the threshold value and the feature amount are compared (step S406). The feature amount is the ratio R
When it is determined that it is greater than 1 (step S406: N
O), all the pixels located on the edge-to-edge line segment in the first direction are regarded as belonging to the graphic region, and labeling, which is a process of assigning numbers, is executed to generate labeling data (step S407). . That is, the result of the region determination is labeled for each pixel. Specifically, the result of the area determination is stored in correspondence with the pixel position. On the other hand, when it is determined that the feature amount is equal to or smaller than the ratio R1 (step S406: YES), it is considered that all the pixels located in the edge-to-edge line segment in the first direction belong to the photographic region and the labeling is performed. Data is generated (step S408). Then, the line segment between the target edges is
It is determined whether or not the line segment is the final line segment between edges (step S408). When it is determined that the line segment between the target edges is not the final line segment between edges (step S40)
8: NO), returning to step S402, the above process is repeated.

Then, based on the edge image of FIG. 15, the position of the line segment between edges in the second direction, which is the direction orthogonal to the first direction, for example, in the sub-scanning direction is detected (step S410). Then, steps S407 and S40
A ratio R2 (= N3 / N2) between the total number of pixels N2 located in the line segment between the edges of interest and the number of pixels N3 belonging to the photo area is calculated based on the labeling data created in step 8 (step S411). . Then, the ratio R2 is compared with a predetermined constant T2 which is a threshold value (step S412).
The constant T2 is a second parameter for separating the photographic area and the graphic area. When it is determined that the ratio R2 is smaller than the constant T2 (step S412: NO), all the pixels existing in the line segment between the target edges regarding the second direction are regarded as belonging to the graphic region, and are labeled. It is corrected (step S413). On the other hand, when it is determined that the ratio R2 is equal to or larger than the constant T2 (step S412: YE
S), all the pixels located on the line segment between the edges of interest in the second direction are regarded as belonging to the photographic area and are relabeled (step S414). Subsequently, it is determined whether or not the line segment between the target edges is the last line segment between the edges in the second direction (step S415). When it is determined that the line segment between the edges of interest is not the final line segment between edges (step S415: NO), the process returns to step S411,
The above process is repeated.

Next, in one area obtained by the area division by binarization, the number N5 of pixels labeled as belonging to the photograph area and the number N4 of pixels labeled as belonging to the graphic area are compared. (Step S416). N
When it is determined that 5 <N4 (step S416: N
O), the area is determined to be a graphic area (step S4).
17). On the other hand, when it is determined that N5 ≧ N4 (step S416: YES), the area is determined to be the photograph area (step S418). Then, the determination of the higher priority area of the photograph area or the graphic area is confirmed, and all the pixels in the area are considered to belong to the higher priority area and are relabeled. And extracted from the image data (step S419). Here, the circumscribing rectangle of the region is calculated by tracing the contour of the region based on the position data, and a rectangular region including pixels labeled as belonging to the region corresponding to the coordinate position in the circumscribing rectangle, It is extracted from the image data.

As described above, in the first photo / graphic region extraction, for the region obtained by performing the region division by binarization, the image feature amount between the edges which are the boundaries of the region is calculated. Thus, it is determined whether the area is a photograph area or a graphic area. When the photo area has a higher priority than the graphic area (first, second, and fifth areas)
Mode), the photo area is determined, and the area is extracted in the form of a rectangular area. On the other hand, if the graphic area has a higher priority than the photograph area (third, fourth, and sixth modes),
The graphic area is determined, and the area is extracted in the form of a rectangular area.

In the present embodiment, as described above, the lightness distribution of the graphic region is uniform to some extent, while the lightness distribution of the photo region is dispersed. Is determined. However, the area determination method is not limited to this, and the area may be determined using, for example, a frequency component extracted from the brightness image as a feature amount.

(Region Division by Edge) Next, with reference to FIG. 20, the procedure of the region division by edge will be described.

The area division by edges is performed on the data remaining after the area having the higher priority is extracted from the photograph area or the graphic area by the first photo / graphic area extraction.

First, the R component image in the image data, G
Edge detection is performed for each of the component image and the B component image by using an edge detection filter such as a Laplacian filter (steps S501 to S503). Then, an OR process is performed to obtain the union of the edges of the detected R component image, G component image, and B component image (step S504),
Further, a closing process is performed in order to eliminate the discontinuity of the edge (step S505). Here, in the case of the first to fourth modes, since the character area is not yet extracted, the edge of the character image is also detected. However, for example, when the size (vertical and horizontal dimensions) of the circumscribed rectangle of the area is smaller than a predetermined threshold value and the ratio of the number of effective pixels inside the area to the size of the circumscribed rectangle is smaller than the predetermined threshold value, the area is It is excluded from the target of area division by edge. As a result, the edge of the character image is
It is no longer considered the boundary of the area.

FIG. 21 is a diagram schematically showing an example of a region obtained by dividing a region by edges,
Is in the first, second, or fifth mode, (b) is
The case of the 3rd, 4th, or 6th mode is shown. here,
In FIG. 21A, by the first photo / graphic area extraction,
The photo region has already been extracted before the graphic region. That is, the area P1 (including the area G3) and the area P2 shown in FIG. 21B have been extracted as the photograph area in FIG. 21A. In addition, in FIG.
By the first photo / graphic area extraction, the graphic area is already extracted before the photo area. That is, FIG. 21 (A)
The area G1 and the area G2 (including the areas P3 and P4) shown in are already extracted as the photographic areas in FIG.

As described above, the area division by the edge is the first
The higher priority area of the photo area or the graphic area is extracted by the photo / graphic area extraction, and the higher priority area overlaps the lower priority area in the remaining data. If the state still remains, or if the area with the higher priority remains within the area with the lower priority, the area with the higher priority is further extracted. Is done to That is, finer area division is performed by detecting the edge.

(Second Photo / Graphic Area Extraction) Next, the procedure of the second photo / graphic area extraction will be described. Second photo /
The graphic region extraction is performed by using the above-mentioned first photograph / image for the region obtained by the region division by the edge shown in FIG.
The same process as the graphic region extraction is performed again. Thus, in the first, second, and fifth modes, the first photo /
In the graphic area extraction, the photo area in the graphic area that has not been extracted as the photo area is extracted. For example, in FIG.
In (A), areas P3 and P4 are additionally extracted as photo areas. Further, in the third, fourth, and sixth modes, the graphic area in the photo area that has not been extracted as the graphic area by the first photo / graphic area extraction is extracted.
For example, in FIG. 21B, the area G3 is additionally extracted as a graphic area.

Incidentally, the second photograph / shown in FIGS.
In the photo / graphic area extraction performed after the completion of the graphic area extraction, among the areas obtained by the area division by the edge shown in FIG. 21, the areas remaining without being extracted by the second photo / graphic area extraction are extracted. To be done.

(Character Area Extraction) Next, the procedure of character area extraction will be described with reference to FIG.

Here, in order to simplify the description, a case will be described in which a process relating to character region extraction is performed on the image data shown in FIG. 23, for example.

First, a region integration process is performed on the image data (step S601). This process is, for example, a process for extracting a character image on a background image or a character area including a character image having different pixel values. Specifically, first, for the brightness image of the image data,
After the smoothing process is performed, the edge image is created by performing binarization using the variation threshold. The binarization process using the variation threshold is, specifically, for example, as shown in FIG. 24, a value obtained by subtracting the offset value from the maximum gradation value of the pixels located at the four corners of the 5 × 5 block is the threshold value. Is a process of binarizing the pixel of interest. Subsequently, the interval between the black pixels in the main scanning direction of the obtained edge image is measured, all the white pixels between the black pixels which are less than the predetermined interval are replaced with the black pixels, and the black pixels are connected in the main scanning direction. Create an edge image. Further, similar processing is repeated in the sub-scanning direction of the obtained connected edge image, and a connected edge image in which black pixels are connected in the main / sub-scanning direction is obtained. In this way, the image processing apparatus 1 connects adjacent black pixels and integrates individual character images that are isolated in the image data into one area, thereby creating one area for each character string that has been collected to some extent. It is possible to extract it as a region.

Next, area extraction processing is performed (step S602). This process is a process of separately extracting a group of connected black pixels as one area. In particular,
First, the obtained connected edge image is labeled for each connected black pixel. At the same time as the labeling, the position information (width, height, and coordinates) of the circumscribed rectangle for each connected black pixel of the same label is detected, and a labeling image is created. Then, based on the circumscribing rectangle detected at the time of labeling and the label number, the region surrounded by the circumscribing rectangle is extracted as a local region from the labeling image. Here, by extracting the circumscribing rectangle including only the pixels having the same label number, it is possible to separately extract the image of the layout in which the circumscribing rectangles overlap each other.
FIG. 25 is a graph of a concatenated edge image obtained by performing binarization processing and a concatenation process of black pixels using a variation threshold, and for each concatenated black pixel of the same label in a labeling image obtained from concatenated edge image data It is a figure which shows a circumscribed rectangle.

Next, the diagonal edge component of the image belonging to each local region extracted in step S602 is extracted as a feature amount (S603), and the local region in which the content ratio of the diagonal edge component is within the predetermined range is It is determined to be a character area (S604). The character area includes many diagonal edge components in a small area as compared with other areas such as figures, photographs, and ruled lines. Therefore, it is possible to determine whether or not the local region is the character region by extracting the diagonal edge component as the frequency component peculiar to the character region and obtaining the content rate in the local region. The extraction of the diagonal direction edge component is equivalent to the process of extracting the high frequency component from the frequency components obtained by the 2 × 2 DCT (discrete cosine transform). That is, by performing DCT with a 2 × 2 matrix on the image in the local area and performing inverse DCT conversion with the high frequency component of the obtained frequency components as “0”, the restored image from which the high frequency component has been removed is obtained. can get. Then, by extracting the difference between the original image and the restored image, only the high frequency component of the original image can be extracted.
Here, high-speed processing is possible by performing the filter processing shown in FIG. FIG. 27 is a diagram showing an example of an oblique edge component image obtained by binarizing the extracted high frequency component. The territorial domain is generally in units of words. Therefore, when the controlled area is a character area, the content rate of the diagonal edge component in the local area, that is, the area belonging to the local area with respect to the area of the local area in FIG.
The ratio of the total number of black pixels of 7 is within a predetermined range (about 0.2 to
20%). Therefore, a local area having the ratio within the above range is determined to be a character area.

Next, a character image creating process is performed (step S605). That is, by binarizing the original image data (image data received from the scanner 2) in the local area determined to be the character area in step S604, the character portion and the background portion thereof are distinguished, and only the character portion is detected. Will be created. The threshold used when binarizing is set for each character area. As a method of setting the threshold for each character area, for example, the following method can be used. First, for each character area, a brightness histogram as shown in FIG. 28A is created using the brightness image of the image data in the character area. Then, the brightness histogram is converted into a percentage with respect to the number of pixels in the character area to perform a second derivative, and "1" is output if the result of the second derivative is a predetermined value or more, and "0" is output otherwise. As a result, a peak detection histogram as shown in FIG. 28B is created and the peak is detected. Then, when the number of detected peaks is 2 or more, the center value of the peaks at both ends, and when the number of peaks is 1, the peak and the left and right rising values of the brightness histogram (“Lef” in FIG. 28A).
When the number of peaks is 0, the central value of the left and right rising values of the brightness histogram is determined as the threshold value. In this way, since the binarization threshold value that differs depending on the peak number of the brightness histogram in the character area is used, it is possible to binarize a character image on a background image, a reverse character image, etc. without missing the image.

Next, image interpolation processing is performed (step S606). That is, the character image including only the character portion is removed from the original image data, and the removed portion is interpolated with the background pixels of the character image. Here, the background pixel of the character image can be specified from the image obtained by binarizing each character area in step S605. The value of the background pixel used for the interpolation is given by calculating the average value for each RGB of the pixels corresponding to the background of the character image in the original RGB image data.

As described above, the image processing apparatus 1 integrates adjacent areas by connecting neighboring black pixels, extracts the integrated area, and calculates the feature amount representing character-likeness, It is determined whether each area extracted using the feature amount is a character area, and then a character image including only a character portion is created from the image data in the area determined to be the character area. Then, the part after the character image consisting of only the character part is removed is interpolated with the background pixels.

In this character area extraction, the character area can be surely extracted even when the character image overlaps the photographic image or the graphic image. However, when the document mode is set so that the photo area or the graphic area is prioritized over the text area, the text image overlaid on the photo image or the graphic image will be added first as a part of the photo area or the graphic area. Will be extracted.

As described above, the photographic area, the graphic area, and the character area are separated from the image data received from the scanner in the area extraction order according to the set original mode shown in FIGS. .

According to the present embodiment, when the photographic area, the graphic area, and the character area are separated from the image data, the extraction order of each area can be set, whereby which area is preferentially extracted. Can be controlled.
Therefore, a high-priority area is preferentially extracted in a state in which the other area is included even if it includes another area, and even if it is included in another area, it is preferentially extracted. To be extracted. In this way, the area with high priority is extracted earlier than the other areas, and thus is not erroneously discriminated from other areas and is not extracted, and the area is subjected to inappropriate processing. Image deterioration is prevented.

FIGS. 29 to 34 are diagrams showing a photograph area, a graphic area, and a character area, which are separated from the image data of FIG. 5 by the area separation processing in the first to sixth modes, respectively. ) Indicates the first extracted region, (B) indicates the second extracted region, and (C) indicates the third extracted region.

When the first priority area is a photograph area (first
And second mode), FIG. 29 (A) and FIG. 30 (A).
As shown in Fig. 3, the photo area is extracted from the received image data first, and then the graphic area and the character area are separated from the remaining data, so that the photo area is affected by the separation processing of other areas. As a result, it will not be extracted in association with the other area. As a result, the photograph area can be reliably extracted without being erroneously discriminated from other areas. Therefore, when the main purpose is to reproduce the photograph area with high image quality, it is possible to reliably extract more photograph areas and execute appropriate processing on the area. In other words, it is possible to prevent, for example, a certain portion in the photograph area from being erroneously determined as a character area, and the binarization processing adapted to the character area being performed in a subsequent step. Also, for example, a certain part in the photo area is misidentified as a graphic area,
It is possible to prevent the drawing area from being filled with a single color by performing the color-reduction processing adapted to the graphic area in the subsequent step. That is, it is possible to prevent the photographic image from being deteriorated due to inappropriate processing being applied to the photographic area. Further, by preferentially extracting the photo area, there is an advantage that the content of the original image data is maintained. That is, even if the area is a graphic area or a character area and is erroneously discriminated as a photographic area, the content of the area is maintained because it can be reproduced as an image.

Further, in the case of extracting the graphic area before the character area from the image data after extracting the photograph area (first
Mode), it is possible to prevent a portion in the graphic area from being erroneously determined to be a character area and to perform processing adapted to the character area in a later step. Therefore, deterioration of the photographic image and the graphic image is reduced. Also,
When the character area is extracted before the graphic area from the image data after extracting the photo area (second mode), the character image in the graphic area can be extracted. Therefore, deterioration of the photographic image and the character image is reduced.

If the first priority area is a graphic area (third area)
And the fourth mode), FIG. 31 (A) and FIG. 32 (A).
As shown in FIG. 3, the graphic area is first extracted from the received image data, and then the photo area and the text area are separated from the remaining data, so that the graphic area is affected by the separation processing of other areas. As a result, it will not be extracted in association with the other area. As a result, the graphic area can be surely extracted without being erroneously discriminated from other areas. Therefore, when the main purpose is to perform a process suitable for a graphic region such as a vector conversion process, it is possible to reliably extract a larger number of graphic regions and execute an appropriate process for the region. In other words, for example, in the case where graphic regions overlap each other in the photo region, it is possible to prevent noise from being generated due to the erroneous determination of the entire photo region and the JPEG compression. Further, for example, a region including a graphic image that is easily mistaken for a character image is erroneously discriminated as a character region, and a binarization process, which is a process adapted to the character region, and a character recognition process are executed. Can be prevented. That is, it is possible to prevent the graphic image from being deteriorated due to inappropriate processing being applied to the graphic area.

Furthermore, in the case of extracting the photo area before the character area from the image data after extracting the graphic area (third part)
Mode), it is possible to prevent a portion in the photograph area from being erroneously determined to be a character area and a process adapted to the character area to be performed in a later step. Therefore, the deterioration of the graphic image and the photographic image is reduced. Also,
When the character area is extracted from the image data after the graphic area is extracted (fourth mode), the character image in the photo area can be extracted. Therefore, the irreversible compression process can be performed on the photographic image without deteriorating the graphic image and the character image.

When the first priority area is a character area (fifth area)
And sixth mode), FIG. 33 (A) and FIG. 34 (A)
As shown in, the character area is extracted from the received image data first, and then the photo area and the graphic area are separated from the remaining data, so that the character area is affected by the separation processing of other areas. As a result, it will not be extracted in association with the other area. As a result, the character area can be surely extracted without being erroneously discriminated from other areas. Therefore, when the main purpose is to perform a process suitable for a character region, such as a character recognition process, it is possible to reliably extract a larger number of character regions and execute an appropriate process on the region. In other words, for example, when a character image overlaps a photographic image or a graphic image in image data,
The character area is not discriminated and the whole is discriminated as a photograph area or a graphic area and is extracted, so that not only the character recognition processing for the character image cannot be sufficiently performed but also the character image is improperly processed. Can be prevented.

Furthermore, in the case of extracting the photo area before the graphic area from the image data after the extraction of the character area (the fifth area)
Mode), for example, it is possible to extract a photographic area included in a graphic area. Therefore, deterioration of the character image and the photographic image is reduced. When the graphic area is extracted before the photo area from the image data after extracting the text area (sixth mode), for example, the graphic area included in the photo area can be extracted.
Therefore, irreversible compression processing can be performed on a photographic image without deteriorating the character image and the graphic image.

The present invention is not limited to the above-mentioned embodiments, but can be variously modified within the scope of the claims.

The image processing apparatus of the present invention is, in addition to the modes shown in the above-described embodiments, a computer such as a scanner, a personal computer, a workstation, a server, a digital copying machine, a facsimile apparatus, and an MFP (multi-function perip).
can be applied to equipment such as heral).

Further, in the above embodiment, the file server 3 is configured to expand the character image and the graphic image from the file received from the image processing apparatus 1 and perform the character recognition process and the vector conversion process, respectively. The image processing apparatus 1 may perform these processes. Also,
The content of each processing block in each area separation processing of FIGS. 7 to 12 can be changed as appropriate.

Further, in the above embodiment, the image processing apparatus 1
Is configured to set an area of the photograph area, the graphic area, and the character area that is preferentially extracted according to the content of the image data, but the present invention is not limited to this.
In the image processing apparatus according to the present invention, the area extracted first may be fixed in advance, and for example, the character area may be fixed as the area extracted first. Further, in the image processing apparatus according to the present invention, the extraction order of the areas may be fixed in advance. For example, the order of the character area, the photographic area and the graphic area or the order of the character area, the graphic area and the photographic area may be extracted. Can be fixed as order.

The image processing apparatus and the image processing method according to the present invention may be implemented by a dedicated hardware circuit for executing each of the above procedures, or by a CPU executing a predetermined program describing each of the above procedures. Can be realized. When the present invention is implemented by the latter, the above-mentioned predetermined program for operating the image processing apparatus may be provided by a computer-readable recording medium such as a flexible disk or a CD-ROM, or via a network such as the Internet. It may be provided online. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a hard disk or the like. Further, this program may be provided as, for example, independent application software, or may be incorporated into the software of the image processing apparatus as one function.

The above-described embodiment of the present invention is not limited to the invention described in claims 1 to 5 in the claims.
The inventions as shown in the following supplementary notes 1 to 6 are included.

[Supplementary Note 1] In the first area division processing, a binary image in which a portion of a photograph area or a figure area in the data remaining after the character area is extracted from the image data and a background portion other than these are distinguished By detecting an edge, it is possible to divide into a plurality of predetermined areas. In the second area division processing, a character area is extracted from the image data and a photographic area is further extracted from the remaining data. The image processing apparatus according to claim 4, wherein the image processing apparatus can be divided into a plurality of areas smaller than the predetermined area by detecting an edge.

[Supplementary Note 2] In the first area division processing, a binary image in which a portion of a photographic area or a figure area in the data remaining after the character area is extracted from the image data and a background portion other than these are distinguished By detecting an edge, it is possible to divide into a plurality of predetermined regions. In the second region division processing, a character region is extracted from the image data and a graphic region is further extracted from the remaining data. The image processing apparatus according to claim 5, wherein the image processing apparatus can be divided into a plurality of areas smaller than the predetermined area by detecting an edge.

[Supplementary Note 3] A reading unit for obtaining image data by reading an original is further provided, and the image data to be processed is obtained by the reading unit. The image processing apparatus according to any one of 1 and 2.

[Supplementary Note 4] An image processing method including an area separating step of separating a photographic area, a graphic area, and a character area from image data to be processed, wherein the area separating step is performed from the image data. An image processing method characterized in that a character region is extracted before a photo region and a graphic region, and the photo region and the graphic region are separated from the remaining data after the character region is extracted from the image data. .

[Supplementary Note 5] An image processing program having an area separation procedure for separating a photograph area, a graphic area, and a character area from image data to be processed, wherein the area separation procedure is performed from the image data. An image processing program, which is a procedure for extracting a character region before a photo region and a graphic region, and separating the photo region and the graphic region from the remaining data after the character region is extracted from the image data. .

[Supplementary Note 6] A computer-readable recording medium recording the image processing program according to Supplementary Note 5.

[0096]

As described above, according to the image processing apparatus of the present invention, the character area is first extracted from the image data to be processed, and then the photograph area and the graphic area are extracted from the remaining data. Can be separated. Therefore, the character area will not be extracted along with the other area due to the influence of the separation processing of the other area. As a result, the character area can be surely extracted without being erroneously discriminated from other areas. Therefore, when the main purpose is to perform a process suitable for a character region, such as a character recognition process, it is possible to reliably extract a larger number of character regions and execute an appropriate process on the region.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of an image processing system including an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of a configuration of an image processing device.

FIG. 3 is a diagram illustrating an example of a document mode setting screen on the operation unit.

FIG. 4 is a flowchart showing a processing procedure in the image processing apparatus.

FIG. 5 is a diagram schematically showing an example of image data received from a scanner.

FIG. 6 is a diagram for explaining the synthesis of regions,
(A) is a figure which shows the state after a synthesis | combination, (B) is a figure which shows the state before a synthesis | combination.

FIG. 7 is a flowchart showing a procedure of region separation processing in a first mode.

FIG. 8 is a flowchart showing a procedure of region separation processing in a second mode.

FIG. 9 is a flowchart showing a procedure of region separation processing in a third mode.

FIG. 10 is a flowchart showing a procedure of region separation processing in a fourth mode.

FIG. 11 is a flowchart showing a procedure of region separation processing in a fifth mode.

FIG. 12 is a flowchart showing a procedure of region separation processing in a sixth mode.

FIG. 13 is a flowchart showing a procedure of area division by binarization.

FIG. 14 is a diagram in which the area other than the base of FIG.
It is a figure which shows a value image.

FIG. 15 is a diagram showing an image composed of the edges of FIG. 14;

FIG. 16 is a flowchart showing a procedure for extracting a first photograph / graphic area.

FIG. 17 is a flowchart showing a procedure of first photo / graphic region extraction continued from FIG. 16;

FIG. 18 is a diagram showing a first histogram.

FIG. 19 is a diagram showing a second histogram.

FIG. 20 is a flowchart showing a procedure of area division by edges.

FIG. 21 is a diagram schematically showing an example of a region obtained by dividing a region by edges, where (A) is the first, second, or fifth mode, and (b) is the third or third mode. The case of the 4th or 6th mode is shown.

FIG. 22 is a flowchart showing a procedure for extracting a character area.

[Fig. 23] Fig. 23 is a diagram showing image data used for explaining a character region extraction.

[Fig. 24] Fig. 24 is a diagram for describing binarization processing using a variation threshold.

FIG. 25 is obtained for each concatenated black pixel of the same label in a concatenated edge image obtained by performing binarization processing by a variation threshold and concatenation processing of black pixels and a labeling image obtained from concatenated edge image data. It is a figure which shows a circumscribed rectangle.

FIG. 26 is a diagram for explaining a filter process used when removing high frequency components from the characteristic frequency components of image data.

FIG. 27 is a diagram showing an example of an oblique edge component image obtained by binarizing the extracted high frequency component.

FIG. 28 is a diagram showing an example of (A) a brightness histogram and (B) a peak detection histogram created from a brightness image of image data in a character area.

FIG. 29 is a flowchart illustrating the first mode region separation process shown in FIG.
From the image data of (A) the photo area extracted first,
It is a figure which shows the figure area extracted to the (B) 2nd, and the character area extracted to the (C) 3rd.

FIG. 30 is a flowchart illustrating the second mode region separation process shown in FIG.
From the image data of (A) the photo area extracted first,
It is a figure which shows the character area extracted to (B) 2nd, and the graphic area extracted to the (C) 3rd.

31 is a diagram illustrating the case of FIG.
From the image data of (A) the graphic area extracted first,
It is a figure which shows (B) the photograph area extracted 2nd, and (C) the character area extracted 3rd.

FIG. 32 is a diagram illustrating the case of FIG.
From the image data of (A) the graphic area extracted first,
It is a figure which shows the character area extracted to (B) 2nd, and the photograph area extracted to the (C) 3rd.

FIG. 33 is a diagram illustrating the case of FIG.
From the image data of (A) the character area extracted first,
It is a figure which shows (B) the photograph area extracted 2nd, and (C) the figure area extracted 3rd.

FIG. 34 is a flowchart illustrating the sixth mode region separation process shown in FIG.
From the image data of (A) the character area extracted first,
It is a figure which shows (B) the graphic area extracted the 2nd, and (C) the photo area extracted the 3rd.

[Explanation of symbols]

1 ... Image processing device, 101 ... control unit, 102 ... storage unit, 103 ... operation unit, 104 ... Input interface unit, 105 ... Output interface unit, 106 ... Area separation unit, 108 ... Image processing unit, 108a ... Photo region processing unit, 108b ... Graphic area processing unit, 108c ... Character area processing unit, 109 ... Document file creation section, 110 ... File format conversion unit, 111 ... bus, 2 ... Scanner, 3 ... file server, 4 ... Computer network.

Continued front page    (72) Inventor Yoshinori Tanaka             2-3-3 Azuchi-cho, Chuo-ku, Osaka             Kokusai Building Minolta Co., Ltd. (72) Inventor Masahiro Ozawa             2-3-3 Azuchi-cho, Chuo-ku, Osaka             Kokusai Building Minolta Co., Ltd. F-term (reference) 5C076 AA01 AA36 BA06 CA10                 5C077 MP05 MP07 PP25 PP27 PP28                       PP47 PQ19 RR02                 5L096 BA17 FA06 FA35 FA44 FA45                       GA34

Claims (5)

[Claims] 1. An image processing apparatus having area separating means for separating a photograph area, a graphic area, and a character area from image data to be processed, wherein the area separating means separates the photograph area from the image data. And a character area earliest extracting means for extracting the character area before the graphic area, and separating the photographic area and the graphic area from the data remaining after the character area is extracted from the image data. Image processing device. 2. The area separating means further comprises a photograph area preceding extracting means for extracting a photograph area prior to a figure area from data remaining after the character area is extracted from the image data. The image processing apparatus according to claim 1. 3. The area separating means further comprises graphic area preceding extraction means for extracting a graphic area prior to a photographic area from the data remaining after the character area is extracted from the image data. The image processing apparatus according to claim 1. 4. The picture area preceding extraction means performs a first area division process capable of dividing a character area extracted from the image data and remaining data into a plurality of predetermined areas, and the divided area is obtained. A first photographic area extracting means for extracting a photographic area, and data remaining after extracting a photographic area by the first photographic area extracting means from data remaining after extracting a character area from the image data; And a second photographic region extracting means for performing a second region dividing process capable of dividing into a plurality of regions smaller than the predetermined region and extracting a photographic region from the divided regions. Item 2. The image processing device according to item 2. 5. The graphic area preceding extraction means performs a first area dividing process capable of dividing a character area extracted from the image data and remaining data, into a plurality of predetermined areas, and the divided area. First graphic area extracting means for extracting a graphic area, and data remaining after extracting the graphic area from the image data by the first graphic area extracting means. And a second figure area extracting means for performing a second area division process capable of dividing into a plurality of areas smaller than the predetermined area and extracting a figure area from the divided areas. Item 3. The image processing device according to item 3.