JP2018005548A - Image processing device, image processing program, data server and image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of reducing malfunction of OCR through a simple method, an image processing program, a data server, and an image processing system.SOLUTION: An image processing device 1 communicably connected to an OCR device 5 for generating text data from character image data comprises: a discrimination unit 12 which discriminates an illustration image region from image page data acquired through reading of a printed matter by a scanner 4; an elimination unit 15 which eliminates from the image page data the illustration image region discriminated by the discrimination unit 12; a transmission unit 16 which transmits, to the OCR device 5, post-elimination image page data in which the illustration image region has been eliminated by the elimination unit 15; and a reception unit 17 which receives, from the OCR device 5, post-conversion page data obtained by converting the character image data included in the post-elimination image page data into text data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing program, a data server, and an image processing system.

Currently, there is an increasing demand for services by converting the contents of books such as libraries into electronic data such as character data, and converting the electronic data into other format data such as voice data. For example, conversion to audio data is expected to be applied to reading services for visually impaired people.
As a method for realizing digitization of a book, an apparatus for scanning and digitizing a printed matter such as an MFP (Multifunction Peripheral) has been known. A text image in an image scanned using an OCR (Optical Character Reader) function can be converted into text data.

However, when the scanned image includes an image such as an illustration that is not a text, there is a problem that the OCR malfunctions and the literacy rate decreases.
FIG. 7A shows an example in which image page data including illustration images 81 (81a, 81b) and text images 82 (82a to 82c) is converted using the OCR function. The text image 82 (82a to 82c) is converted into text data 92 (92a to 92c) by the OCR function. However, the illustration image 81 (81a, 81b) becomes data 91 (91a, 91b) in which the illustration is recognized as a character or garbled. For this reason, the literacy rate of the converted page data 90 is lowered.

  In order to reduce malfunction of OCR, a text image is extracted and an OCR function is used on the extracted text image (for example, Patent Document 1).

JP-A-2015-204015

  The technique described in Patent Document 1 extracts a text image, but the user needs to specify the text image. Further, it is necessary to extract the text image in units of documents, and the processing is complicated.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image processing program, a data server, and an image processing system that can reduce OCR malfunction by a simpler method.

The present invention solves the above problems by the following means.
A first invention is an image processing device communicably connected to a character recognition device that generates text data from character image data, wherein non-characters are obtained from image page data obtained by optically reading a print medium. An image area discriminating means for discriminating an image area, an erasing means for erasing the non-character image area discriminated by the image area discriminating means from the image page data, and the non-character image area being erased by the erasing means Transmitting means for transmitting post-erase image page data to the character recognition device, and receiving post-conversion page data obtained by converting character image data included in the post-erase image page data into text data from the character recognition device And an image processing apparatus.
According to a second aspect of the present invention, there is provided an image region determination unit that determines a non-character image region from image page data obtained by optically reading a print medium, and the non-character image region that is determined by the image region determination unit. Erasing means for erasing from page data, and character recognition for generating converted page data by converting character image data contained in the erased image page data from which the non-character image area has been erased by the erasing means to text data And an image processing apparatus.
According to a third aspect of the present invention, in the image processing device according to the first or second aspect, the image area determination unit performs a dilation / expansion process on the image page data to include a plurality of pieces included in the image page data. Extracting each contour for the element, drawing a circumscribed rectangle of the contour equal to or greater than a threshold, and determining the non-character image region by at least one of the area and the length of the vertical and horizontal sides; An image processing apparatus is characterized.
According to a fourth aspect of the present invention, in the image processing device according to any one of the first to third aspects, the erasing unit fills the non-character image area with a background color of the image page data, or An image processing apparatus, wherein a non-character image area is erased from the image page data by cutting out a character image area from the image page data.
According to a fifth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects, the position information of the non-character image area determined by the image area determination unit is acquired, and the acquired position information And a non-character image data generating means for generating non-character image data in which the non-character image region is associated with each other.
According to a sixth invention, in the image processing device according to the fifth invention, the non-character image region is arranged in the converted page data based on the non-character image data generated by the non-character image data generating means. An image processing apparatus comprising: a document generation unit that generates an electronic document.
The seventh invention is an image processing program for causing a computer to function as any one of the image processing apparatuses from the first invention to the sixth invention.
An eighth invention is a data server communicably connected to the image processing device of the fifth or sixth invention, wherein the converted page data received from the image processing device and the non- A data server comprising a database that stores character image data in association with the image page data.
A ninth invention is an image processing apparatus according to any one of the first to sixth inventions, and an image generating apparatus that optically reads a print medium and generates image page data to be analyzed by the image processing apparatus And an image processing system.
A tenth aspect of the present invention is the image processing apparatus according to any one of the first aspect and the third to sixth aspects of the invention, and image page data analyzed by the image processing apparatus by optically reading a print medium. Image processing comprising: an image generating device to generate; and a character recognition device that converts character image data included in the post-erase image page data output by the image processing device into text data and generates post-conversion page data System.

  According to the present invention, it is possible to provide an image processing apparatus, an image processing program, a data server, and an image processing system that can reduce OCR malfunction by a simpler method.

1 is a diagram illustrating an overall configuration of an image processing system according to an embodiment. It is a figure which shows the functional block of the image processing apparatus which concerns on this embodiment. It is a figure which shows the file structural example of the data server which concerns on this embodiment. It is a flowchart which shows the image analysis process in the image processing apparatus which concerns on this embodiment. It is a flowchart which shows the illustration image discrimination | determination process in the image processing apparatus which concerns on this embodiment. It is a figure for demonstrating the process in the image processing apparatus which concerns on this embodiment. It is a figure for demonstrating the outline | summary of the conventional process and the process which concerns on this embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.
(Embodiment)
<Image processing system 100>
FIG. 1 is a diagram showing an overall configuration of an image processing system 100 according to the present embodiment.
FIG. 2 is a diagram illustrating functional blocks of the image processing apparatus 1 according to the present embodiment.
FIG. 3 is a diagram illustrating a file configuration example of the data server according to the present embodiment.

  An image processing system 100 shown in FIG. 1 includes an image processing apparatus 1, a scanner 4 (image generation apparatus), an OCR apparatus 5 (character recognition apparatus), and a data server 6, each via a communication network N. It is connected. In the image processing system 100, the scanner 4 scans each page of a printed matter (print medium) such as a book and outputs image page data. Then, in the image processing system 100, the image processing apparatus 1 generates post-erasure image page data obtained by erasing the illustration image (non-character image) from the image page data. Further, in the image processing system 100, the OCR device 5 recognizes the image page data after erasure and generates post-conversion page data obtained by converting the character image into text. Then, the image processing system 100 stores various data generated by this system in the data server 6. In this way, the image processing system 100 enables secondary use of the printed matter such as generating audio data using the data of the data server 6.

<Image processing apparatus 1>
The image processing apparatus 1 is a server that generates post-erasure image page data obtained by erasing an illustration image from image page data.
As illustrated in FIG. 2, the image processing apparatus 1 includes a control unit 10, a storage unit 30, and a communication interface unit 39.
The control unit 10 is a central processing unit (CPU) that controls the entire image processing apparatus 1. The control unit 10 executes various functions in cooperation with the hardware described above by appropriately reading and executing an operating system (OS) and application programs stored in the storage unit 30.

The control unit 10 includes a reception unit 11, a determination unit 12 (image area determination unit), a structuring unit 13, an illustration image registration unit 14 (non-character image data generation unit), and an erasing unit 15 (erasing unit). , A transmission unit 16 (transmission unit), a reception unit 17 (reception unit), and a data registration unit 18 (data recording unit).
The accepting unit 11 accepts image page data generated by the scanner 4.
The determination unit 12 determines an illustration image region (non-character image region) including an illustration image from the image page data.
The structuring unit 13 generates structured data of the image page data.
The illustration image registration unit 14 generates illustration image data (non-character image data) including an illustration image region and its position data, and stores it in the intermediate data storage unit 33.

The erasure unit 15 erases the illustration image area from the image page data, and generates post-erasure image page data.
The transmission unit 16 transmits the erased image page data to the OCR device 5.
The receiving unit 17 receives the converted page data from the OCR device 5. The post-conversion page data is the result of character recognition processing performed on the post-erase image page data. The character image (character image data) included in the post-erase image page data is converted to text (text data). It is a thing.
The data registration unit 18 causes the data server 6 to store the illustration image data stored in the intermediate data storage unit 33 and the converted page data received by the receiving unit 17.
Details of each process will be described later.

The storage unit 30 is a storage area such as a hard disk or a semiconductor memory element for storing programs, data, and the like necessary for the control unit 10 to execute various processes.
Here, the computer refers to an information processing device including a control unit, a storage device, and the like, and the image processing device 1 is an information processing device including the control unit 10, the storage unit 30 and the like, and is included in the concept of a computer. It is.
The storage unit 30 includes a program storage unit 31 and an intermediate data storage unit 33.

The program storage unit 31 is a storage area for storing various programs. The program storage unit 31 stores an image processing program 31a. The image processing program 31a is a program for performing various functions executed by the control unit 10 of the image processing apparatus 1.
The intermediate data storage unit 33 is a storage area for temporarily storing data generated by processing executed by the control unit 10 of the image processing apparatus 1.
The communication interface unit 39 is an interface unit for performing communication among the scanner 4, the OCR device 5, and the data server 6 via a communication network.

<Scanner 4>
Returning to FIG. 1, the scanner 4 is, for example, an MFP, and is an apparatus that scans and digitizes printed matter. The scanner 4 scans the printed matter page by page and generates image page data. Then, the scanner 4 transmits the generated image page data to the data server 6. Although not shown, the scanner 4 includes a control unit, a storage unit, a reading unit, a communication interface unit, and the like.

<OCR device 5>
The OCR device 5 is, for example, a personal computer (PC), and performs character recognition processing on the erased image page data to generate post-conversion page data. The OCR device 5 receives the image page data from the image processing device 1, generates converted page data, and transmits it to the image processing device 1. Although not shown, the OCR device 5 includes a control unit, a storage unit, a communication interface unit, and the like.

<Data server 6>
The data server 6 is a server that stores various data.
The data server 6 stores the image page data received from the scanner 4 in the storage device 60. The data server 6 stores the converted page data and illustration image data received from the image processing apparatus 1 in the storage device 60.
As shown in FIG. 3, the storage device 60 of the data server 6 sets one folder for each document that is a printed matter, and stores various data for each page in the folder. The data server 6 stores image page data as indicated by the “document B” folder before processing by the image processing apparatus 1. In addition, after the processing by the image processing apparatus 1, the data server 6, as indicated by the “document A” folder, the image page data, the converted page data and the illustration image data corresponding to the image page data. Are stored in association with each other. Although not shown, the data server 6 includes a control unit, a communication interface unit, and the like in addition to the storage device 60 shown in FIG.
The communication network N is a network between devices, and is an internet line or the like.

<Processing of Image Processing Device 1>
Next, processing of the image processing apparatus 1 will be described.
FIG. 4 is a flowchart showing image analysis processing in the image processing apparatus 1 according to the present embodiment.
FIG. 5 is a flowchart showing an illustration image discrimination process in the image processing apparatus 1 according to the present embodiment.
FIG. 6 is a diagram for explaining processing in the image processing apparatus according to the present embodiment.
FIG. 7 is a diagram for explaining an overview of conventional processing and processing according to the present embodiment.

In step S (hereinafter referred to as “S”) 10 in FIG. 4, the control unit 10 (accepting unit 11) of the image processing apparatus 1 acquires a file including image page data to be processed from the data server 6. For example, the control unit 10 acquires an unprocessed “document B” folder illustrated in FIG. 3.
In S11, the control unit 10 selects one image page data to be processed from the acquired file. Here, when the file includes a plurality of image page data, the control unit 10 performs processing one by one. For example, the control unit 10 acquires the “first page” folder included in the “document B” folder illustrated in FIG. 3.
In S12, the control unit 10 (discrimination unit 12) performs an illustration image discrimination process.

Here, the illustration image discrimination processing will be described with reference to FIG.
In S30 of FIG. 5, the control unit 10 converts the image page data to gray scale. The control unit 10 represents the fineness of gray gradation by, for example, 8 bits or 16 bits. Here, the following description will be made assuming that gray scale is used, but color may be used.
In S31, the control unit 10 acquires the paper color (margin color) of the image page data. For example, the control unit 10 acquires color information of the four corners of the image page data. The process of acquiring the paper color is performed for each image page data because there is an individual difference in the background color information and the paper color is different for each image page data.

In S32, the control unit 10 performs expansion / contraction processing on the image page data. As the expansion / contraction process, the control unit 10 blurs the character image (element) and the illustration image (element) included in the image page data, and repeats expansion and expansion / contraction.
FIG. 6A shows an example of the image page data 70. FIG. 6B shows the page data 71 after the expansion / contraction process after the image page data 70 is subjected to the expansion / contraction process. As shown in FIG. 6B, the character image and the illustration image included in the image page data 70 are expressed as a lump by the expansion / contraction process.

Returning to FIG. 5, in S <b> 33, the control unit 10 performs a binarization process on the page data after the expansion / contraction process, and extracts a contour.
In S <b> 34, the control unit 10 deletes the contour that is less than the threshold value. Here, the threshold value may be specified in advance by calculating an average value of the character size, or may be manually input before this processing is performed. By this processing, character images less than the threshold are excluded as not being illustration images. FIG. 6C shows the page data 72 being processed after the outline less than the threshold is deleted. The page data 72 being processed includes a contour 72a and a contour 72b.
In S35, the control unit 10 determines whether or not a contour remains in the page data being processed. When the outline remains (S35: YES), the control unit 10 moves the process to S36. On the other hand, if no contour remains (S35: NO), the control unit 10 ends this process and moves the process to FIG. The case where no outline remains means the case where the image page data consists only of a character image, for example.
In S <b> 36, the control unit 10 fills the inside of the outline into a lump and draws a circumscribed rectangle of the lump. FIG. 6D shows in-process page data 73 in which an outline rectangle is drawn. The page data 73 being processed includes circumscribed rectangles 73a and 73b.

  In S37, the control unit 10 selects an illustration image region based on the drawn circumscribed rectangle. The control unit 10 calculates the area of the circumscribed rectangle and the lengths of the vertical and horizontal sides of the circumscribed rectangle. Then, the control unit 10 selects an area having a calculated area larger than the average value of the character size to make an illustration image area. By this processing, the control unit 10 selects the circumscribed rectangle 73b shown in FIG. 6D as an illustration image region. In addition, the control unit 10 selects an image having a calculated length of a vertical or horizontal side longer than a predetermined value and sets it as an illustration image region. By this processing, the control unit 10 selects the circumscribed rectangle 73a shown in FIG. 6D as an illustration image region. Then, the control part 10 complete | finishes this process, and moves a process to FIG.

Returning to FIG. 4, in S <b> 13, the control unit 10 (structuring unit 13) generates structured data of the image page data. The structured data is, for example, data described in XML (eXtensible Markup Language), which is one of languages for describing structured documents. The structured data represents each image constituting the image page data using a predetermined start tag and end tag, and describes the arrangement position of each image.
In S14, the control unit 10 (illustration image registration unit 14) acquires position data of the illustration image region from the structured data generated in S13. Then, the control unit 10 causes the intermediate data storage unit 33 to store illustration image data including the illustration image region determined by the illustration image determination process (S12) and the position data of the illustration image region.

In S15, the control unit 10 (erasing unit 15) erases the illustration image area from the image page data by painting the illustration image area with paper color with respect to the image page data. Here, the paper color is acquired in the illustration image discrimination processing (S31 in FIG. 5). By this processing, the image page data after erasure after erasing the illustration image area does not include the illustration image and becomes only the character image.
In S <b> 16, the control unit 10 (transmission unit 16) transmits the erased image page data to the OCR device 5.

In the OCR device 5, the control unit of the OCR device 5 reads the erased image page data, performs character recognition processing, and generates post-conversion page data. Here, as described above, since the image page data after erasure is data of only a character image, the OCR device 5 can acquire text data from the character image by a general character recognition process. And since the acquired text data becomes a thing with few malfunctions, the improvement of a literacy rate can be anticipated.
FIG. 7B shows an example when the above-described processing is performed. The image processing apparatus 1 paints the illustration image 81 (81a, 81b) of the image page data 80 to the paper color. Then, the OCR device 5 performs OCR conversion on the page data 85 being processed that is only the text image 82 (82a to 82c) (character image data). In this figure, the filled portion of the page data 85 being processed is indicated by a dotted line for easy understanding. The OCR device 5 performs character recognition processing on the page data 85 being processed that does not include the illustration image 81, so that the obtained converted page data 90 is the text data 92 obtained by converting the text image 82 (82 a to 82 c). (92a to 92c) only. Therefore, even if conversion by OCR is performed, the illustration is not recognized as a character or garbled by the illustration image 81.
Then, the control unit of the OCR device 5 transmits the acquired converted page data to the image processing device 1.

In S <b> 17 of FIG. 4, the control unit 10 (receiving unit 17) receives post-conversion page data corresponding to the post-erase image page data from the OCR device 5.
In S <b> 18, the control unit 10 (data registration unit 18) transmits the illustration image data stored in the intermediate data storage unit 33 and the converted page data received by the receiving unit 17 to the data server 6.
In the data server 6, the control unit of the data server 6 stores the received illustration image data and converted page data in the storage device 60 in association with the image page data.
In S19, the control unit 10 determines whether or not all image page data included in the acquired file has been processed. When the process is performed on all the image page data (S19: YES), the control unit 10 ends this process. On the other hand, when all the image page data has not been processed (S19: NO), the control unit 10 moves the process to S11 and applies to the unprocessed image page data included in the acquired file. Repeat this process.

  By this processing, the image processing system 100 allows the image processing apparatus 1 to distinguish and erase only the illustration image from the image page data including the character image and the illustration image, and the erased image page data including only the character image is OCR. Transmit to the device 5. Therefore, post-conversion page data obtained as a result of character recognition processing by the OCR device 5 includes only text data obtained from a character image, so that the literacy rate can be improved.

Thus, according to the image processing system 100 of the present embodiment, the following effects are obtained.
(1) The image processing apparatus 1 generates post-erase image page data in which the illustration image area is deleted from the image page data, and causes the OCR apparatus 5 to perform processing. Therefore, the literacy rate can be improved by eliminating the illustration images in advance before the processing by the OCR device 5.
In addition, since the image processing apparatus 1 only needs to generate the image page data after erasure, there is no need to perform processing in units of documents, and the processing can be made easier.
(2) The image processing apparatus 1 performs an expansion / contraction process on the image page data to extract the respective outlines of the character image and the illustration image included in the image page data, and circumscribes a rectangle with an outline equal to or greater than a threshold value. Is drawn, and the illustration image region is determined by at least one of the area and the length of the vertical and horizontal sides. Therefore, the illustration image included in the image page data can be automatically extracted. Note that this illustration image area discrimination can be more effective in a practical book without plate-making data or the like in which characters are mainly used and a table or figure for explanation is inserted.

(3) In the image processing apparatus 1, the illustration image can be erased by painting the illustration image area on the background color of the image page data. By using the background color of the image page data, it is possible to cause the OCR device to perform processing on the image page data after erasure when there is no illustration image.
The OCR device 5 only needs to perform character recognition processing on the received image page data after erasure and generate post-conversion page data, and may have a general-purpose OCR function. Therefore, realization is easy.
(4) Secondary such as generating audio data from text data using the converted page data by storing the illustration image data and the converted page data in the data server 6 in association with the image page data. Can be used. Further, it is possible to utilize data such as creation of another teaching material using illustration image data and text data of converted page data.
(5) Since the image processing apparatus 1 performs processing in units of files, it can perform batch processing. The image processing apparatus 1 can automatically process a plurality of image page data in a file.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above. In addition, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. In addition, although embodiment mentioned above and the deformation | transformation form mentioned later can also be used in combination as appropriate, detailed description is abbreviate | omitted.

(Deformation)
(1) In the present embodiment, the image processing apparatus has been described by way of example in which the image page data after erasure is transmitted to the OCR apparatus and the converted page data generated by the OCR apparatus is received. However, the present invention is not limited to this. . The image processing apparatus may also have an OCR function. By doing so, only the image processing apparatus can perform processing up to generating page data after conversion from image page data.
(2) In this embodiment, the image processing apparatus has been described as an example in which the illustration image area is erased from the image page data by painting the illustration image area on the background color of the image page data. However, the present invention is not limited to this. . The illustration image may be deleted from the image page data by cutting out the illustration image area from the image page data.

  (3) In the present embodiment, the storage device of the data server has been described as storing various data, but the present invention is not limited to this. An electronic document may be generated using the converted page data and the illustration image data.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 4 Scanner 5 OCR apparatus 6 Data server 10 Control part 12 Discriminating part 14 Illustration image registration part 15 Erasing part 16 Transmission part 17 Receiving part 18 Data registration part 30 Storage part 31a Image processing program 60 Storage apparatus 100 Image processing system

Claims (10)

An image processing device communicably connected to a character recognition device that generates text data from character image data,
Image area discrimination means for discriminating a non-character image area from image page data obtained by optically reading a print medium;
Erasing means for erasing the non-character image area determined by the image area determining means from the image page data;
Transmitting means for transmitting the image page data after erasure in which the non-character image area has been erased by the erasing means to the character recognition device;
Receiving means for receiving, from the character recognition device, post-conversion page data obtained by converting character image data included in the post-erasure image page data into text data;
Providing
An image processing apparatus. Image area discrimination means for discriminating a non-character image area from image page data obtained by optically reading a print medium;
Erasing means for erasing the non-character image area determined by the image area determining means from the image page data;
Character recognition means for converting the character image data included in the post-erasure image page data from which the non-character image area has been erased by the erasure means into text data, and generating post-conversion page data;
An image processing apparatus comprising: The image processing apparatus according to claim 1 or 2,
The image area determination means performs expansion / contraction processing on the image page data to extract respective outlines for a plurality of elements included in the image page data, and draws a circumscribed rectangle having an outline equal to or greater than a threshold value. Determining the non-character image region by at least one of the area and the length of the vertical and horizontal sides;
An image processing apparatus. In the image processing device according to any one of claims 1 to 3,
The erasing means erases the non-character image area from the image page data by filling the non-character image area with a background color of the image page data or cutting the non-character image area from the image page data. To do,
An image processing apparatus. In the image processing device according to any one of claims 1 to 4,
Non-character image data generation means for acquiring position information of the non-character image area determined by the image area determination means and generating non-character image data in which the acquired position information and the non-character image area are associated with each other. Preparing,
An image processing apparatus. The image processing apparatus according to claim 5.
Based on the non-character image data generated by the non-character image data generation means, the document generation means for generating an electronic document in which the non-character image area is arranged in the converted page data;
An image processing apparatus.   An image processing program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 6. A data server communicably connected to the image processing apparatus according to claim 5 or 6,
A database for storing the converted page data and the non-character image data received from the image processing apparatus in association with the image page data;
A data server characterized by An image processing apparatus according to any one of claims 1 to 6,
An image generation device that optically reads a print medium and generates image page data to be analyzed by the image processing device;
An image processing system comprising: An image processing device according to any one of claims 1 and 3 to 6,
An image generation device that optically reads a print medium and generates image page data to be analyzed by the image processing device;
A character recognition device that converts character image data included in the post-erase image page data output by the image processing device into text data, and generates post-conversion page data;
An image processing system comprising:

Images