JP2017156982A - Image conversion program, image conversion device, and image conversion method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate structured data in which producibility of an original document is assured, retrieval property and re-editing property of the document are maintained, and degradation in visibility of the original is suppressed.SOLUTION: An image conversion device (image conversion program) acquires first image data of which an image reading part is made to read a document or an imaging part is made to image the document, applies a vectorizing process to the first image data for conversion to structured data, analyzes the structured data for obtaining object information and performing a rasterizing process on the structured data for re-conversion to second image data, compares the first image data with the second image data for extracting a differential part, based on an object information, specifies an object region in a specified range to which an object arranged to the differential part belongs, acquires image information corresponding to the object region from the first image data, updates the structured data by using the acquired image information, and outputs the updated structured data.SELECTED DRAWING: Figure 9

Description

  The present invention relates to an image conversion program, an image conversion apparatus, and an image conversion method, and more particularly to an image conversion program, an image conversion apparatus, and an image conversion method for generating structured data from image data.

  In recent years, in order to save resources, a document printed on paper is read by a scanner or the like, converted into image data, and stored. In addition, since image data cannot be used to search for a specific object in the original or to be edited again, it is also possible to convert the image data into vector data (also referred to as vector data) and save the vector data. ing. However, misrecognition may occur in the process of converting image data into vector data (referred to as vector conversion or vectorization).

  As a coping method when such erroneous recognition occurs, for example, Japanese Patent Application Laid-Open No. H10-260260 discloses an image processing apparatus that performs image processing of a document image obtained by reading a document, and a reading unit that reads the document, First conversion means for converting the original image read by the reading means into vector data, second conversion means for converting the vector data into image data, first image data of the original image, and the second conversion Comparing the first image data or the vector data as an electronic file corresponding to the document image based on the comparison result of the comparison means and the comparison result of the comparison means. An image processing apparatus including a selection unit for selecting is disclosed.

JP 2005-157450 A

  In the above-mentioned Patent Document 1, whether to save the entire document as image data or vector data is selected based on the comparison result between the original image data of the document and the converted image data. When saved in, text information and graphics information are lost, so the searchability and reeditability of the manuscript are lost.

  As another method, it is possible to replace only a portion that may be erroneously recognized in the vectorization process with image data. In this method, vector data and raster data are mixed in a predetermined area. Therefore, the visibility of the document may be deteriorated.

  The present invention has been made in view of the above problems, and its main purpose is to maintain the searchability and re-editability of the original document while ensuring the reproducibility of the original original document, and to visually check the original document. An object of the present invention is to provide an image conversion program, an image conversion apparatus, and an image conversion method capable of generating structured data in which deterioration of property is suppressed.

  One aspect of the present invention is an image conversion program that operates on an apparatus capable of performing vectorization processing and rasterization processing. The image conversion program causes the image reading unit to read an original on the image reading unit. A first process for acquiring first image data obtained by capturing an image of a document; a second process for converting the first image data into structured data by performing the vectorization process; and analyzing the structured data. A third process for obtaining object information, a fourth process for performing the rasterization process on the structured data and reconverting the second image data, and the first image data and the second image. A fifth process for comparing the data with each other and extracting the difference portion, and on the basis of the object information, the object area within a predetermined range to which the object arranged in the difference portion belongs is specified. A sixth process for performing, a seventh process for acquiring image information corresponding to the object region from the first image data, updating the structured data using the acquired image information, and the structured data after the update An eighth process of outputting data is executed.

  An image conversion apparatus according to an aspect of the present invention acquires an image reading unit or an imaging unit, and first image data obtained by reading an original by the image reading unit or first image data obtained by imaging the original by the imaging unit. A data acquisition unit, a vectorization processing unit that performs vectorization processing on the first image data and converts it into structured data, an analysis unit that analyzes the structured data and acquires object information, and Rasterization processing is performed on the structured data to compare the first image data with the second image data, and a difference portion is extracted. Based on the comparison unit and the object information, an object region in a predetermined range to which the object arranged in the difference portion belongs is specified, and the object area is determined from the first image data. Acquires image information corresponding to the object region, using the obtained image information and updating it said structured data, a data updating unit which outputs the structured data after updating, characterized in that it comprises a.

  One aspect of the present invention is an image conversion method in a system including a control device capable of executing vectorization processing and rasterization processing and an image reading unit or an apparatus including an imaging unit, and the control device includes the image reading unit. A first process for acquiring first image data obtained by reading a document or first image data obtained by imaging the document by the imaging unit, and performing the vectorization process on the first image data, A second process for converting to structured data; a third process for analyzing the structured data to obtain object information; and performing the rasterization process on the structured data to obtain second image data. Based on the object information, a fourth process for re-transforming, a fifth process for comparing the first image data and the second image data to extract a difference portion, and the object information. A sixth process for specifying an object area within a predetermined range to which an object placed in a part belongs, a seventh process for acquiring image information corresponding to the object area from the first image data, and the acquired image information And updating the structured data by using and executing the eighth process of outputting the updated structured data.

  According to the image conversion program, the image conversion apparatus, and the image conversion method of the present invention, while maintaining the reproducibility of the original document, the searchability and reeditability of the document are maintained, and the visibility of the document is deteriorated. Suppressed structured data can be generated.

  The reason is that the image conversion apparatus (image conversion program) acquires the first image data obtained by causing the image reading unit to read the original or the first image data obtained by causing the image pickup unit to capture the original, and obtains the first image. The data is vectorized and converted to structured data, the structured data is analyzed to obtain object information, the structured data is rasterized and converted to second image data, The first image data and the second image data are compared to extract a difference portion, and based on the object information, an object region in a predetermined range to which an object arranged in the difference portion belongs is specified, and the first image This is because the image information corresponding to the object region is acquired from the data, the structured data is updated using the acquired image information, and the updated structured data is output.

It is a schematic diagram which shows an example of the image conversion system which concerns on one Example of this invention. It is a schematic diagram which shows the other example of the image conversion system which concerns on one Example of this invention. It is a schematic diagram which shows the other example of the image conversion system which concerns on one Example of this invention. It is a schematic diagram which shows the other example of the image conversion system which concerns on one Example of this invention. It is a schematic diagram which shows the other example of the image conversion system which concerns on one Example of this invention. It is a schematic diagram which shows the other example of the image conversion system which concerns on one Example of this invention. It is a block diagram which shows the structure of the image converter which concerns on one Example of this invention. 1 is a block diagram illustrating a configuration of an image forming apparatus according to an embodiment of the present invention. It is a flowchart figure which shows operation | movement of the image conversion apparatus which concerns on one Example of this invention. It is an example of the input image which concerns on one Example of this invention. It is an example of the display image by the document creation application of the structured data which concerns on one Example of this invention. It is a figure explaining a mode that a character string is recognized from the text object which concerns on one Example of this invention. It is a figure explaining a mode that a figure is recognized from the graphics object which concerns on one Example of this invention. It is a figure which shows the example of a classification | category of the object area | region of the structured data of FIG. It is a figure which shows the rasterization result of the structured data which concerns on one Example of this invention. It is an example of the comparison result of the figure 1 which concerns on one Example of this invention. It is a figure which shows the comparison result of the figure 1 which concerns on one Example of this invention at a pixel level. It is a figure explaining the identification process of the object area | region of the update target which concerns on one Example of this invention, (a) is a figure which shows a different part, (b) is a figure which shows the position of the object arrange | positioned in a different part, (C) is a diagram showing an object region to be updated. It is an example of the display image by the document creation application of the structured data after the update which concerns on one Example of this invention. It is another example of the input image which concerns on one Example of this invention. It is a figure which shows the example of a classification | category of the object area | region of the structured data of FIG.

  As shown in the background art, a document printed on paper is read by a scanner or the like and converted into image data, and the image data is converted into vector data and stored. However, image data is converted into vector data. Misrecognition may occur in the vectorization process for conversion. To deal with this problem, Patent Document 1 selects whether to save the entire document as image data or vector data based on the comparison result between the original image data and the converted image data. When saved as image data, the searchability and reeditability of the document are lost. In addition, there is a method of replacing only erroneously recognized portions in the vectorization process with image data. However, in this method, vector data and raster data are mixed in a predetermined area, so that the visibility of the document may be deteriorated. There is.

  Therefore, in one embodiment of the present invention, the input image data is vectorized and converted to structured data, the structured data is rasterized and converted again to image data, and converted to input image data. A difference part is extracted by comparing with image data, an object area in a predetermined range to which an object arranged in the difference part belongs is specified, and the entire specified object area is updated using the input image data (structure) The object area specified in the structured data is replaced with image data, or instruction data for overwriting the object area specified in the structured data with the image data is created).

  In this way, by updating not only the part erroneously converted in the vectorization process of the input image data but also the entire object area of the predetermined range to which the object arranged in the part belongs is updated with the input image data. It is possible to generate structured data that maintains the retrievability and reeditability of the original while suppressing the deterioration of the original visibility while ensuring the reproducibility of the original.

  In order to describe the above-described embodiment of the present invention in more detail, an image conversion program, an image conversion apparatus, and an image conversion method according to an embodiment of the present invention will be described with reference to FIGS. 1 to 6 are schematic diagrams illustrating an example of an image conversion system according to the present exemplary embodiment, FIG. 7 is a block diagram illustrating a configuration of an image conversion apparatus, and FIG. 8 is a block diagram illustrating a configuration of an image forming apparatus. It is. FIG. 9 is a flowchart showing the operation of the image conversion apparatus of the present embodiment, and FIGS. 10 to 21 are diagrams for explaining the image conversion method of the present embodiment.

  As shown in FIG. 1, an image conversion system 10 according to this embodiment includes an image conversion device 20 that can execute vectorization processing and rasterization processing, and an MFP that includes an image reading unit that reads image data from a document printed on paper. And an image forming apparatus 30 such as (Multi-Functional Peripherals). These are connected using IEEE1394, Parallel, etc., or LAN (Local Area Network) or WAN (Wide Area Network) defined by standards such as Ethernet (registered trademark), Token Ring, FDDI (Fiber-Distributed Data Interface), etc. Etc. are connected via a network.

  In FIG. 1, the image conversion system 10 includes the image conversion apparatus 20 and the image forming apparatus 30. However, as shown in FIG. 2, the image conversion system 10 includes the image conversion apparatus 20 and an image reading apparatus 40 such as a scanner. You may comprise. Further, when image data is acquired by capturing an image of a document printed on paper with a camera or the like, the image conversion system 10 may be composed of an image conversion device 20 and an image pickup device 50 as shown in FIG. . When the image forming apparatus 30, the image reading apparatus 40, and the imaging apparatus 50 can execute the vectorizing process and the rasterizing process, the image conversion system 10 is connected to the image forming apparatus 30 or the image as illustrated in FIGS. 4 to 6. The reading device 40 and the imaging device 50 may be configured alone (that is, a configuration in which the image forming device 30, the image reading device 40, and the imaging device 50 function as an image conversion device). Hereinafter, each device will be described in detail on the assumption of the configuration of FIG.

[Image converter]
The image conversion device 20 is a computer device such as a personal computer, an image forming device 30, an image reading device 40, a control device that controls the imaging device 50, a portable terminal such as a smartphone or a tablet, and the like, as shown in FIG. As described above, the control unit 21, the storage unit 25, the network I / F unit 26, the display unit 27, and the operation unit 28 are configured.

  The control unit 21 includes a CPU (Central Processing Unit) 22 and a memory such as a ROM (Read Only Memory) 23 and a RAM (Random Access Memory) 24. The CPU 22 controls the control program (stored in the ROM 23 and the storage unit 25 ( A document display application capable of displaying structured data is developed in the RAM 24 and executed, thereby controlling the entire operation of the image conversion apparatus 20. Further, as shown in FIG. 7B, the control unit 21 functions as a data acquisition unit 21a, a vectorization processing unit 21b, an analysis unit 21c, a rasterization processing unit 21d, a comparison unit 21e, a data update unit 21f, and the like.

  The data acquisition unit 21a acquires, as an input image, image data (first image data) obtained by the image reading unit 38 scanning the document from the image forming apparatus 30. In the case of the system configuration of FIG. 2, the image reading device 40 acquires image data obtained by scanning a document. In the case of the system configuration of FIG. 3, an image obtained by the imaging device 50 capturing an image of the document. Get the data.

  The vectorization processing unit 21b performs a known area determination process on the acquired input image, classifies the input image into a text area, a graphics area, and an image area, and then performs a known vectorization process, Convert to vector data. Specifically, the text area is converted into a text code by performing a known OCR (Optical Character Recognition) process, and the graphics area is converted into a graphic drawing command. Then, the vectorize processing unit 21b converts the vector data into structured data that can be displayed by the document display application based on the result of the vectorize process. The document display application is, for example, Microsoft (registered trademark) Word (registered trademark), Excel (registered trademark), PowerPoint (registered trademark), Adobe (registered trademark) Acrobat (registered trademark), or the like. The structured data is data described in PDF (Portable Document Format), ODF (Open Document Format), OOXML (Office Open XML) format, and the like, and is data in which attributes are described for each object.

  The analysis unit 21c analyzes the structured data and acquires object information. Specifically, the object attributes and the drawing position information included in the structured data are acquired, and an object area obtained by dividing the drawing area of each attribute object is set based on the mutual relationship between the objects. Specifically, for a text region, a position such as a space, a comma, or a period is specified from the text code, a character string is recognized from the specified position, and position information of each character string is acquired. Then, a text area is set from the positional relationship of the recognized character string (upper and lower position information, and left and right object types). For the graphics area, the figure is recognized from the drawing command, and the position information of each figure is acquired. Then, the graphics area is set based on the connectivity and accessibility of each recognized figure.

  The rasterization processing unit 21d performs a known rasterization process on the structured data generated by the vectorization processing unit 21b by RIP (Raster Image Processor), and reconverts the image data (second image data). .

  The comparison unit 21e compares the input image data (first image data) with the reconverted image data (second image data), and extracts a difference portion. The comparison of the image data is performed for each object region set by the analysis unit 21c in pixel units, and the number of pixels having a difference (or the ratio of the number of pixels having a difference to the total number of pixels in the object region) is calculated. If there is a part exceeding a predetermined threshold, the part is extracted as a difference part. The threshold value may be set to a different value depending on the attribute of the object. In addition, the comparison unit 21e displays a comparison result (an image clearly showing the difference portion) between the input image data and the reconverted image data on the display unit 27 as necessary, and the difference portion will be described later. To select whether to update the structured data to be updated.

  The data update unit 21f determines whether or not a difference portion has been extracted. If the difference portion has been extracted, the data update unit 21f specifies an object region to which an object placed in the difference portion belongs based on the object information, and the input image Then, the image information corresponding to the specified object region is acquired, the structured data is updated using the acquired image information, and the updated structured data is output (for example, stored in the storage unit 25 or the like). Specifically, the instruction data for replacing the object area specified in the structured data with the input image corresponding to the object area or overwriting the input area corresponding to the object area in the object area specified in the structured data. Or create. Further, the data updating unit 21f displays the result of updating the specified object region with the input image (a display image in which the structured data after the update is displayed by the document display application) on the display unit 27 as necessary. A selection operation for accepting whether or not to adopt the data update is accepted.

  The data acquisition unit 21a, the vectorization processing unit 21b, the analysis unit 21c, the rasterization processing unit 21d, the comparison unit 21e, and the data update unit 21f may be configured as hardware, or the control unit 21 may be configured as the data acquisition unit 21a. The image conversion program may be configured to function as the vectorization processing unit 21b, the analysis unit 21c, the rasterization processing unit 21d, the comparison unit 21e, and the data update unit 21f, and the CPU 22 may execute the image conversion program.

  The storage unit 25 is configured by an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, a program for the CPU 22 to control each unit, information on the processing function of the own device, input image data, a vectorization processing unit The structured data converted by 21b, the image data reconverted from the structured data by the rasterizing processing unit 21d, the comparison result of the comparing unit 21e, the structured data updated by the data updating unit 21f, and the like are stored.

  The network I / F unit 26 includes a NIC (Network Interface Card), a modem, and the like, connects the image conversion apparatus 20 to the image forming apparatus 30, and acquires image data from the image forming apparatus 30.

  The display unit 27 includes a liquid crystal display (LCD), an organic EL (electroluminescence) display device, and the like. The comparison result of the comparison unit 21e (an image clearly showing the difference) and the update result of the data update unit 21f (Displayed image of the updated structured data displayed by the document display application) or the like is displayed.

  The operation unit 28 includes a mouse, a keyboard, and the like, and enables a selection operation for the comparison result of the comparison unit 21e, a selection operation for the update result of the data update unit 21f, and the like.

[Image forming apparatus]
The image forming apparatus 30 is an MFP or the like, and includes a control unit 31, a storage unit 35, a network I / F unit 36, a display operation unit 37, an image reading unit 38, a print processing unit 39, and the like as shown in FIG. Is done.

  The control unit 31 includes a CPU 32 and a memory such as a ROM 33 and a RAM 34. The CPU 32 develops and executes the control program stored in the ROM 33 and the storage unit 35 in the RAM 34, thereby performing the operation of the entire image forming apparatus 30. Control.

  The storage unit 35 is configured by an HDD, an SSD, or the like, and stores a program for the CPU 32 to control each unit, information on processing functions of the own apparatus, image data read by the image reading unit 38, and the like.

  The network I / F unit 36 includes a NIC, a modem, and the like, connects the image forming apparatus 30 to the image conversion apparatus 20, and transmits image data and the like to the image conversion apparatus 20.

  The display operation unit 37 is a touch panel provided with a pressure-sensitive operation unit (touch sensor) in which transparent electrodes are arranged in a grid pattern on the display unit, displays various screens related to printing processing, and performs various operations related to printing. to enable.

  The image reading unit 38 is a part that optically reads image data from a document placed on a document table, and a light source that scans the document and a CCD (Charge Coupled) that converts light reflected by the document into an electrical signal. Devices) and image sensors such as CMOS (Complementary Metal Oxide Semiconductor), and A / D converters for A / D converting electrical signals output from the image sensors.

The print processing unit 39 is a print engine that executes print processing. Specifically, the photosensitive drum charged by the charging device is irradiated with light corresponding to the image from the exposure device to form an electrostatic latent image, and the toner charged by the developing device is attached and developed. The image is primarily transferred to the transfer belt, secondarily transferred from the transfer belt to the paper, and a fixing device fixes the toner image on the paper.
[Image reading device]
The image reading device 40 is a scanner or the like provided with an image reading unit, and the image reading unit optically receives image data from a document placed on a document table, similarly to the image reading unit 38 of the image forming device 30. read. Specifically, it is composed of a light source that scans a document, an image sensor such as a CCD or CMOS that converts light reflected from the document into an electrical signal, an A / D converter that A / D converts the electrical signal, and the like. The
[Imaging device]
The imaging device 50 is a digital camera or the like including an imaging unit, and the imaging unit images a document and optically reads image data. Specifically, it includes an optical system such as a lens and a finder, an image sensor such as a CCD or CMOS, and an A / D converter that performs A / D conversion on an electrical signal output from the image sensor.

  1 to 8 show an example of the image conversion system 10 of the present embodiment, and the configuration thereof can be changed as appropriate. For example, FIG. 7 shows a case where the control unit 21 of the image conversion apparatus 20 is caused to function as the data acquisition unit 21a, vectorization processing unit 21b, analysis unit 21c, rasterization processing unit 21d, comparison unit 21e, and data update unit 21f. The control unit 31 of the image forming apparatus 30 (or the control unit of the image reading device 40 or the imaging device 50) functions as a data acquisition unit, vectorization processing unit, analysis unit, rasterization processing unit, comparison unit, and data update unit ( The image forming apparatus 30, the image reading apparatus 40, and the imaging apparatus 50 may have a data acquisition unit, a vectorization processing unit, an analysis unit, a rasterization processing unit, a comparison unit, and a data update unit.

  Hereinafter, an image conversion method using the image conversion apparatus 20 of the image conversion system 10 configured as shown in FIG. 1 will be described. The CPU 22 executes the processing of each step shown in the flowchart of FIG. 9 by developing the image conversion program stored in the ROM 23 or the storage unit 25 in the RAM 24 and executing it.

  First, the control unit 21 (data acquisition unit 21a) captures image data obtained by scanning the document by the image reading unit 38 or the image reading device 40 of the image forming apparatus 30 via the network I / F unit 26, and imaging. The apparatus 50 acquires image data obtained by imaging a document as an input image (S101). FIG. 10 shows an example of the input image 60. The input image 60 includes a text object, a graphics object, and an image object.

  Next, the control unit 21 (vectorize processing unit 21b) performs region discrimination processing on the input image, and the input image is converted into a text region 60a, a graphics region 60b, and an image region 60c as shown in FIG. Then, vectorization processing is performed and converted into vector data (S102). Specifically, the area determined to be the text area 60a is converted into a text code by performing a known OCR process. The area determined to be the graphics area 60b is converted into a graphic drawing command.

  Next, the control unit 21 (vectorization processing unit 21b) converts the vector data into structured data that can be displayed by the document display application based on the result of the vectorization processing (S103). As described above, the structured data is data described in the format of PDF, ODF, or OOXML. FIG. 11 shows a display image 61 of structured data (display result when displayed by a document display application). In this example, two places surrounded by a thick broken line are erroneously converted in the vectorization process, the characters “B & W” are erroneously converted to “BBW”, and the ellipse figure is erroneously converted to a perfect circle.

  Next, the control part 21 (analysis part 21c) analyzes structured data, and acquires object information (S104). Specifically, the object attributes and drawing position information included in the structured data are acquired, and the object areas (text area and graphic area) obtained by dividing the drawing area of each attribute object based on the interrelationship between the objects. Set the area. This object area is an area that defines a range to be replaced with an input image by data update described later (to be updated).

  FIG. 12 shows an example of setting a text area. For example, taking the sentence in FIG. 12A as an example, each character is converted into a text code by OCR processing. Next, a character string is recognized from the position of a space, a comma, or a period. FIG. 12B shows the result of recognizing the character string. In this example, nine character strings are recognized. Then, a text area (for example, a text area for each line) to be updated is set from the positional relationship (upper and lower position information, left and right object types) of the recognized character strings.

  FIG. 13 shows a setting example of the graphics area. For example, the figure in FIG. 13A will be described as an example. This figure is an object at the lower left of the display image 61 of the structured data in FIG. 11, and as shown in FIG. It is expressed by 5 figures. Then, a graphics area (here, a graphics area including five figures) to be updated is set based on connectivity / accessibility of each figure.

  FIG. 14 is a diagram in which the structured data display image 61 of FIG. 11 is classified by object region. In this example, it is classified into 12 text areas (Text1 to Text12), 1 image area (Image1), and 2 graphics areas (Graphics1 to Graphics2). The classification in FIG. 14 is an example and can be changed as appropriate. For example, in FIG. 14, the text objects are classified for each line, but the text objects are classified for each paragraph, and Text1 to Text7 and Text8 to Text12 are each set as one text area, or up to a period is one text area. Or you may.

  Next, the control unit 21 (rasterization processing unit 21d) performs rasterization processing on the structured data to form an image, and reconverts it into image data (second image data) (S105). FIG. 15 is a diagram showing a rasterized result 62 of structured data. The appearance is basically the same as the display image 61 of the structured data in FIG. 11 (display result when displayed by the document display application).

  Next, the control unit 21 (comparison unit 21e) compares the input image 60 (first image data) in FIG. 10 with the rasterized result 62 (second image data) of the structured data in FIG. A difference portion is extracted (S106). This comparison of image data is performed for each object area shown in FIG.

  FIG. 16 shows a comparison result focusing on the graphic 1 in FIG. 13. When the input image 60 in FIG. 10 is compared with the rasterized result 62 of the structured data in FIG. FIG. 17 is a diagram comparing FIG. 16 in units of pixels. In this example, a thick solid square is a pixel constituting the graphic 1 (circular circle) in FIG. 15, and a hatched portion is a pixel constituting the horizontal curve of the graphic 1 (ellipse) in FIG. In the example of FIG. 17, a perfect circle is formed by 100 pixels. If a pixel that intersects the horizontal curve is a matching pixel, 13 pixels are matched and a pixel that does not intersect the horizontal curve is a mismatched pixel. , 87 pixels do not match. As a result, the mismatch rate is 87% for the pixels forming the perfect circle. Here, if the threshold of the ratio of non-matching pixels to all the pixels is 20%, the ratio of this region exceeds the threshold, so that it is determined as a difference portion. Here, the ratio of the mismatched pixels to all the pixels is compared with the threshold value, but the number of mismatched pixels may be compared with the threshold value. These threshold values can be individually set according to the attributes of the object.

  Next, the control unit 21 (data update unit 21f) determines whether a difference portion has been extracted (S107). When the difference portion is not extracted, the control unit 21 (data update unit 21f) outputs the structured data as it is (for example, stores it in the storage unit 25) (S111). On the other hand, when the different part is extracted, the control unit 21 (data update unit 21f) specifies the object area to which the object arranged in the different part belongs (S108). 18A and 18B are diagrams for explaining the object area specifying process. FIG. 18A shows the extracted difference portion, and FIG. 18B shows the position of the object placed in the extracted difference portion. Is shown. From the position in FIG. 18B, the object area to which the object belongs is specified. FIG. 18C shows the identified object area (Text9, Graphics1) by hatching.

  Next, the control unit 21 (data update unit 21f) acquires image information corresponding to the specified object region from the input image (S109), and updates the structured data using the acquired image information (S110). . FIG. 19 shows an example of a display image 63 (display result when displayed by the document display application) of the structured data after update in which the specified object region (Text9, Graphics1) is replaced with image information of the input image. Here, in order to make the updated portion of the structured data easy to understand, the portion replaced with the image information of the input image is highlighted with a bold line. Thereafter, the control unit 21 (data update unit 21f) outputs (for example, stores in the storage unit 25) the updated structured data (S111).

  In this way, not only the part erroneously converted by the vectorization processing of the input image data but also the entire object area to which the object placed in the part belongs is updated with the input image data, and the related area is made the same. Unify the attribute objects to maintain the reproducibility of the original document, maintain the searchability and reeditability of the document, and generate structured data that suppresses deterioration of the document visibility. Can do.

  In the above flow, the image conversion program automatically replaces the object area of the structured data with the input image, but the control unit 21 (comparison unit 21e) displays the comparison result (image that clearly shows the difference). The control unit 21 (the data update unit 21f) displays the update result (update) by displaying on the unit 27 and allowing the user to select whether or not to update the structured data for the difference portion (accepting the selection operation). A display image of the subsequent structured data displayed by the document display application) is displayed on the display unit 27, and the user is allowed to select whether or not to update the structured data (selection operation is accepted). Also good.

  Further, in specifying the object area to be updated, when the character string included in the image area is determined to be a text area, the entire image area including the text area can be replaced with the input image. FIG. 20 is an example of an input image in which a text region (a character string “bizhub PRESS C8000”) is included in the image region. FIG. 21 illustrates classification of structured data obtained by vectorizing and converting the input image into object regions. FIG. In this example, since the area (text area) of Text8 and Text9 is included in the area (image area) of Image1, if a difference portion is extracted from Text8 or Text9, the structure is obtained by replacing Image1 in the structured data. Data can be updated.

  In addition, this invention is not limited to the said Example, The structure and control can be changed suitably, unless it deviates from the meaning of this invention.

  For example, in the above-described embodiment, a document including a text object, a graphics object, and an image object and a document including a text object and an image object have been described as examples. The image conversion method of the invention can be similarly applied.

  In the above embodiment, the case where the updated structured data is stored in the storage unit 25 of the image conversion apparatus 20 is shown. However, the updated structured data is stored in an external storage device (connected to the image conversion apparatus 20 ( It may be stored in a cloud server).

  The present invention relates to an image conversion program for generating structured data from image data, a recording medium on which the image conversion program is recorded, an image conversion apparatus on which the image conversion program operates, and an image conversion in an image conversion system including the image conversion apparatus. Available to the method.

DESCRIPTION OF SYMBOLS 10 Image conversion system 20 Image conversion apparatus 21 Control part 21a Data acquisition part 21b Vectorization process part 21c Analysis part 21d Rasterization process part 21e Comparison part 21f Data update part 22 CPU
23 ROM
24 RAM
25 Storage Unit 26 Network I / F Unit 27 Display Unit 28 Operation Unit 30 Image Forming Apparatus 31 Control Unit 32 CPU
33 ROM
34 RAM
35 Storage Unit 36 Network I / F Unit 37 Display Operation Unit 38 Image Reading Unit 39 Print Processing Unit 40 Image Reading Device 50 Imaging Device 60, 70 Input Image 60a Text Area 60b Graphics Area 60c Image Area 61 Display Image of Structured Data 62 Rasterization result of structured data 63 Display image of structured data after update

Claims (23)

An image conversion program that operates on an apparatus capable of performing vectorization processing and rasterization processing,
In the device,
A first process for acquiring first image data obtained by causing an image reading unit to read a document or first image data obtained by causing an image pickup unit to capture an image of a document;
A second process for converting the first image data into structured data by performing the vectorization process;
Analyzing the structured data to obtain object information;
A fourth process for performing the rasterization process on the structured data and re-converting the structured data into second image data;
A fifth process for comparing the first image data with the second image data and extracting a difference portion;
A sixth process for identifying a predetermined area of the object region to which the object arranged in the difference portion belongs based on the object information;
A seventh process for obtaining image information corresponding to the object area from the first image data;
Updating the structured data using the acquired image information, and executing the eighth process of outputting the updated structured data;
An image conversion program characterized by that. In the third process, the attribute and drawing position of the object included in the structured data, and information on the object area obtained by dividing the drawing area of the object of each attribute are acquired.
The image conversion program according to claim 1, wherein: In the third process, the object area is set based on the mutual relationship of the objects included in the structured data.
The image conversion program according to claim 2, wherein: In the fifth process, for each object area, a part where the number or ratio of pixels having a difference exceeds a predetermined threshold is extracted.
The image conversion program according to claim 2 or 3, wherein In the fifth process, the threshold value is set according to the attribute of the object.
The image conversion program according to claim 4, wherein: In the sixth process, when the difference portion is a text area and the text area is included in the image area, the image area is specified as the object area.
The image conversion program according to any one of claims 2 to 5, wherein In the fifth process, an image clearly showing the difference portion is displayed on a display unit, and a selection operation as to whether or not to update the structured data is accepted.
An image conversion program according to any one of claims 1 to 6, wherein In the eighth process, the display image of the structured data after update is displayed on the display unit, and a selection operation as to whether or not to adopt the update of the structured data is accepted.
8. The image conversion program according to claim 1, wherein the image conversion program is any one of claims 1 to 7. The structured data is data described in a format of PDF (Portable Document Format), ODF (Open Document Format), or OOXML (Office Open XML).
The image conversion program according to any one of claims 1 to 8, wherein An image reading unit or an imaging unit;
A data acquisition unit for acquiring first image data obtained by reading the original document by the image reading unit or first image data obtained by imaging the original document by the imaging unit;
A vectorization processing unit that performs vectorization processing on the first image data and converts the first image data into structured data;
Analyzing the structured data to obtain object information;
A rasterization processing unit that performs rasterization processing on the structured data and reconverts the second image data;
A comparison unit that compares the first image data and the second image data and extracts a difference portion;
Based on the object information, a predetermined range of an object region to which an object arranged in the difference portion belongs is specified, image information corresponding to the object region is acquired from the first image data, and the acquired image A data update unit that updates the structured data using information and outputs the updated structured data.
An image conversion apparatus characterized by that. The analysis unit acquires information on an object area and a drawing position of the object included in the structured data, an object area obtained by dividing a drawing area of the object of each attribute,
The image conversion apparatus according to claim 10. The analysis unit sets the object area based on the interrelationship of objects included in the structured data.
The image conversion apparatus according to claim 11. The comparison unit extracts, for each object region, a portion where the number or ratio of pixels having a difference exceeds a predetermined threshold value,
The image conversion apparatus according to claim 11 or 12, characterized in that: The comparison unit sets the threshold according to the attribute of the object.
The image conversion apparatus according to claim 13. The data update unit is configured to specify the image area as the object area when the difference portion is a text area and the text area is included in the image area;
The image conversion apparatus according to claim 11, wherein the image conversion apparatus is an image conversion apparatus. The structured data is data described in a format of PDF (Portable Document Format), ODF (Open Document Format), or OOXML (Office Open XML).
The image conversion apparatus according to claim 10, wherein the image conversion apparatus is an image conversion apparatus. An image conversion method in a system including a control device capable of executing vectorization processing and rasterization processing and a device including an image reading unit or an imaging unit,
The controller is
A first process for acquiring first image data obtained by causing the image reading unit to read a document or first image data obtained by causing the imaging unit to capture an image of the document;
A second process for performing the vectorization process on the first image data and converting it to structured data;
A third process for analyzing the structured data and obtaining object information;
A fourth process for performing the rasterization process on the structured data and re-converting it into second image data;
A fifth process for comparing the first image data with the second image data and extracting a difference portion;
Based on the object information, a sixth process for specifying a predetermined range of the object region to which the object arranged in the difference portion belongs;
A seventh process of acquiring image information corresponding to the object region from the first image data;
Updating the structured data using the acquired image information, and outputting the updated structured data.
An image conversion method characterized by that. In the third process, the attribute and drawing position of the object included in the structured data, and information on the object area obtained by dividing the drawing area of the object of each attribute are acquired.
The image conversion method according to claim 17. In the third process, the object area is set based on the mutual relationship of the objects included in the structured data.
The image conversion method according to claim 18, wherein: In the fifth process, for each object area, a part where the number or ratio of pixels having a difference exceeds a predetermined threshold is extracted.
20. The image conversion method according to claim 18 or 19, wherein: In the fifth process, the threshold value is set according to the attribute of the object.
The image conversion method according to claim 20, wherein: In the sixth process, when the difference portion is a text area and the text area is included in the image area, the image area is specified as the object area.
The image conversion method according to any one of claims 18 to 21, wherein the image conversion method is performed. The structured data is data described in a format of PDF (Portable Document Format), ODF (Open Document Format), or OOXML (Office Open XML).
The image conversion method according to any one of claims 17 to 22, wherein the image conversion method is performed.

Images