JP2009075879A - Image processor, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily grasp the configurations of an electronic document, and to perform edition for every image region or character region in the page of the document. <P>SOLUTION: A region extraction part 11 extracts an image region including an image and the image data of a character region including a character string from the page image data of electronic document data, and a registration part 13 registers the electronic document data and the page image data of the electronic document data and the region image data extracted by the region extraction part 11 about the page image data in a storage part 14 by associating them one another, and a retrieval display part 2 displays a document structure configured of the page configuration of the electronic document data and the region image data in a page on the basis of the data registered in the storage part 14 at a display part 20. A request control part 21 performs edition to the region image data, and performs rewriting for reflecting the edition content on the electronic document data of the storage part 14 through a retrieval part 22. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus including a computer, an image processing method, and a program.

With the rapid spread of computers, it has become common to store documents previously stored by filing paper as computerized information.
Conventionally, an image region including an image and a character region including a character string are recognized and extracted from a document in which images and character strings such as diagrams, tables, and photographs are mixed, and the extracted image is stored and extracted as a search target. There has been an information processing apparatus (see, for example, Patent Document 1) in which a stored image can be easily searched using characters as search text.
In such an information processing apparatus, necessary information can be searched from a large amount of information digitized.
Japanese Patent Laid-Open No. 11-25113

However, in the conventional information processing apparatus as described above, not only the document but also individual images and texts constituting the document are stored as digitized information. There has been a problem that it is difficult to manage documents because there is no means for confirming in what configuration electronic documents and images are stored.
Further, since no means for operating the extracted image itself is provided, there is a problem that the reusability is poor.
The present invention has been made in view of the above points, and can easily comprehend the structure of a digitized document and can edit it for each image area or character area in the page of the document. The purpose is to improve document management and reusability

In order to achieve the above object, the present invention provides the following image processing apparatus, image processing method and program.
(1) Extraction means for respectively extracting image data of an image area including an image and image data of a character area including a character string from page image data of the electronic document data, the electronic document data, and the page of the electronic document data Registration means for associating and registering image data and image data extracted by the extraction means for the page image data, and a page configuration of the electronic document data and an image area in the page based on the data registered by the registration means In an image processing apparatus comprising a display means for displaying a document structure consisting of an area structure of a character area and an edit means for editing the image data, the image data edited by the edit means is converted into the electronic data Selecting means for selecting whether or not to associate the page data of the document data with the selecting means; Means for associating the image data edited by the editing means with the page image data of the electronic document data instead of the image data associated with the page image data of the electronic document data. An image processing apparatus.

(2) In the image processing apparatus as described above, the editing unit performs editing for replacing the image data registered in association with the page image data of the electronic document data with other designated image data. If the image size of the specified other image data is different from the image size of the registered image data, the size of the image of the specified other image data is An image processing apparatus provided with adjusting means for matching the size of the image data.

(3) In the image processing apparatus as described above, the editing unit adds a plurality of pieces of image data registered in association with the page image data of the electronic document data to add editing into one image data. An image processing apparatus.
(4) In the image processing apparatus as described above, the editing means is an image processing means for performing editing for dividing image data registered in association with page image data of the electronic document data into a plurality of image data. apparatus.

(5) In the image processing apparatus as described above, the editing means is means for adding editing for adding new image data to a designated position in the page image data.
(6) In the image processing apparatus as described above, the editing means is means for adding editing for deleting image data registered in association with the page image data.

(7) An extraction process for extracting image data of an image area including an image and image data of a character area including a character string from page image data of the electronic document data, and the electronic document data and the page of the electronic document data A registration step for registering the image data and the image data extracted by the extraction step in association with each other, and a page configuration of the electronic document data and an image area in the page based on the data registered by the registration step An image processing method comprising: a display step for displaying a document structure comprising a region structure of a character region and a character region; an editing step for editing the image data; and the image data edited by the editing step for the electronic data A selection process for selecting whether or not to associate with page image data of document data, and the selection process In this case, instead of the image data associated with the page image data of the electronic document data, the image data edited by the editing step is associated with the page image data of the electronic document data. An image processing method provided with a process.

(8) In the image processing method as described above, the editing step includes a step of performing editing for replacing the image data registered in association with the page image data of the electronic document data with other designated image data. If the image size of the specified other image data is different from the image size of the registered image data, the size of the image of the specified other image data is An image processing method provided with an adjusting step for matching the size of the image data.

(9) In the image processing method as described above, the editing step is a step of adding a plurality of image data registered in association with the page image data of the electronic document data to make one image data. An image processing method.
(10) In the image processing method as described above, the editing step is an image processing in which editing is performed to divide image data registered in association with page image data of the electronic document data into a plurality of image data. Method.

(11) In the image processing method as described above, the editing step is a step of adding an edit for adding new image data to a designated position in the page image data.
(12) In the image processing method as described above, the editing step is a step of adding an edit for deleting image data registered in association with the page image data.

(13) An extraction procedure for extracting image data of an image area including an image and image data of a character area including a character string from page image data of the electronic document data, and the electronic document data and the electronic document. A registration procedure for registering the page image data of the data and the image data extracted by the extraction procedure in association with each other, and the page configuration and the page of the electronic document data based on the data registered by the registration procedure In a program for executing a display procedure for displaying a document structure composed of an image area and a character area, an editing procedure for editing the image data, and image data edited by the editing procedure To select whether or not to associate with the page image data of the electronic document data. When it is selected that the procedure is associated with the selection procedure, the image data edited by the editing procedure is replaced with the image data of the electronic document data instead of the image data associated with the page image data of the electronic document data. A program for executing a procedure associated with page image data.

(14) In the program as described above, the editing procedure is a procedure for adding editing to replace the image data registered in association with the page image data of the electronic document data with other designated image data. If the image size of the specified other image data is different from the image size of the registered image data, the size of the image of the specified other image data is registered. A program for executing an adjustment procedure that matches the image size of the image data.

(15) In the program as described above, the editing procedure is a procedure for adding a plurality of image data registered in association with the page image data of the electronic document data to make one image data. program.
(16) In the program as described above, the editing procedure is a procedure for performing editing for dividing image data registered in association with page image data of the electronic document data into a plurality of image data.

(17) In the program as described above, the editing procedure is a procedure for adding editing for adding new image data to a designated position in the page image data.
(18) In the program as described above, the editing procedure is a procedure for adding editing for deleting image data registered in association with the page image data.

The image processing apparatus and the image processing method according to the present invention can easily grasp the configuration of a digitized document and can edit each image area or character area in the page of the document so as to manage the document. And reusability can be improved.
In addition, the program according to the present invention allows the computer to easily grasp the configuration of the digitized document and edit it for each image area or character area in the document page. A function for enabling reusability to be improved can be realized.

Hereinafter, the best mode for carrying out the present invention will be specifically described with reference to the drawings.
〔Example〕
FIG. 1 is a block diagram showing a functional configuration of an image processing apparatus according to an embodiment of the present invention.
This image processing apparatus is a general information processing apparatus including a personal computer, and extracts a figure, a table, and a text from a document as an image, associates the extracted image with an extraction source page, and displays the document structure. is there.
The extracted image is defined as a region image.

This image processing apparatus includes a registration processing unit 1 and a search display unit 2.
The registration processing unit 1 optically reads a paper document to obtain digitized electronic document data, and an image area including images of diagrams and tables from page image data of the electronic document data. An area extraction unit 11 that extracts image data and image data of a character area including a character string (text), respectively, recognizes characters about the image data of the image area and the text and captions in the image data of the character area, and recognizes the recognition The text extraction unit 12 that extracts the text and the image data extracted by the region extraction unit 11 are associated with the text extracted by the text extraction unit 12, and the page image data of the extraction source is further associated with the storage device. The registration unit 13 is registered in the storage unit 14.

Details of the data registered by the registration unit 13 will be described later.
The registration processing unit 1 can handle documents of any format such as a document read by a scanner, a document created by word processing software, and a structured document including SGML, HTML, and XML.
On the other hand, the search display unit 2 includes a display unit 20 including an LCD as a user interface unit, a request control unit 21 that controls a series of flow from user request reception to response from an input device (not shown), and designation. The search unit 22 is configured to search for images and text from the storage unit 14 according to the keyword, and the display information generation unit 23 is configured to generate display data based on the search result and transmit the display data to the display unit 20.

[Description of data registered in the storage unit]
2 to 4 are detailed explanatory diagrams of information registered in the storage unit 14 by the registration unit 13 illustrated in FIG.
The information registered in the storage unit 14 includes electronic document data 30 that is stored document data itself, electronic document metadata 31 accompanying the electronic document data 31 (shown in FIG. 2), and page images constituting the electronic document data. Data 32, page image metadata 33 accompanying it (shown above in FIG. 3), and an image of a character region including an image region or character string including an image extracted from each page image data constituting electronic document data The area image data 34, which is data, and the area image metadata 35 (shown in FIG. 4) associated therewith are stored in association with each other.

First, the electronic document metadata 31 includes an electronic document ID for uniquely identifying electronic document data, a storage location indicating a registration location in the storage unit 14 for electronic document data, and how many pages of electronic document data are included. It consists of each data of the number of pages indicating whether it is a document.
Next, the page image metadata 33 specifies a page ID for uniquely specifying page image data, coordinate information for specifying an area of the page image data, and electronic document data including page image data. Data of area image information for specifying image data of image areas or character areas extracted from page image data.
When the coordinate information is, for example, (x0, y0, x1, y1), x0 is the X coordinate at the upper left of the image, y0 is the Y coordinate at the upper left of the image, and x1 with respect to the origin set in the page image data. Represents the X coordinate at the lower right of the image, and y1 represents the Y coordinate at the lower right of the image. In the case of page image metadata, x0 and y0 are always set to 0.

  Next, the area image metadata 35 corresponds to an area image ID for uniquely specifying image data (area image data) of an image area or character area (corresponding to an image area image ID for image data of an image area). The image data of the area corresponds to the character area image ID), the storage location indicating the registration location in the storage unit 14 of the area image data, and the image type indicating the type of the area image data (at least text, table, Data to be classified into the figure), affiliation information for specifying which page image data of which electronic document data the area image data is related to, and coordinate information for specifying the area in the page of the area image data Key text that is character string information extracted from the area image data, and hyperlink recognized as link information in the key text. Each consisting of data.

[Explanation of display examples]
Next, an example of a screen for displaying the document structure of the electronic document will be described with reference to FIGS.
First, FIG. 5 is a screen example when the user displays a list of electronic documents stored in the storage unit 14 via the search display unit 2.
With this screen, the user can refer to a list of electronic documents stored in the storage unit 14.
The thumbnail image displayed here is a link to a page list screen constituting the electronic document, and the page list screen is displayed when the user clicks on the thumbnail image.

For example, when the thumbnail image 40 of the electronic document in FIG. 5 is clicked, a page list screen is displayed as shown in FIG.
On this page list screen, as indicated by arrow 41, a list of page images for each page of the electronic document is displayed.
At this time, since a page is designated in the display target column 42 on the screen, each page image of the electronic document is displayed in a list.
Here, when a document tree is designated in the display target field 42 in the screen, a page list screen as shown in FIG. 7 is displayed, and as shown by an arrow 43, a page image for each page of the electronic document and the electronic document. And the area image in the page is displayed in a tree format for each page image.

[Description of display example in tree format]
As shown in FIG. 7, the tree format indicating the document structure of the electronic document has a three-stage structure. From the left in the figure, the electronic document, each page image of the electronic document data, and the region image for each page are displayed. Each thumbnail and metadata is displayed.
Here, the image of the first page of the document is displayed as an electronic document. However, since it overlaps with the first page of the page image, an icon indicating that the document is a document or another image may be displayed.
In addition, as metadata, a part of the metadata shown in FIG. 2 is displayed here, but all metadata may be displayed.
Further, if metadata is added by the user, it may be displayed.

Further, each image in the tree can be expanded and omitted, and by clicking an abbreviated icon (-icon) 44 displayed on the tree, an unnecessary area image can be omitted, or an expanded icon can be displayed. By clicking the (+ icon) 45, it is possible to expand the omitted area image.
Note that these expanded icons and abbreviated icons are not displayed for pages that do not have region images as child nodes.

If there is a hyperlink such as a URL as the metadata of the area image, it is displayed, and when the displayed hyperlink is selected, the page jumps to the Internet page to which the hyperlink is linked. It is also possible to refer to the linked site displaying the page.
In this embodiment, description is made on the assumption that screen display is performed in HTML, and the development and omission control of the region image is performed using Javascript (registered trademark). However, the implementation means is limited to this. is not.

[Explanation of tree formation display processing]
Next, processing when switching from the page list screen to the document tree screen will be described.
FIG. 8 is a flowchart showing processing when switching from the page list screen to the document tree screen.
For the first page in the page currently displayed in the page list shown in FIG. 6, for example, with reference to affiliation information that is metadata of the first page, an electronic document ID that is information for specifying the electronic document data to which the page belongs and Since the storage location of the electronic document is obtained, the electronic document data that matches the affiliation information is searched based on the information (step (indicated by “S” in the figure) 1).
For the electronic document data obtained by the above search, page image data is extracted from the electronic document data, the electronic document data is used as a root node, and the data of each page extracted from the electronic document data is used as a child node. Display data forming a tree is created (S2).

Next, for the extracted page image metadata, it is determined whether or not there is a value in the area image information of the page image metadata (S3).
If it is determined in step 3 that the value does not exist, this means that there is no region image related to the page, and this process is terminated without forming a tree. In step 4, the electronic document ID, which is information for specifying the region image extracted from the page image metadata, and the storage location of the region image are obtained in a list format. Therefore, the region image data corresponding to the list length is obtained based on the information. (Image data of image area or character area) is searched (S4).
Display data that forms a display tree with all region image data obtained by the search in step 4 as child nodes of the page is created (S5).

It is determined whether or not all the pages have been checked. If not checked, the process returns to step 5 to repeat the above process, and if all the pages have been checked, this process ends.
Through the processing described above, display data in a tree display with the electronic document as a root node is formed, and the display data is transmitted to the display unit 20, whereby the document structure of the electronic document (page configuration and image area or character area in each page) And the user can easily confirm the document structure of the electronic document by referring to the tree.

[Description of sorting area image]
Next, if the tree is formed without any rules in the process of step 5 in FIG. 8, the region image at the top of the page and the region image at the bottom of the page are displayed confusedly, and the page image and the region image are displayed. It may be difficult to understand the correspondence relationship.
In order to avoid this, it is preferable to perform sorting based on coordinate information that is metadata of the region image.

FIG. 9 is an explanatory diagram of sorting of region images.
Reference numerals 50 to 54 denote area images of image areas or character areas, respectively, and the order of the figure numbers means the display order based on the sorting result.
Here, when the upper left corner of the page is set to X coordinate 0 and Y coordinate 0, the display order is determined according to the following 1 and 2.
1. A region image having a Y coordinate starting point value close to 0 is preferentially displayed.
2. When the value of the start point of the Y coordinate is the same, the value of the start point of the X coordinate closer to 0 is displayed preferentially.
By following the above two points, it is possible to display the area images sorted in the order of the area images 50 to 54 on the tree.
Further, this sort is possible even when a display method other than the tree is adopted.

[Explanation of tree display expansion and omission display processing]
Next, by transmitting display data of tree display expansion or omission to the display unit 20, unnecessary area images can be omitted for the user, and only necessary information can be displayed.
FIG. 10 is a flowchart showing a process of omitting the area image being developed in the tree display.
In the tree display displayed on the display unit 20, it is determined whether or not an omission icon (-icon) attached to the page has been clicked (S11).

If it is determined in step 11 that the abbreviated icon has not been clicked, this process ends. If it is determined that the clicked icon has been clicked, it is determined whether an area image belonging to the page is being displayed (S12).
If it is determined in step 12 that the area image is being displayed, the area image belonging to the corresponding page is hidden and is not displayed on the screen (S13).
Further, the abbreviated icon is switched to the expanded icon and displayed (S14), and this process ends. Thus, the user can recognize that the area image is omitted by switching the icon to the expanded icon.

In addition, processing to expand the area image that is omitted in the tree display,
In the tree display displayed on the display unit 20, it is determined whether or not the expansion icon (+ icon) attached to the page has been clicked. If it is determined that the expansion icon has not been clicked, this processing is terminated and the click is performed. If it is determined that the display of the region image belonging to the corresponding page is omitted, and if it is determined that the display of the region image is omitted, the region image belonging to the corresponding page is expanded and displayed on the screen. Display it.
Further, the expanded icon is switched to the abbreviated icon and displayed, and this process ends.

Next, in this image processing apparatus, when displaying the page constituting the electronic document, the area of the area image is displayed in a rectangular shape, the specified area image is dragged and dropped, and attached to another application such as word processing software. Can do.
[Description of preview display screen]
For example, in the page list screen shown in FIG. 6, a link is pasted to the thumbnail image of each page, and when the thumbnail image is clicked, a preview screen of the page indicated by the thumbnail image is displayed.

[Description of preview screen]
FIG. 11 is a diagram illustrating an example of a preview screen.
In this preview screen, the area image area extracted from the page is displayed as a rectangular frame indicated by a wavy frame in the figure, and the user drags and drops the rectangular frame to another application to display the area image in the rectangular frame. Data can be attached.
At this time, if the image type of the area image is text, the attached image is not the area image but the key text of the area image metadata.
The rectangular frame is displayed with a black dotted line in the figure, but may be displayed with other colors and line types.
In this embodiment, the description is made assuming that a Java applet (registered trademark) is adopted as means for realizing drag and drop. However, the means for realizing the invention is not limited to this, and other means may be used. It may be used.

[Description of preview screen display processing]
Next, processing for switching from the page list screen to the preview screen will be described.
FIG. 12 is a flowchart of processing for switching from the page list screen to the preview screen.
A preview image of the page image data is acquired based on the storage location that is the metadata of the page specified by the user (S21).
Further, it is determined whether or not the region image information value exists in the page image metadata (S22).

If it is determined in step 22 that there is no value, this means that there is no region image related to the page, and this processing ends without forming a rectangular link area, but it is determined that a value exists. Then, since the electronic document ID that is information for specifying the region image extracted from the page and the storage location of the region image are obtained in a list format, all the region images for the list length are searched based on the information. (S23).
The coordinate information indicating the image area is acquired from the metadata of the entire area image obtained by the search (S24).
Based on the coordinate information indicating the acquired image area, a rectangular link area (rectangular frame) is formed on the preview image, the URI of the download destination based on the storage location is set as the link destination, and the area image is specified as the link destination (S25).
Thereby, a preview display with a rectangular link area can be performed for each area image in the page.

[Description of processing when dragging and dropping an area image]
Next, processing when the user drags and drops an area image of a page displayed as a preview will be described.
FIG. 13 is a flowchart illustrating processing when dragging and dropping an area image of a page displayed as a preview.
For example, it is determined whether or not any link area in the page shown in FIG. 11 (in the rectangular frame indicated by the wavy line in the figure) has been clicked by the user (S31).
If it is determined in step 31 that it has not been clicked, this process ends. If it is determined that it has been clicked, the image type of the metadata of the linked area image of the clicked link area is acquired, and based on this value It is determined whether the area image is a text image (S32).

If it is determined in step 32 that the linked area image is a text image, the key text registered in the linked area image is downloaded (S33).
On the other hand, if it is determined in step 32 that the linked area image is not a text image, that is, if the image type is image or table, the area image is downloaded (S35).
After the key text is downloaded by clicking in step 33 and the region image is downloaded by clicking in step 35, the data display position is moved in accordance with the drag and drop operation of the download data by the user, and the drop destination is set to another application. The key text or region image is attached to the data by the software, and this process is terminated.
In this way, the user can attach text or an image to another application by dragging and dropping the image in the link area (rectangular frame).

In the above description, the case where the structure of the electronic document in the image processing apparatus is displayed in the tree format has been described. However, the method for displaying the relationship between the electronic document, the page, and the region image is not limited to the tree format.
Next, the display format of the area image other than the tree will be described.
[Description of display formats other than tree format-1]
FIG. 14 is an explanatory diagram of a display format other than the tree format.
This display process can be performed by using the above-described content, and the description using the flowchart will be omitted, and only the display content will be described.
The screen illustrated in FIG. 14 is a screen that is displayed when “page + region image” is selected in the display target column 60, and is configured by three frames.

The upper frame 61 displays the file name of the electronic document “My Document”, the left frame 62 displays a list of thumbnail images of each page constituting the electronic document, and the frame extends from the center to the right. Reference numeral 63 denotes a list of area images related to the page selected in the thumbnail image.
In this screen, the thumbnail image of the third page of the electronic document is selected in the left frame 62, whereby a region image (image region image and character region) associated with the third page in the frame 63 extending from the center to the right portion. Image) is displayed.
Each area image has a link to the above-described preview screen, and when the user clicks one of the area images, it is possible to transition to the preview screen of the clicked area image.

At that time, the corresponding area may be highlighted on the preview screen based on the coordinate information of the transition source region image.
FIG. 15 is an explanatory diagram illustrating an example of area emphasis display on the preview screen.
In this preview screen, the corresponding area 64 is highlighted (indicated by a bold frame in the figure) based on the coordinate information of the transition source region image.

[Description of display format other than tree format-2]
FIG. 16 is an explanatory diagram of another example of the display format other than the tree format.
This display process can also be performed by using the above-described content, and the description using the flowchart will be omitted, and only the display content will be described.
The screen shown in FIG. 16 is a screen that is displayed when “region image” is selected in the display target column 60, and as shown by arrow 65, only the region images of each page are displayed as a list across the pages. ing.
As shown in this screen, when the display target is a region image, it is possible to further narrow down the region image metadata by the image type. In this figure, only the image whose image type is text is displayed. Is shown.
Here, when the user wants to cancel the narrowing down, it is possible to display a list of all image types by selecting all the image types.

Since the image processing apparatus displays a document structure including a page configuration and a region image configuration in each page for an electronic document, the relationship between the document, the page, and the region image becomes clear, and the document management is improved for the user.
Further, by displaying the document structure of the electronic document, the page, and the region image in a tree format, the entire document can be easily grasped visually, and the document management is improved for the user.
Furthermore, by enabling the tree expansion and display of omission, the user can obtain only necessary information.
In addition, since it is possible to sort between area images based on the metadata registered in the area image and to narrow down the area images, it is possible to display according to the user's preference.

Furthermore, by sorting based on the coordinate information of the region image, it becomes easier for the user to visually grasp the positional relationship between the page and the region image.
Further, by displaying the text extracted from the region image as a hyperlink, the user can open and refer to the link destination simply by selecting the hypertext.
Further, by enabling the drag and drop of the area image, the reusability of the area image by the user is enhanced.
Furthermore, when the image type of the region image to be dragged and dropped is text, the user can edit the text in the application at the attachment destination by attaching key text that is metadata instead of as an image. .

Next, various types of processing for the page image of the electronic document data in the image processing apparatus will be described.
In this image processing apparatus, when the request control unit 21 detects an input of any of editing, replacement, combination, division, addition, and deletion work by the user, the edit, replacement, combination, Based on the display information of the division, addition, or deletion work screen, any one of the edit, replacement, combination, division, addition, and deletion work screens is displayed on the display unit 20.

  Then, the request control unit 21 changes the content of the image data in the image area or character area in the page for each page image data of the electronic document data to be processed stored in the storage unit 14 via the search unit 22. Editing processing, replacement processing for replacing image data in the page or character area with other image data, combining processing for combining a plurality of image data in the page or character area, Division processing that divides image data in the image area or character area into a plurality of image data, additional processing that adds other image data to image data in the image area or character area in the page, image area or character in the page Each processing of the deletion process for deleting the image data of the area is performed.

[Editing of image data in the image area or character area in the page]
FIG. 17 is a flowchart showing editing processing of image data in an image area or a character area in a page image in this image processing apparatus.
FIG. 18 is a diagram showing a tree-format list screen showing the document structure of the electronic document during the image data editing process.
FIG. 19 is a diagram showing page image data and page image metadata before editing.
FIG. 20 is a diagram showing area image data and area image metadata before editing.
FIG. 21 is a diagram showing an editing window displayed during editing operation.
FIG. 22 is a diagram showing edited area image data and area image metadata.

FIG. 23 is a diagram showing a selection dialog screen for selecting whether or not to reflect the edited content in the electronic document data.
FIG. 24 is a diagram illustrating an example of edited page image data and page image metadata.
FIG. 25 is a diagram showing a tree-format list screen showing the document structure of the electronic document when the edited content is reflected in the entire electronic document.
In each of the above drawings, data that is not particularly important in the following description and some drawings are omitted.

  As shown in FIG. 17, in the editing process in this image processing apparatus, in step (indicated by “S” in the figure) 41, when it is detected that the edit button displayed on the area image to be edited is pressed, step 42 is performed. Then, the editing window of the area image corresponding to the pressed edit button is displayed. In step 43, the contents of the area image displayed in the editing window are edited based on the instruction input. It is determined whether or not pressing of the cancel button has been detected. If pressing is detected, the process ends. If no pressing has been detected, whether or not pressing of the edit completion button in the editing window has been detected in step 45. If no depression is detected, the process returns to step 43.

  On the other hand, if it is detected in step 45 that the edit completion button has been pressed, area image metadata for the edited area image is created in step 46, and edited area image data is created based on the area image metadata. Then, the edited region image data and region image metadata are associated with each other and stored and stored in a new storage location in the storage unit, and in step 47, an instruction to be reflected in the entire image (entire electronic document data) is input. If it is determined whether or not the input of the instruction to be reflected is detected, the process is terminated. If the input of the instruction to be reflected is detected, the edited region image data and region image metadata are obtained in step 48. So that the page image data and the page image metadata of the electronic document data are rewritten, and this process ends.

Next, an editing process, an operation procedure, and a display screen example for the region image will be described.
For example, when editing the page image 70 shown in FIG. 18A, if there is a selection input of editing work from the user, a list display in a tree format showing the document structure as shown in FIG. Is displayed.
In this list display, an edit button is displayed on the right side of each area image displayed as a list image of area images in the page.
For example, when editing the region image 71, the user presses an edit button 72 located on the right side of the region image 71.

Here, the contents of the page image data and the page image metadata of the page image 70 before editing are as shown in FIG. 19, and the contents of the area image data and the area image metadata of the area image 71 of the page image 70. The contents are as shown in FIG. In FIG. 19 and FIG. 20, information not used in the editing process is not shown.
When the edit button 72 is pressed, the request control unit 21 refers to “text” of “image type” in the region image metadata of the region image 71 shown in FIG. 20 and recognizes that the editing target is text. Similarly, “ABC” is read from “key text” of the region image metadata of the region image 71, and the editing window shown in FIG.
In the text window 73 of this editing window, the above “ABC” is displayed.

Here, the user deletes the character string “ABC” displayed in the text window 73 shown in FIG. 21A, and newly displays the characters “XYZ” as shown in FIG. When a column is input and the edit completion button 74 in the edit window is pressed, the request control unit 21 creates new region image metadata and region image data based on the region image metadata as shown in FIG. These are stored in the storage unit 14 via the search unit 22 in association with each other.
In this region image metadata, a new ID is assigned to “region image ID”, the storage location of the region image data of the text “XYZ” is stored in “storage location”, and “text” is stored in “image type”. The character string “XYZ” is stored in the “key text”. The area image data of the text “XYZ” is generated from the key text “XYZ”.
On the other hand, when the cancel button 75 in the editing window shown in FIG. 21 is pressed, changes made to the area image 71 so far are discarded, and the state of the area image 71 before editing is restored.

After the edit completion button 74 is pressed, a selection dialog screen as shown in FIG. 23 is displayed. When the “Reflect” button 76 in the selection dialog screen is pressed, the request control unit 21 searches the search unit 22. The link processing for rewriting the page image data before editing and the page image metadata stored in the storage unit 14 is performed.
In this linking process, the area image ID “1001” of the area image information of the page image metadata before editing shown in FIG. 19 and the storage location “repository / 1001.pdf” are stored in the area image ID “1004” of the area image metadata. ”And the storage location“ repository / 1004.pdf ”, and the area image before editing in the page image data before editing shown in FIG. 19 is also rewritten to the area image after editing to have contents as shown in FIG. .
Thus, the editing result added to the area image 71 is reflected in the entire electronic document data.

As a result, a page image 78 as shown in FIG. 25A is obtained, and as shown in FIG. 25B, in the tree format list display showing the document structure, instead of the region image before editing, The edited region image 79 is displayed.
On the other hand, when the “do not reflect” button 77 in the selection dialog screen shown in FIG. 23 is pressed, the editing result is not reflected in the entire electronic document data, and remains the region image 71 before editing. . In this case, the page image data and the page image metadata are not changed.
In this way, the region image can be modified and the modified content can be reflected in the electronic document data. Therefore, it is easy to edit only the text in the electronic document data or only the image, and the electronic document data. Can improve the editing work efficiency. In addition, the usability of electronic document data can be improved.

[Replacement of image data in the image area or character area in the page]
FIG. 26 is a flowchart showing the replacement processing of the image data of the image area or the character area in the page image in this image processing apparatus.
FIG. 27 is a diagram showing a list screen representing a tree format showing the document structure of the electronic document at the time of image data replacement processing.
FIG. 28 is a diagram showing page image data and page image metadata before replacement.
FIG. 29 is a diagram showing area image data and area image metadata before replacement.

FIG. 30 is a diagram showing a replacement window displayed during the replacement operation.
FIG. 31 is a diagram showing a replacement image selection dialog window displayed during the replacement operation.
FIG. 32 is a diagram showing area image data and area image metadata after replacement.
FIG. 33 is a diagram of an example of page image data and page image metadata after replacement.
FIG. 34 is a diagram showing a tree-format list screen showing the document structure of the electronic document when the contents after replacement are reflected in the entire electronic document.
In each of the above drawings, data that is not particularly important in the following description and some drawings are omitted.

  As shown in FIG. 26, the replacement processing in this image processing apparatus is performed when step (indicated by “S” in the figure) 51 detects that the replacement button displayed on the replacement target area image is pressed. In 52, a replacement window of the area image corresponding to the pressed replacement button is displayed. In step 53, it is determined whether or not the pressing of the cancel button in the replacement window is detected. If no pressing is detected, it is determined in step 54 whether or not the pressing of the replacement image selection button in the replacement window is detected. If the pressing is not detected, the process of step 54 is repeated.

  On the other hand, if it is detected in step 54 that the replacement image selection button is pressed, a replacement image selection dialog window is displayed in step 55, and the area image data to be replaced is selected and opened in step 56 in the replacement image selection dialog window. If the pressing of the open button is not detected when the replacement area image data is selected and the opening button is not detected, the process returns to step 55 to select the replacement area image data and pressing the opening button. If detected, in step 57, the selected area image to be replaced is displayed in the replacement window, and in step 58, it is determined whether or not the pressing of the replacement execution button in the replacement window is detected.

  If it is determined in step 58 that the pressing of the replacement execution button has not been detected, the process returns to step 57. If it is determined that the pressing of the replacement execution button has been detected, in step 59, the image size of the selected region image to be replaced is determined. It is determined whether or not the image size of the area image to be replaced is different. If it is different, the image size of the area image to be replaced selected in step 60 is replaced with the image size of the area image to be replaced. The region image data to be processed is converted by performing enlargement or reduction processing, or converted by performing trimming processing, and the process proceeds to step 61.

If it is determined in step 59 that the image size of the selected area image to be replaced and the image size of the area image to be replaced are not different (same), the process proceeds to step 61 as it is.
In step 61, the region image data to be replaced is replaced with the selected region image data to be replaced (or the region image data subjected to the conversion process), and the region image data after replacement and the region image metadata are associated with each other. As described above, the page image data and the page image metadata of the electronic document data are rewritten, and this process ends.

Next, an area image replacement process, an operation procedure, and a display screen example will be described.
For example, when the area image in the page image 80 shown in FIG. 27A is replaced with another area image, a document as shown in FIG. Display a tree-like list of structures.
In this list display, a replacement button is displayed on the right side of each area image displayed as a list image of area images in the page.
For example, when replacing the region image 81 with another image, the user presses a replacement button 82 located on the right side of the region image 81.

Here, the contents of the page image data and the page image metadata of the page image 80 before replacement are as shown in FIG. 28, and the contents of the area image data of the area image 81 of the page image 80 and the contents of the area image metadata. The contents are as shown in FIG. In FIG. 28 and FIG. 29, description of each piece of information that is not used in this replacement processing is omitted.
When the replace button 82 is pressed, the request control unit 21 refers to the “picture” of the “image type” of the area image metadata of the area image 81 shown in FIG. 29 and recognizes that the replacement target is a figure. Similarly, the area image data is read from the “storage location” of the area image metadata of the area image 81, and the replacement window shown in FIG. 30 is displayed on the display unit 20.
The area image before replacement is displayed in the area on the left side of the replacement window.

Here, when the user presses the replacement image selection button 83 in the replacement window shown in FIG. 30, the request control unit 21 searches the storage unit 14 for area image data that can be replaced via the search unit 22 and displays it. A replacement image selection dialog window as shown in FIG. 31 is displayed on the display unit 20 via the information generation unit 23.
In this replacement image selection dialog window, a list of information including file names, sizes, and update dates / times of a plurality of replaceable area image data is displayed.

When the user selects and inputs a file name of arbitrary area image data from a list of a plurality of replaceable area image data in the replacement image selection dialog window and presses an open button 85, the request control unit 21 passes the search unit 22. Then, the area image data of the file name selected and input from the storage unit 14 is read out and acquired, and the right area in the replacement window displayed on the display unit 20 via the display information generation unit 23 based on the area image data The area image to be replaced is displayed.
If the cancel button is pressed, the file name selected and input is cleared and the screen returns to the file name selection and input screen again.

After pressing the open button 85 in the replacement image selection dialog window, a region image based on the region image data of the file name selected and input as a region image to be replaced in the region on the right side in the replacement window as shown in FIG. Is displayed.
Here, when the cancel button in the replacement window is pressed, the replacement image selection dialog window is displayed again, and the region image based on the region image data of the file name selected there is displayed in the replacement window.

When the user presses the replacement execution button 84 in the replacement window, the request control unit 21 stores the pre-replacement area image data, the area image metadata, and the pre-replacement image data stored in the storage unit 14 via the search unit 22. A process of rewriting page image data and page image metadata is performed.
Here, when the image size of the area image 87 to be replaced is different from the image size of the area image 87 to be replaced, the area image data of the area image 87 is subjected to processing including enlargement, reduction, and trimming, and the area image to be replaced Conversion to be the same size as 81.

In the rewriting process, the area image data before replacement shown in FIG. 29 is replaced with the selected area image data, and the area image ID “1005” of the area image metadata shown in FIG. 29 and the storage location “repository / “1005.pdf” is rewritten to the area image ID “1006” and the storage location “repository / 1006.pdf” of the area image metadata, respectively, so that the contents are as shown in FIG.
Furthermore, the area image ID “1005” and the storage location “repository / 1005.pdf” of the area image information of the page image metadata before replacement shown in FIG. 28, the area image ID “1006” and the storage location “repository / 1006”. .Pdf ", the page image data before replacement is also rewritten to the area image after replacement with the selected area image, and the contents are as shown in FIG.
In this way, the area image after replacement is reflected in the entire electronic document data.

As a result, a page image 86 as shown in FIG. 34 (a) is obtained. As shown in FIG. 34 (b), in the tree format list display showing the document structure, instead of the area image before replacement, The region image 87 after replacement is displayed. Here, the area image 81 before replacement may be displayed with the link removed.
In this way, an arbitrary region image in the electronic document data stored in the storage unit can be replaced with another region image, so that the OCR recognition defect for the region image in the stored electronic document data is corrected. be able to. It is also possible to create a document having data intended by the user.

[Combination of image data in the image area or character area in the page]
FIG. 35 is a flowchart showing a process for combining image data of image areas or character areas in a page image in the image processing apparatus.
FIG. 36 is a diagram showing a list screen representing a tree format showing the document structure of the electronic document at the time of image data combination processing.
FIG. 37 is a diagram showing page image data and page image metadata before combining.
FIG. 38 is a diagram showing region image data and region image metadata before combining one region image to be combined.
FIG. 39 is a diagram showing region image data and region image metadata before combining the other region image to be combined.

FIG. 40 is a diagram showing a combination window displayed during the combination operation.
FIG. 41 is a diagram showing region image data and region image metadata after combination.
FIG. 42 is a diagram illustrating an example of page image data and page image metadata after combination.
FIG. 43 is a diagram showing a tree-format list screen showing the document structure of the electronic document when the combined contents are reflected in the entire electronic document.
In each of the above drawings, data that is not particularly important in the following description and some drawings are omitted.

  As shown in FIG. 35, the combination processing in this image processing apparatus is performed at step 71 (indicated by “S” in the drawing) 71 of the combination button displayed for the page image including a plurality of region images to be combined. When the pressing is detected, in step 72, a combined window for each region image in the page image corresponding to the pressed combined button is displayed, and in step 73, a plurality of region images to be combined in the combined window are selected. In step 74, it is determined whether or not pressing of the cancel button is detected. If pressing is detected, this process is terminated.

  If the pressing of the cancel button is not detected in the determination of step 74, it is determined in step 75 whether or not the pressing of the combining execution button in the combining window is detected. If the pressing is not detected, the process of step 75 is performed. If it is detected repeatedly that the combining execution button is pressed, a plurality of region images selected in the combining window are combined into one region image in step 76, and the combined region image data and region image metadata are created. The page image data and the page image metadata of the electronic document data are rewritten so as to be stored and stored, and the combined region image data and the region image metadata are associated with each other, and this process ends.

Next, region image combination processing, operation procedures, and display screen examples will be described.
For example, when a plurality of region images in the page image 90 shown in FIG. 36A are combined into one, if there is a selection input of a combining operation from the user, a document as shown in FIG. Display a tree-like list of structures.
In this list display, a combination button is displayed on the right side of each page image including a plurality of area images among the page images of the electronic document list image.
For example, when performing a combination process on the page image 91, the user presses a combination button 92 located on the right side of the page image 91.

Here, the contents of the page image data and the page image metadata of the page image 90 before the combination are as shown in FIG. 37, and the region image data and the region image of one of the region images to be combined in the page image 90. The contents of the metadata are as shown in FIG. 38, and the area image data of the other area image to be combined in the page image 90 and the contents of the area image metadata are as shown in FIG. Shall. In FIG. 37 to FIG. 39, the description of each piece of information that is not used in this combining process is omitted.
When the join button 92 is pressed, the request control unit 21 refers to the region image information of the page image metadata shown in FIG. 37, reads the region image data from the storage location of each region image in the page, and displays the display unit. 20 displays the combined window shown in FIG.

In this combined window, each area image in the page is displayed as it is in the arrangement position in the page. Here, as shown in FIG. 40 (a), two area images indicated by dotted line frames in the figure are displayed. , The broken line frame 93 displayed in one of the region images is expanded to the right so as to include the region image on the right as shown in FIG. A join execution button 95 is pressed.
When the join execution button 95 is pressed, the request control unit 21 creates new area image data obtained by combining the area images shown in FIGS. 38 and 39 into one.
Thus, by newly specifying the coordinates of the four points of the region image in the combined window, redefining the rectangular region based on the coordinates of the four points, and extracting the region image again, one new region image is created. .

Then, the new region image data is stored in the storage unit 14 via the search unit 22, and the region image metadata of the new region image data is created based on the region image ID assigned to the region image data and the storage location. Then, it is stored in correspondence with the new area image data.
Further, the request control unit 21 performs a process of rewriting the page image data and page image metadata before combination stored in the storage unit 14 via the search unit 22.
The contents of the area image data and area image metadata newly created by the above combination processing are as shown in FIG. 41. The area image metadata includes the area image ID “1008” and the storage location “repository”. /1008.pdf ”and the image type“ figure ”are stored.

In the rewriting process, the region image ID “1002”, the storage location “repository / 1002.pdf”, the region image ID “1007”, and the storage location of the region image information of the page image metadata before combining shown in FIG. The “repository / 1007.pdf” is deleted, the area image ID “1008” and the storage location “repository / 1008.pdf” are newly rewritten, and the page image data before combination is also replaced with the combined area image. It is rewritten to the area image including it, and it becomes the content as shown in FIG.
In this way, the combined region image is reflected in the entire electronic document data.
As a result, a page image 96 as shown in FIG. 43 (a) is obtained. As shown in FIG. 43 (b), in the tree-format list display showing the document structure, instead of the page image before the combination, The combined page image 97 is displayed, and the area image list image is also displayed as a combined area image 98 instead of the two area images before combining.

On the other hand, when the cancel button is pressed in the combination window, the changes made to the area image so far are discarded and the state before the combination is restored.
In this way, a plurality of region images can be combined to create one new region image and reflected in the entire electronic document data, so that the OCR recognition defect can be corrected and the user can The intended data can be created.

[Division processing of image data in image area or character area in page]
FIG. 44 is a flowchart showing a process for dividing image data in an image area or a character area in a page image in this image processing apparatus.
FIG. 45 is a diagram showing a list screen representing a tree format indicating the document structure of the electronic document at the time of image data division processing.
FIG. 46 is a diagram showing a split window displayed during the split operation.
FIG. 47 is a diagram showing one area image data and area image metadata after division.
FIG. 48 is a diagram showing a tree-format list screen showing the document structure of the electronic document when the contents after division are reflected in the entire electronic document.
In each of the above drawings, data that is not particularly important in the following description and some drawings are omitted.

  As shown in FIG. 44, in the division processing in this image processing apparatus, in step (indicated by “S” in the figure) 81, the division button displayed on the page image including the region image to be divided is pressed. If detected, in step 82, a divided window for the area image in the page image corresponding to the pressed division button is displayed. In step 83, the area image to be divided in the divided window is selected. It is determined whether or not the pressing of the button is detected. When the pressing is detected, this process is terminated.

  If it is determined in step 84 that the cancel button has not been pressed, it is determined in step 85 whether or not a split execution button in the split window has been pressed. If no press is detected, the processing in step 85 is performed. If it is detected repeatedly that the division execution button is pressed, the area image selected in the division window in step 86 is divided into a plurality of designated areas, and the divided area image data and area image metadata are created. The page image data and the page image metadata of the electronic document data are rewritten so as to associate the divided region image data with the region image metadata, and this process is terminated.

Next, an area image division process, an operation procedure, and a display screen example will be described.
For example, when the region image in the page image 100 shown in FIG. 45A is divided into two, if there is a selection input of the division work from the user, the document structure as shown in FIG. Displays a tree-format list display.
In this list display, a division button is displayed on the right side of the page image including the region image among the page images of the electronic document list image.
For example, when the division process is performed on the page image 101, the user presses the division button 102 located on the right side of the page image 101.

Here, the contents of the page image data and the page image metadata of the page image 100 before the division are as shown in FIG. 42, and the area image data and area image metadata of the area image to be divided in the page image 100. Assume that the contents of are as shown in FIG.
When the division button 102 is pressed, the request control unit 21 refers to the region image information of the page image metadata shown in FIG. 42, reads the region image data from the storage location of each region image in the page, and displays the display unit. 20 displays the divided window shown in FIG.

In this divided window, each area image in the page is displayed as it is in the arrangement position in the page. Here, as shown in FIG. 46 (a), two area images indicated by a dotted frame in the figure are displayed. In the case of dividing into two, as shown in FIG. 46B, after performing an operation of designating an area to be divided as indicated by broken line frames 103 and 104 for this area image, the division execution button 105 is pressed. .
When the division execution button 105 is pressed, the request control unit 21 creates new area image data of the two specified areas from the area image data shown in FIG.
In this way, by newly specifying the coordinates of the four points of the area image in the divided window, redefining the rectangular area based on the coordinates of the four points, and re-extracting two or more area images, a plurality of new areas Create an image.

Then, two new region image data are stored in the storage unit 14 via the search unit 22, and the region image of each new region image data is based on the region image ID assigned to each region image data and the storage location. Metadata is created and stored in association with each new area image data.
Further, the request control unit 21 performs a process of rewriting the page image data before division and the page image metadata stored in the storage unit 14 via the search unit 22.

One of the contents of the area image data and the area image metadata newly created by the division processing is as shown in FIG. 38, and the area image metadata includes an area image ID “1002” and a storage. The location “repository / 1002.pdf” and the image type “figure” are stored.
The other is the contents as shown in FIG. 47, and the area image metadata stores the area image ID “1007”, the storage location “repository / 1007.pdf”, and the image type “diagram”. ing.

In the rewriting process, the area image ID “1008” and the storage location “repository / 1008.pdf” of the area image information of the page image metadata before division shown in FIG. Rewriting is performed to add “1002” and storage location “repository / 1002.pdf”, region image ID “1007” and storage location “repository / 1007.pdf”, and the page image data before the division is also divided into the region images. Is rewritten into an area image including the contents shown in FIG.
Thus, the divided area image is reflected in the entire electronic document data.

  As a result, a page image 106 as shown in FIG. 48 (a) is obtained. As shown in FIG. 48 (b), in the tree format list display showing the document structure, instead of the page image before division, The divided page image 107 is displayed, and the area image list image also displays area images 108 and 109 of the area image obtained by dividing them into two instead of the area image before division.

On the other hand, when the cancel button is pressed in the division window, the changes made to the area image so far are discarded and the state before division is restored.
In this way, since one area image can be divided and a plurality of new area images can be created and reflected in the entire electronic document data, the OCR recognition defect can be corrected and the user intends Data can be created.

[Adding image data of image area or character area to the page]
FIG. 49 is a flowchart showing the process of adding image data of the image area or character area to the page image in this image processing apparatus.
FIG. 50 is a diagram showing a list screen representing a tree format showing the document structure of the electronic document at the time of image data addition processing.

FIG. 51 is a diagram showing page image data and page image metadata before addition.
FIG. 52 is a diagram showing an additional image selection dialog window displayed during the addition operation.
FIG. 53 is a diagram showing an entire image display screen displayed during the addition operation.
FIG. 54 is a diagram of an example of page image data and page image metadata after addition.
FIG. 55 is a diagram showing a tree-format list screen showing the document structure of the electronic document when the contents after addition are reflected in the entire electronic document.
In each of the above drawings, data that is not particularly important in the following description and some drawings are omitted.

  As shown in FIG. 49, in the addition processing in this image processing apparatus, in step (indicated by “S” in the figure) 91, the addition button displayed on the page image for adding a new region image is pressed. If detected, in step 92, an additional image selection dialog window for selecting an area image to be newly added in the page image corresponding to the pressed add button is displayed, and in step 93, an area to be added in the additional image selection dialog window. In step 94, it is determined whether or not the pressing of the cancel button in the additional image selection dialog window has been detected. If the pressing is detected, the process ends. If the pressing is not detected, In step 95, it is determined whether or not pressing of the open button in the additional image selection dialog window is detected. If you are not out, it repeats the processing of step 94.

  On the other hand, if the depression of the open button is detected in the determination in step 94, the entire image display screen is displayed in step 95, the addition position of the selected region image is determined in the entire image display screen, and the entire image is determined in step 97. It is determined whether or not the pressing of the additional execution button in the display screen is detected. If the pressing of the additional execution button is not detected, the process of step 97 is repeated, and if the pressing of the additional execution button is detected, the process proceeds to step 98. In order to create region image metadata for the region image data to be added, store the region image data to be added and the region image metadata in association with each other, and associate the region image data to be added with the region image metadata. The page image data and page image metadata of the document data are rewritten, and this process is terminated.

  In addition, in the above processing, the image size of the region image selected for addition is stored in the page or matched with the size of another region image. Conversion may be performed by performing enlargement or reduction processing to an arbitrarily specified size, or by performing trimming processing.

Next, an area image addition process, an operation procedure, and a display screen example will be described.
For example, when a new area image is added to the page image 110 shown in FIG. 50A, if a user selects and inputs additional work, a document structure as shown in FIG. 50B is displayed. Display a list of trees.
In this list display, an add button is displayed on the right side of each page image of the electronic document list image.
For example, when an additional process is performed on the page image 111, the user presses the add button 112 located on the right side of the page image 111.

Here, the contents of the page image data and the page image metadata of the page image 110 before addition are as shown in FIG. In FIG. 51, the description of each piece of information that is not used in this additional process is omitted.
When the add button 112 is pressed, the request control unit 21 searches the addable area image data stored in the storage unit 14 by the search unit 22 and displays the additional image selection dialog shown in FIG. Display the window.
In this additional image selection dialog window, a list of information including file names, sizes, and update dates / times of a plurality of area image data that can be added is displayed.
If the cancel button in the additional image selection dialog window is pressed, the file name selected and input is cleared, and the file name selection input screen is displayed again.

  When the user selects and inputs a file name of arbitrary area image data from a list of a plurality of area image data that can be added in the additional image selection dialog window and presses the open button 113, the request control unit 21 passes through the search unit 22. The area image data of the file name selected and input from the storage unit 14 is read out and acquired, and further, based on the page image data and page image metadata of the page image corresponding to the pressed add button 112, An overall image display screen as shown in FIG. 53 is displayed on the display unit 20 via the display information generation unit 23.

On this entire image display screen, each area image in the page image is displayed as it is arranged, and an area image based on the selected area image data is displayed in the margin part. When the area image 114 based on the selected area image data is arranged at a position to be added in the page image and the add execution button 115 in the entire image display screen is pressed, the area image of the selected area image data is displayed. Metadata is created, the region image metadata is associated with the selected region image data and stored in the storage unit 14 via the search unit 22, and the region image metadata is stored in the selected region image data. Based on the above, a process of rewriting the page image data before addition and page image metadata stored in the storage unit 14 is performed.
Here, when the cancel button in the entire image display screen is pressed, the changes made to the page image so far are discarded, and the page image returns to the state before the addition.

In the rewriting process, the area image ID “1003” and the storage location “repository / 1003.pdf” of the area image data newly added this time are added to the area image information of the page image metadata before addition shown in FIG. The content is rewritten as shown in FIG.
In this way, the addition of the area image is reflected in the entire electronic document data.

As a result, a page image 116 as shown in FIG. 55A is obtained, and as shown in FIG. 55B, in the list display in the tree format showing the document structure, the page image before the addition of the region image is displayed. Instead, the page image 117 after the addition of the region image is displayed. Further, the newly added region image 118 is displayed in a tree format for the region image list image in the page image.
In this way, the OCR recognition defect can be corrected by adding the selected arbitrary region image to the electronic document data. Also, electronic document data having data intended by the user can be created.

[Deleting image data in the image area or character area in the page]
FIG. 56 is a flowchart showing image data deletion processing of an image area or a character area in a page image in this image processing apparatus.
FIG. 57 is a diagram showing a list screen representing a tree format showing the document structure of the electronic document at the time of image data deletion processing.
FIG. 58 is a diagram showing an entire image display screen displayed during the deletion operation.
FIG. 59 is a diagram showing a tree-format list screen showing the document structure of the electronic document when the contents after deletion are reflected in the entire electronic document.
In each of the above drawings, data that is not particularly important in the following description and some drawings are omitted.

  As shown in FIG. 56, the deletion processing in this image processing apparatus is performed when step 101 (indicated by “S” in the figure) 101 detects that the delete button displayed on the area image to be deleted is pressed. Display an entire image display screen for confirming the region image corresponding to the pressed delete button, and in step 103, determine whether or not the pressing of the cancel button in the entire image display screen is detected. This process is terminated. However, if the pressing is not detected, it is determined in step 104 whether or not the pressing of the delete execution button in the entire image display screen is detected. If the pressing is not detected, When the process of step 104 is repeated and the pressing of the delete execution button is detected, the selected area image is deleted from the page image in step 105, and the content after the deletion As reflect rewrites the page image data and the page image metadata of the electronic document data, the process ends.

Next, an area image deletion process, an operation procedure, and a display screen example will be described.
For example, in the case of deleting the region image in the page image 120 shown in FIG. 57A, if there is a selection input for deletion work from the user, a tree format showing a document structure as shown in FIG. Display the list display.
In this list display, a delete button is displayed to the right of each area image displayed in the list image of area images in the page.
For example, when deleting the area image 121, the user presses the delete button 122 located on the right side of the area image 121.

Here, the contents of the page image data and the page image metadata of the page image 120 before deletion are as shown in FIG.
When the delete button 122 is pressed, the request control unit 21 refers to the region image information of the page image metadata shown in FIG. 19, reads each region image data in this page, and displays it on the display unit 20 as shown in FIG. The entire image display screen shown is displayed.
On the entire image display screen, the area images in the page image are displayed as they are, and the area image 123 corresponding to the delete button 122 is highlighted to indicate that it has been selected for deletion.

When the user confirms the area image 123 to be deleted on the entire image display screen and presses the delete execution button 124, the request control unit 21 stores the page image data stored in the storage unit 14 via the search unit 22. And rewrite the page image metadata.
In this rewriting process, the area image ID “1003” and the storage location “repository / 1003.pdf” of the deleted area image are deleted from the area image information of the page image metadata shown in FIG. The area image designated to be deleted is rewritten with the deleted contents so as to have the contents as shown in FIG.
In this way, the contents obtained by deleting an arbitrary area image from the page image are reflected in the entire electronic document data.

As a result, a page image 125 as shown in FIG. 59 (a) is obtained. As shown in FIG. 59 (b), in the list display in the tree format indicating the document structure, the page image before deletion of the region image is displayed. Instead, the page image 126 after the area image is deleted is displayed. Further, with respect to the region image list image in the page image, the deleted region image and the delete button displayed on the right side thereof are deleted.
On the other hand, when the cancel button is pressed on the entire image display screen, the changes made to the page image so far are discarded, and the page image returns to the state before deletion.
In this way, the OCR recognition defect can be corrected by deleting an arbitrary area image from the page image of the electronic document data. Also, electronic document data having data intended by the user can be created.

  The image processing apparatus, the image processing method, and the program according to the present invention can also be applied to personal computers such as desktop personal computers and notebook personal computers.

It is a block diagram which shows the function structure of the image processing apparatus of the Example of this invention. FIG. 3 is a detailed explanatory diagram of information registered in a storage unit by the registration unit illustrated in FIG. 1. It is explanatory drawing of a continuation of FIG. It is explanatory drawing of a continuation of FIG. It is explanatory drawing of the example of a screen which displays the document structure of an electronic document.

It is explanatory drawing of the example of a screen which similarly displays the document structure of an electronic document. It is explanatory drawing of the example of a screen which similarly displays the document structure of an electronic document. It is a flowchart figure which shows the process at the time of switching from a page list screen to a document tree screen. It is explanatory drawing of the sorting of a region image.

It is a flowchart figure which shows the process which abbreviate | omits the area | region image currently expand | deployed by tree display. It is a figure which shows an example of a preview screen. It is a flowchart figure of the process which switches from a page list screen to a preview screen. FIG. 10 is a flowchart illustrating processing when dragging and dropping an area image of a page displayed as a preview.

It is explanatory drawing of display formats other than a tree format. It is explanatory drawing which shows the example of area emphasis display on a preview screen. It is explanatory drawing of the other example of display formats other than a tree format. It is a flowchart figure which shows the edit process of the image data of the image area | region or character area | region in a page image in the image processing apparatus of this Example.

It is a figure which shows the list screen of a tree format which shows the document structure of the electronic document at the time of the edit process of the image data in the image processing apparatus of this Example. It is a figure which shows the page image data and page image metadata before editing in the image processing apparatus of this Example. It is a figure which shows the area | region image data and area | region image metadata before editing in the image processing apparatus of this Example.

It is a figure which shows the edit window displayed at the time of editing operation in the image processing apparatus of this Example. It is a figure which shows the area | region image data and area | region metadata after editing in the image processing apparatus of this Example. It is a figure which shows the selection dialog screen which selects whether the content after edit in the image processing apparatus of this Example is reflected in electronic document data.

It is a figure of an example of the page image data after edit in the image processing apparatus of this Example, and page image metadata. It is a figure which shows the list screen of a tree format which shows the document structure of an electronic document when the content after edit in the image processing apparatus of this Example is reflected in the whole electronic document. It is a flowchart figure which shows the replacement process of the image data of the image area | region or character area | region in a page image in the image processing apparatus of this Example.

It is a figure which shows the list screen showing the tree format which shows the document structure of the electronic document at the time of the replacement process of the image data in the image processing apparatus of this Example. It is a figure which shows the page image data and page image metadata before replacement in the image processing apparatus of this embodiment. It is a figure which shows the area | region image data and area | region image metadata before replacement in the image processing apparatus of this Example.

It is a figure which shows the replacement window displayed at the time of replacement operation in the image processing apparatus of this Example. It is a figure which shows the replacement image selection dialog window displayed at the time of replacement operation in the image processing apparatus of this embodiment. It is a figure which shows the area | region image data and area | region metadata after replacement in the image processing apparatus of this Example.

It is a figure of an example of the page image data after replacement | exchange in the image processing apparatus of this Example, and page image metadata. It is a figure which shows the list screen of a tree format which shows the document structure of an electronic document when the content after replacement in the image processing apparatus of this Example is reflected in the whole electronic document. It is a flowchart figure which shows the combination process of the image data of the image area | region or character area | region in a page image in the image processing apparatus of this Example.

It is a figure which shows the list screen showing the tree format which shows the document structure of the electronic document at the time of the joint process of the image data in the image processing apparatus of this Example. It is a figure which shows the page image data and page image metadata before the combination in the image processing apparatus of this embodiment. It is a figure which shows the area | region image data and area | region image metadata before combining of the one area | region image to combine in the image processing apparatus of this Example.

It is a figure which shows the area | region image data and area | region image metadata before the combination of the other area | region image to combine in the image processing apparatus of this Example. It is a figure which shows the joint window displayed at the time of the joint operation in the image processing apparatus of this Example. It is a figure which shows the area | region image data and area | region image metadata after a combination in the image processing apparatus of this Example.

It is a figure of an example of the page image data and page image metadata after the combination in the image processing apparatus of this embodiment. It is a figure which shows the list screen of a tree format which shows the document structure of an electronic document when the content after the combination in the image processing apparatus of this Example is reflected in the whole electronic document. It is a flowchart figure which shows the division | segmentation process of the image data of the image area | region or character area | region in a page image in this image processing apparatus in the image processing apparatus of this Example.

It is a figure which shows the list screen showing the tree format which shows the document structure of the electronic document at the time of the division | segmentation process of the image data in the image processing apparatus of this Example. It is a figure which shows the division | segmentation window displayed at the time of the division | segmentation operation in the image processing apparatus of this Example. It is a figure which shows one area | region image data and area | region image metadata after the division | segmentation in the image processing apparatus of this Example.

It is a figure which shows the list screen of a tree format which shows the document structure of an electronic document when the content after the division | segmentation in the image processing apparatus of this Example is reflected in the whole electronic document. It is a flowchart figure which shows the addition process of the image data of the image area | region or character area | region in the page image in the image processing apparatus of this Example. It is a figure which shows the list screen showing the tree format which shows the document structure of the electronic document at the time of the addition process of the image data in the image processing apparatus of this Example.

It is a figure which shows the page image data before addition and page image metadata in the image processing apparatus of this Example. It is a figure which shows the additional image selection dialog window displayed at the time of addition operation in the image processing apparatus of this Example. It is a figure which shows the whole image display screen displayed at the time of addition operation in the image processing apparatus of this Example.

It is a figure of an example of the page image data and page image metadata after the addition in the image processing apparatus of this embodiment. It is a figure which shows the list screen of a tree format which shows the document structure of an electronic document when the content after the addition in the image processing apparatus of this Example is reflected in the whole electronic document. It is a flowchart figure which shows the deletion process of the image data of the image area | region or character area | region in a page image in the image processing apparatus of this Example.

It is a figure which shows the list screen showing the tree format which shows the document structure of the electronic document at the time of the deletion process of the image data in the image processing apparatus of this Example. It is a figure which shows the whole image display screen displayed at the time of deletion operation in the image processing apparatus of this Example. It is a figure which shows the list screen of a tree format which shows the document structure of an electronic document when the content after deletion in the image processing apparatus of this Example is reflected in the whole electronic document.

Explanation of symbols

1: Registration processing unit 2: Search display unit 10: Document input unit 11: Area extraction unit 12: Text extraction unit 13: Registration unit 14: Storage unit 20: Display unit 21: Request control unit 22: Search unit 23: Display information Generation unit 30: electronic document data 31: electronic document metadata 32: page image data 33: page image metadata 34: area image data 35: area image metadata

Claims (18)

Extraction means for extracting image data of an image area including an image and image data of a character area including a character string from page image data of the electronic document data, the electronic document data, and page image data of the electronic document data; Registration means for associating and registering the image data extracted by the extraction means for the page image data, and the page configuration of the electronic document data, the image area and the character area in the page based on the data registered by the registration means And an image processing apparatus including a display unit that displays a document structure including the area configuration of
Editing means for editing the image data, selection means for selecting whether or not to associate the image data edited by the editing means with the page image data of the electronic document data, and selecting when the selection means associates A means for associating the image data edited by the editing means with the page image data of the electronic document data, instead of the image data associated with the page image data of the electronic document data. An image processing apparatus.   The editing means is means for adding editing for replacing image data registered in association with page image data of the electronic document data with other designated image data. When the image size is different from the image size of the registered image data, the adjustment is performed so that the image size of the specified other image data matches the image size of the registered image data. 2. The image processing apparatus according to claim 1, further comprising means.   2. The editing unit according to claim 1, wherein the editing unit is a unit for adding a plurality of pieces of image data registered in association with page image data of the electronic document data to make one image data. Image processing device.   The image processing apparatus according to claim 1, wherein the editing unit is a unit that adds editing that divides image data registered in association with page image data of the electronic document data into a plurality of image data.   2. The image processing apparatus according to claim 1, wherein the editing means is means for adding editing for adding new image data at a specified position in the page image data.   The image processing apparatus according to claim 1, wherein the editing unit is a unit that adds editing for deleting image data registered in association with the page image data. An extraction step of extracting image data of an image area including an image and image data of a character area including a character string from page image data of the electronic document data, and the page image data of the electronic document data and the electronic document data; A registration step for registering the image data extracted in the extraction step in association with the page image data, and a page configuration of the electronic document data, an image region in the page, and a character region based on the data registered in the registration step In an image processing method comprising a display step for displaying a document structure consisting of
An editing step for editing the image data, a selection step for selecting whether or not the image data edited by the editing step is to be associated with the page image data of the electronic document data, and a selection when the selection is performed by the selection step And a step of associating the image data edited by the editing step with the page image data of the electronic document data, instead of the image data associated with the page image data of the electronic document data. An image processing method.   The editing step is a step of adding editing for replacing the image data registered in association with the page image data of the electronic document data with the other designated image data. When the image size is different from the image size of the registered image data, the adjustment is performed so that the image size of the specified other image data matches the image size of the registered image data. The image processing method according to claim 7, further comprising a step.   8. The editing step according to claim 7, wherein the editing step is a step of adding a plurality of pieces of image data registered in association with page image data of the electronic document data into one image data. Image processing method.   8. The image processing method according to claim 7, wherein the editing step is a step of adding an edit for dividing image data registered in association with page image data of the electronic document data into a plurality of image data.   8. The image processing method according to claim 7, wherein the editing step is a step of adding an edit for adding new image data at a designated position in the page image data.   8. The image processing method according to claim 7, wherein the editing step is a step of adding an edit to delete image data registered in association with the page image data. An extraction procedure for extracting image data of an image area including an image and image data of a character area including a character string from page image data of the electronic document data, respectively, and a page of the electronic document data and the electronic document data A registration procedure for associating and registering image data and image data extracted by the extraction procedure for the page image data, and a page configuration of the electronic document data and an image area in the page based on the data registered by the registration procedure In a program for executing a display procedure for displaying a document structure consisting of an area configuration of a text area and a text area,
An editing procedure for editing the image data, a selection procedure for selecting whether or not the image data edited by the editing procedure is to be associated with the page image data of the electronic document data, and a selection to associate with the selection procedure And a step of associating the image data edited by the editing procedure with the page image data of the electronic document data, instead of the image data associated with the page image data of the electronic document data. Program for.   The editing procedure is a procedure for adding editing for replacing image data registered in association with page image data of the electronic document data with other specified image data. When the image size is different from the image size of the registered image data, the adjustment is performed so that the image size of the specified other image data matches the image size of the registered image data. The program according to claim 13 for executing a procedure.   14. The program according to claim 13, wherein the editing procedure is a procedure for adding a plurality of pieces of image data registered in association with page image data of the electronic document data to make one image data.   The program according to claim 13, wherein the editing procedure is a procedure of adding an edit for dividing image data registered in association with page image data of the electronic document data into a plurality of image data.   The program according to claim 13, wherein the editing procedure is a procedure of adding editing for adding new image data at a specified position in the page image data.   The program according to claim 13, wherein the editing procedure is a procedure of adding an edit that deletes image data registered in association with the page image data.

Images