JP2000011192A - Inter-image positioning method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an inter-image positioning method for obtaining images whose inter-image positions are aligned as desired. SOLUTION: This is an inter-image positioning method for operating the inter-image positioning of continuous pages in images inputted for each page, and this method comprises a step for judging whether or not the image is a desired image, a step for obtaining a template being the reference of positioning from the image which is judged as the desired image, and step for correcting the inter-image position based on the template.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an authoring system for creating an electronic book content by inputting a book as an image base, and more particularly, to aligning a desired page in image data of the input book. The present invention relates to a method of performing an image alignment.

[0002]

2. Description of the Related Art Presently, with the development of hardware and software, the publication of electronic books has been activated as a new book form replacing existing paper media. These electronic books are capable of supporting so-called multimedia data such as audio, images, moving images, animations, etc., but are basically in a text (character code) -based format.

[0003] On the other hand, books on paper media (so-called "books") still count 500,000 titles a year, and the total number is enormous. There is only paper media.

[0004]

However, such a conventional electronic book has the following problems because it is based on a text (character code).

[0005] To author a book on paper media, text data must be created using manual or OCR. Therefore, a lot of time is required for content creation, and it is difficult to supply a large amount of content in a timely manner.

[0006] Also, books, such as comics and photo magazines, in which data other than texts occupy a large portion, are difficult to convert into contents.

[0007] Under such circumstances, the number of electronic book contents in the related art is as small as about several hundreds, and the contents are biased toward dictionary systems. For this reason, the diffusion rate of e-books is 1
% Is not yet reached. In particular, the problem of a small number of types of content is fatal, and is a major hindrance to the spread of electronic books.

Therefore, as a means for solving the above problem, it is conceivable to obtain electronic book contents by inputting on an image basis. This has the following advantages.

Basically, content can be created simply by scanning an existing paper-media book, and a large amount of content can be supplied in a short period of time.

[0010] It is possible to supply contents such as comics and photo magazines, which were not possible with text-based electronic books.

[0011] Even if there is a character that does not conform to the current character code system, such as an external character, a variant character, or an old document, it can be easily captured.

Since it does not depend on the language (character code), overseas development of the entire viewer and authoring system (spread)
Is easy.

With the above advantages, the image-based electronic book can solve all the problems of the text-based electronic book.

However, in order to obtain electronic book contents by inputting on an image basis, an ADF (Auto Docum) is required.
ent Feeder) to perform various processes such as document structuring, but the following problems arise.

In the case of an electronic book, a viewer has a reference straight line (for example, an edge portion of a CRT or a liquid crystal) in the case of an electronic book.
The inclination is emphasized more, giving the user discomfort. Therefore, a process for correcting this is required. Even if this is manually performed, a considerable amount of work is required, and the authoring time is significantly increased.

In particular, if there is a tilt or deviation in the body page of an electronic book, a great discomfort is caused when viewed with a viewer.

Further, if all pages are checked for reviewing electronic book contents, the authoring time is increased, and a large amount of contents cannot be supplied in a short period of time. Therefore, appropriate error processing is required.

An object of the present invention is to provide an inter-image registration method capable of obtaining an image in which the positions of desired images are aligned in view of the above problems.

[0019]

According to a first aspect of the present invention, there is provided a method for aligning images, comprising the steps of:
An image-to-image alignment method for aligning images of consecutive pages, the method comprising: determining a desired image; and a template serving as a reference for alignment based on the image determined to be a desired image. And a step of correcting a position between images based on the template.

According to a second aspect of the present invention, there is provided a method of aligning images,
2. The method according to claim 1, wherein the template is rectangular position information including a tangent line obtained from a set of edge points obtained by scanning the input image.

According to a third aspect of the present invention, there is provided a method for aligning images,
3. A method according to claim 2, wherein, if it is determined that the image is a desired image, if the positional information of the input image and the positional information of the template are out of a predetermined range, generating warning data. Is further provided.

According to a fourth aspect of the present invention, there is provided a method of aligning images,
An image-to-image alignment method for aligning the body images of successive pages in an image input for each page of a book, wherein the step of determining that the body image is a body image is performed. A step of obtaining a template serving as a reference for alignment from the determined image; and a step of correcting a position between text images based on the template.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) FIG. 1 shows a block diagram of an inter-image positioning apparatus according to the present embodiment. In this apparatus, an original of a book separated for each page is read for each page as image data by an image input unit 101, and a page outline detection unit 102, a page content area extraction unit 103, a tilt correction unit 104, a page position correction unit 105,
In addition, the page information processing unit 107 performs processing described later using various buffers, and the image output unit 106 outputs the image data corrected for the alignment between pages.

FIG. 2 is a conceptual diagram showing the structure of a novel book as a book input to the apparatus. As is clear from FIG. 2, the title (cover) page, the table of contents page,
Body pages (even and odd pages), index pages,
It consists of a back cover page. Most of these pages are
The main body page is a main body page. In the main body page, a header area (in this example, a page is indicated), a footer area (in this example, what chapter is indicated), and a content area ( Body area). In the present invention, alignment between pages is performed by using such a feature of the book configuration.

Next, the processing of each section in FIG. 1 will be described. The image input unit 101 takes in a binary or multi-valued image from a scanner or the like and stores it in the input image buffer 108. This image may be a monochrome image or a color image. The approximate orientation of the image input is correct, starting at the beginning of the book or at the end of the book in page order. When using an auto feeder, taking into account that the input is performed at an incorrect inclination, it is desirable to make the input area of the scanner larger than the size of the page of the book when capturing an image with a scanner. This case will be described. If the page is larger than the input area of the scanner, it is divided and input, and the input image buffer 1
08 should be connected.

Next, the page outline detecting section 102 will be described. The image input by the image input unit 101 has an actual page area of a book and a background area behind it because the page size is smaller than the input area of the scanner. The page outline detection unit 102 distinguishes a background area and a page area from the input image, and extracts the outline of the page area. Here, when an existing book is separated and used for each page, the page end, which is the back portion, may be inclined or torn, and thus is often not an accurate rectangle. Therefore, the shape of the page outline can be approximated by a rectangle.

As a method of detecting the outline of the page area, a method of detecting an edge portion from an image and detecting that each angle of the rectangle is 90 degrees from the edge point, or scanning the image and detecting a change in luminance. There is a method of extracting a set of large edge points, obtaining a straight line from the set, and extracting a contour line.

FIG. 3 shows an example of a method for detecting the contour of the page area.
This will be described with reference to FIG. FIG. 3 is an explanatory diagram of the state of this method. FIG. 4 is a flowchart of the method. In the present invention, since the outline of the page area is a rectangle, a method for detecting the four outermost straight lines of the image will be described.

First, the case of detecting the straight line of the contour at the left end of the page area shown in FIG. 3A will be described. First, a line to be scanned is selected. Here, since scanning is performed in the horizontal direction, the top row is selected. Since scanning is performed from the left end to the right end, the point on the left end is set as an initial value (i = 0). The image is scanned in order, and the brightness at that point is obtained from the input image buffer 108. It is determined whether the point is an edge point. The determination method takes a first derivative in the horizontal direction. For example, a method using a Sobel filter is used. If it is determined that the point is an edge point, the coordinates of that point are stored, scanning of that line ends, and the next line is selected. This scan is performed for all lines up to the bottom row.
As a result of the above, a set of coordinates of the edge points is obtained. Many of them are gathered on one straight line, and the straight line is calculated. As a typical example of this method, Hough transform is performed. The above processing is performed for each of the four directions of the input image (corresponding to D = 0 to 3 in the flow of FIG. 4) (FIG. 3).
(See (b).) Four straight lines are obtained, and these straight lines are detected as the outline of the page area (FIG. 3C).

Next, the page content area extracting unit 103 will be described with reference to FIGS. The page content area extraction unit 103 extracts a page content area from an image within the outline of the page area obtained by the page outline detection unit 102. FIG. 5 is a diagram illustrating the state of this method.
FIG. 6 is a flowchart of this method.

As shown in FIG. 5A, an image is scanned line-sequentially, and an edge point is first extracted. This is the same method as the page outline extraction unit 102. The edge point is an edge with a character, a ruled line of a diagram or table, or an edge with a speech balloon portion of a comic. In this example, an edge point with a character area is obtained. If the obtained edge point set is a character string or the like, it does not necessarily exist on one straight line. Therefore, in the present invention, an outer tangent line (straight line) of these edge sets is obtained (FIG. 5B), and this straight line is used as a boundary line of the page content area (FIG. 5C).

Referring again to FIG. 6, how to determine the external tangent will be described. First, an edge point set is obtained. 2 from edge point set
A point is selected, and a straight line equation passing through these two points is obtained. When the coordinates of two points are (x1, y1) and (x2, y2), the straight line equation is (y2-y1) x- (x2-x1) y- (x1y2-x
2y1) = 0.

It is determined on which side of this straight line the other unselected edge point lies. This determination formula is: F (x, y) = (y2-y1) x + (x2-x1) y-
Assuming that x1y2 + x2y1, the point (x, y) is on the origin side when F (x, y) <0 and outside when F (x, y)> 0.

If all points are on the same side, the straight line equation is a tangent. Otherwise, another two new points are selected. If all the sets are performed, the tangent can be obtained.

By performing the above for each of the four scanning directions (corresponding to D = 0 to 3 in the flow of FIG. 6), it is possible to extract the page content area of the figure surrounded by the external tangent (FIG. 5). (C)).

Next, the tilt correction unit 104 will be described. The inclination correction unit 104 includes a page content area extraction unit 103
The rotation of the input image is corrected with respect to the reference coordinate axis based on the external tangent graphic extracted in step (1).
This process is performed for all pages of the book.

FIG. 7 shows how the image is rotated. The center of this rotation transformation is the coordinates (cx, cy) of the corner of the page content area
To When the input image is rotated by θ, the coordinates of a point on the input image are (x, y), and the coordinates on the corrected corrected image are (x ′, y ′). The conversion formula for this rotation is

[0038]

(Equation 1)

Is as follows.

In the rotation process, this formula is applied to each pixel of the input image buffer 108, and luminance or color is assigned to the converted coordinates on the corrected image buffer 109.

Next, the page information processing section 107 will be described. The page information processing unit 107 determines whether the input book image is a body page or another page. This determination method is a method of comparing the size or shape of the figure of the circumscribed tangent extracted by the page content area extraction unit 103, and determining that the image is a text page if the size or shape is within a predetermined size or shape. . In addition, since input is performed in the order of pages, the size of the rectangle of the external tangent line extracted by the page content area extraction unit 103 is substantially constant in the case of a text page, and therefore, the size of the rectangular shape of the external tangent line If the size continues to be almost constant, it is determined that the page is the body page from the first page, or the pages are sequentially read in the page order. There are also methods. If the page information processing unit 107 determines that the page is the body page, the page information buffer 1
10 stores the position information of the rectangular page content area of the circumscribed tangent line, and uses it as a template for positioning between body images, that is, between pages. If the position information of the template indicating that the page is a text page serving as a reference for page alignment has already been stored in the page position information buffer 110, the process proceeds to the page position correction unit 106.

Next, the page position correction unit 105 will be described. Although the inclination of the input image is corrected, the position of the page content area of the text differs depending on the input image due to a mechanical shift or the like at the time of reading. This means that if the text page is output as it is without correcting the position and the content of the text page is viewed as an electronic book with a viewer or the like, there is a displacement of the text page, which causes discomfort. Therefore, the page position correction unit 105 sets the page position information buffer 110
Then, the position of the image of the body page whose inclination has been corrected is translated and corrected so as to match the position information of the template indicating that the body page is stored once. As a result, the image output unit 106 can obtain image data with no positional deviation between the main text pages. That is, once a template is obtained from a page determined to be a text page, the text page following that page is aligned with reference to the template.

The template includes all of the header area, the footer area, and the content area of the body page. Each area is divided as shown in FIG. The information may be stored in the page position information buffer 110 in such a form. Such division facilitates character recognition, keyword extraction, and the like, and facilitates document structuring.

In the above description, all the processes are continuously automated and processed. However, the page outline detection unit 102, the page content area extraction unit 103, the inclination correction unit 10
Step 4 may be performed manually or.

(Second Embodiment) Normally, a book image is
It is automatically and sequentially input, and it is not known at the time of input whether the next input image is a body page or other. In addition, even if it is a body page, for example, the size of the page content area is different on the last page of the chapter. Therefore, even if the position information is determined to be a text page, the position information of the template in the page position information buffer 110 is significantly different from the position information of the page content area of the newly input image (when the position information is out of the predetermined range). ) Is regarded as an error and an error buffer (not shown)
An error bit is written for each page, and warning data is generated and left. Thus, after the automatic processing of the entire book is completed, the user can immediately see the page that must be manually corrected by viewing the warning data from the error buffer on the display means (not shown). .

The above-described series of processing can be realized by a program, but the program may be stored in a computer-readable recording medium such as an optical disk or a floppy disk, and may be read and used as needed.

[0047]

According to the first aspect of the present invention, it is possible to quickly align desired continuous images among images input for each page.

According to the second aspect of the present invention, since an external tangent is used, an accurate template can be obtained even in a character area, and the accuracy of positioning can be improved.

According to the third aspect of the present invention, since a registration failure between images can be detected, it is possible to conveniently perform, for example, correction during or after authoring.

According to the fourth aspect of the present invention, it is possible to quickly align the text images among the text images input for each page. As a result, the electronic book content can be created in a short period of time, and when the electronic book is viewed by the viewer, the positions of the body image when viewed by the viewer are aligned, so that discomfort can be eliminated.

[Brief description of the drawings]

FIG. 1 shows a block diagram of a first embodiment.

FIG. 2 is a conceptual diagram illustrating a configuration of a book.

FIG. 3 is a diagram for explaining page outline detection;

FIG. 4 is a flowchart of page outline detection.

FIG. 5 is a diagram for explaining page content area extraction.

FIG. 6 is a flowchart of page content area extraction.

FIG. 7 is a diagram for explaining rotation conversion.

FIG. 8 is an example of data stored in a page position information buffer.

[Explanation of symbols]

 101 Image Input Unit 102 Page Outline Detection Unit 103 Page Content Area Extraction Unit 104 Skew Correction Unit 105 Page Position Correction Unit 106 Image Output Unit 107 Page Information Processing Unit 108 Input Image Buffer 109 Corrected Image Buffer 110 Page Position Information Buffer

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Kazuyuki Nago 22-22, Nagaikecho, Abeno-ku, Osaka-shi, Osaka F-term (reference) 5B050 BA16 DA06 EA06 EA12 EA13 EA20 FA17

Claims (4)

[Claims] An image registration method for performing registration between images of consecutive pages in an image input for each page, comprising: a step of determining that the image is a desired image; An image alignment method comprising: obtaining a template as a reference for alignment from an image determined to be an image; and correcting a position between images based on the template. 2. The inter-image position according to claim 1, wherein the template is rectangular position information composed of external tangents obtained from a set of edge points obtained by scanning the input image. Matching method. 3. The method according to claim 2, further comprising the step of generating warning data when the position information of the input image and the position information of the template are out of a predetermined range when it is determined that the image is a desired image. 3. The method for aligning images according to claim 2, wherein: 4. An inter-image registration method for performing registration between body images on successive pages in an image input for each page of a book, the method comprising: determining that the image is a body image; An image alignment method comprising: obtaining a template as a reference for alignment from an image determined to be a text image; and correcting a position between text images based on the template. .