JPH0683928A - Constitution system for compound information - Google Patents

Abstract

PURPOSE:To generate the composite information at low cost by the constitution system for information by combining information represented with character strings with information represented with images. CONSTITUTION:A trapezoidal shape identifying device 4 determines the area of a layout image on a mount image at the time of the generation of a newspaper from image information inputted from an image data input device 1. A text data structuring device 5, on the other hand, extracts position information from the information, represented with the character strings, which is inputted from a text data input device 2 and converts it into a rule of coordinate representation on the mount image to structure the information. A stored data generating device 6 generates the composite information by combining those pieces of information represented with the character strings and images. Consequently, the composite information can be generated at low cost without any human operation for relating information and segmentation of the image information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a complex information or multimedia information processing system having a plurality of expressions such as text data and image data, and a plurality of information expressed by different expressions. Therefore, the present invention relates to a method of constructing complex information including the whole.

[0002]

2. Description of the Related Art Conventionally, in a system called a multimedia database, which handles complex information including various kinds of information such as character strings, voices, and images, a person directly needs a keyword or a keyword for searching. It was common to input a selection of different images. For example, in the case of a system in which articles in newspapers and magazines are searched by keyword using an information search system and the selected articles are delivered by facsimile, a person specifies the position of each article from the image input from the scanner and cuts it out. , Information such as keywords and dates was added to build complex information.

On the other hand, if the information digitized by a single medium is image data, an image of a newspaper page is transmitted from a newspaper company to a printing plant, or if it is text data, a process of producing a newspaper page. At CTS (Comput
er Typed Setting), it is provided as an information retrieval system that handles all information such as text, headings, and dates as character strings.

[0004]

However, in the conventional composite information generating means, information having different information transmission media such as image information or information expressed by a character string is treated as completely different information. In addition, human beings have performed the work of associating image information with keywords by character strings and cutting out image information related to information by character strings. As a result, the cost of building complex information is
There was a problem of being expensive.

The present invention has been made to solve the above problems, and an object thereof is to combine information represented by a character string and information represented by an image to combine complex information at low cost. It is to provide a complex information construction method that can be created.

[0006]

In order to achieve the above object, in the invention of claim 1, the information expressed by using at least one character string and the information expressed by using the character string are used. The image information of the mount image having at least one or more layout images, which is the image information composed of at least one or more element images, which is the image information expressing the same content as the information described above, and the individual characters. When three types of information, that is, position information indicating at which position on the layout image the information expressed in a column exists according to a predetermined rule are input, the individual information on the mount image is displayed. Means for determining a region of the layout image, and particularly means for converting the input position information into position information according to a rule on the layout image,
A unit for converting position information on the layout image into position information in accordance with a rule on the mount image according to information on a region where the layout image exists on the mount image; The information expressed by is structured so that it can be handled in relation to each other.

According to the second aspect of the invention, the input element image and the layout image composed of one or more element images are image information having regularity of positions by columns and rows, and are expressed by a character string. If the position information input together with the input information is expressed by the rule of columns or rows on the layout image, the position input on the layout image and the position input together with the information expressed by the character string In order to correlate the information, the means for determining the area of the layout image, in particular, means for identifying the height of one step from the distribution of black dots on the layout image and the distribution of black dots in each step on the layout image A unit is provided which has means for identifying the line spacing, and correlates the position information represented by the row and line that the information represented as the character string has with the position on the layout image. According to the rules of the position on the image is set to associating configure the information and elements image represented by the character string.

According to the third aspect of the present invention, the means for determining the area of the layout image particularly when the area for expressing the main body of information and the area for additional information exist, particularly the main body of the information, Means for determining a boundary line on the layout image information between the area to be expressed and the additional information area, and an area for expressing the main body of the information from the mount image information using the position of the determined boundary line. Of the image information, and the structure of the row and the row is extracted from the image information of only the main body of the information, and the position of the row and the row in the entire layout image information is determined.

Further, in the invention of claim 4, means for detecting the outer frame of the layout image when the input layout image is tilted, and the main information of the layout image in accordance with a predetermined standard. A means for identifying a position where a boundary line separating the additional information exists, and an image expressing the main information from the position information of the boundary line between the outer frame information, the main information, and the additional information. And means for calculating the center point of the area, and the image information after the mount image is rotated with the center point of the image having the main information as the center of rotation is input.

[0010]

According to the first aspect of the present invention, the mount image which is the input image itself, the layout image which is an area expressing information on the mount image, and the element images which represent individual information in the layout image For the image information consisting of, the area of the layout image on the mount image is determined, while the position of the element image attached to each information represented by the character string is input as the coordinate expression on the layout image. Then, by converting the coordinate expression on the layout image into the rule of the coordinate expression on the mount image, the information expressed by the character string and the information expressed by the image are combined,
I am able to create complex information. This eliminates the work of directly associating image information with keywords using character strings and cutting out image information related to information using character strings, reducing the overall cost of information construction. Is possible.

For example, both the image information on the page and the information on the date and page number are obtained from the hard copy information such as newspaper page and magazine page, and the digital facsimile image information distributed by the newspaper company to the printing plant. Enter the information represented by a character string that has the same information contained in the information represented by an image, such as the text of an article, and enter the date, page number, and image information of the information represented by the character string. Input the position information based on the ratio in the vertical and horizontal directions above, convert the position of each information included in the image to the representation of the position information that corresponds to the input character string, and use the character string. By combining the information generated from the information and the information generated from the image, the information expressed by the image and the information expressed by the character string can be converted into usable information in pairs. To.

On a newspaper page, a magazine page, a book, etc. as image information, the information is regularly arranged by columns or rows. According to the second aspect of the present invention, when information indicating the position of certain information expressed by columns or rows is input to an image structured by columns and rows, the information of the image is displayed by the spaces between the rows or the columns. Candidates for column height and row width are extracted, average column height and row width are calculated from the obtained candidates, and image information is generated from the calculated column height and row width. Make it easy to identify the shape of the mount when doing.

For example, on a newspaper page, two types of image information, that is, a body of information such as an article portion and an additional portion such as an advertisement portion under the article are combined and provided as one image information. ing. Of these, the information of the article section is arranged at the positions represented by columns and rows on the mount, but the advertisement under the article has a completely different arrangement method. As a result, the under-article advertisement may cause an error in identifying the step width and line width of the mount. According to the third aspect of the present invention, the boundary line between the article part and the under-article advertisement drawn in the full width of the area of the newspaper image or the like is determined, and the determined body area of the information such as the article part is cut out from the image information. By estimating the line and line of the mount from only the image of the main body of the information such as the article part,
Accurately identify the step width and line width of the mount.

When an image input device such as an image scanner or a CCD camera is used as a device for inputting information based on an image, the image may be inclined and input. According to the invention of claim 4, the outer frame surrounding the information represented by the image is identified, and the inclination of the entire image is calculated from the inclinations of the vertical ruled lines and the horizontal ruled lines of the outer frame of the image. By identifying the boundary of the area under the article advertisement, calculating the center of the area of the article from the coordinates of the boundary, and rotating the tilt of the image with the center of the article as the center point, there is less distortion in the article part. Obtain a corrected image.

As described above, according to the present invention, the position of the layout image on the mount image can be accurately determined, and the coordinate position on the layout image can be converted from the input position information. As a result, according to the present invention, even if the rule for determining the input position is different from the rule for the position on the image information,
Both can be handled in association with each other.

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings.

[Embodiment 1] FIG. 1 is a system configuration diagram of a composite information construction system showing a first embodiment of the present invention.
INDUSTRIAL APPLICABILITY The present invention is mainly effective for a general information retrieval system that uses a combination of information that treats information expressed as characters as an image and information that is expressed as a character string. Explain using a newspaper as information. Newspapers are information that mainly deals with character strings, such as headings and texts, and are usually handled as image information printed on paper.

In this embodiment, at least one or more image data input devices 1 for inputting information expressed as images, and at least one or more text data input devices 2 for inputting information expressed as texts are provided. An image data pre-processing device 3 that performs image data shaping processing such as correction of image inclination on an image input by the image data input device 1, and a line of a mount when a newspaper is created from the shaped image. A mount shape identifying device 4 for identifying the position of a column, and a text data structuring device for extracting and structuring information such as date, page, and position of a line from information of a character string input by the text data input device 2. 5 and stored data raw for performing processing for associating the structured article information and the paper image with each other and storing them in the database. A device 6, the generated storing complex information of seven devices of the composite information storage device 7.

For example, in the case of the newspaper image in this embodiment,
As shown in FIG. 7B, the mount image G described in the claims.
Reference numeral ₃ is the image itself input by the image data input device 1, layout image G ₂ is the image of the entire article excluding the newspaper name and date, and element image G ₁ is the image of the individual article.

FIG. 2 is a system configuration diagram of the image data preprocessing device 3 in the above configuration. Since the input image data may be slightly inclined, the image data preprocessing device 3 according to the present embodiment as shown in FIG.
Is a tilt correction mechanism 11 that determines the size of the tilt from the information of the outer frame of the newspaper image and corrects the tilt, and a decipher removal mechanism 12 that removes regions such as newspaper names and dates outside the outer frame called the page number and extracts the layout image. One subsystem consisting of

The above tilt correction mechanism 11 in the present embodiment.
Is an image buffer 13 that stores the input image data, a black spot counting module 14 that counts black spots for each line of the image in the image buffer 13, and a counting buffer 15 that stores the number of black spots on all the lines. An inclination calculation module 16 for calculating the inclination of the image image from the data of the counting buffer 15 and an image correction module 17 for rotating the image data of the image buffer 13 from the obtained inclination.

The processing of the tilt correction mechanism 11 in this embodiment is performed according to the procedure shown in the flowchart of FIG. First, the image data input by the image data input device 1 is stored in the image buffer 13, and as a step 31 from the image buffer 13, data is taken out line by line in the horizontal direction from the bottom to the top of the newspaper image by setting the pointer. , Step 3
2, the black spot counting module 14 measures the number of points determined to be black spots on the line within a predetermined range centered on at least two or more points in one line, and calculates the number of black spots. The measurement buffer 15 sequentially stores them. When the number of black dots on all the lines is accumulated in the counting buffer 15, the slope calculation module 16 executes step 3
As 3, the pointer is set in order to extract the count value from the count value of the black dot at the bottom of the image of one candidate point in the count buffer 15 toward the upper part of the image. Step 34
As a candidate line, the line number of the line that exceeds the line candidate sunspot count for the first time is used as a candidate line and the slope is calculated. It is stored in the start line candidate buffer in the module 16.

Next, in step 35, the number of black spots in the counting buffer 15 is further compared with the number of line candidate black spots, and the line smaller than the number of line candidate black spots is determined as an end line in the slope calculation module 16. Store in the candidate buffer. Next, at step 36, the difference between the value of the end line candidate buffer and the value of the start line candidate buffer is calculated and compared with a reference frame line width given in advance. If the difference value is within the range of the reference frame line width, in step 37, the average (intermediate value) of the values of the end line candidate buffer and the start line candidate buffer is calculated for each candidate point in the slope calculation module 16. Stored in the candidate line buffer of. If the difference value is outside the range of the reference line width, the process returns to step 34 and the start line candidate is examined. Next, in step 38, if the processing of all the candidate points is not completed, the process returns to step 33, and the pointer is set to another candidate point.

When the candidate lines have been determined for all the candidate points, in step 39, the tilt angle is calculated from the distance between each candidate point and the distance between the candidate lines. For example, when two candidate points are used as the tilt angle, when the newspaper image has 3000 dots per line, for example, 10 candidate points are selected.
Assuming that there are 00 dots and 2000 dots, and the candidate lines at that time are 3910 lines and 3917 lines, respectively, the difference between the lines is 7, and the interval between the candidate points is 1000 dots, so tan ⁻¹ (7/1000) = The degree of inclination is 0.4 °.

In step 40, the data in the image buffer 13 is rotated in the image correction module 17 using the calculated tilt magnitude. For example, when the inclination of the candidate line is 0.4 ° with respect to the clockwise direction, if it is supposed to rotate around the point at the lower left corner, it is sufficient to rotate 0.4 ° because of the rotation in the first quadrant. Since the image data takes a positive value in the downward direction of the image with the upper left corner as the origin, it is necessary to rotate the mount image by -0.4 ° with the upper left corner as the origin. For example, if the horizontal coordinate of the image is x, the vertical coordinate is y, the calculated horizontal coordinate is U, and the calculated vertical coordinate is V, the rotation algorithm is = Xcos (-0.4 °) -ysin (-0.4
.Degree.), V = xsin (-0.4.degree.) + Ycos (-0.4.degree.) To calculate a new coordinate.

The image data whose inclination has been corrected in this way is sent to the page number removing mechanism 12 and separated into an article part and a region outside the frame. The defibrillation mechanism 12 includes an image buffer 18 that stores an image whose inclination has been corrected, and a frame line that measures the number of points determined to be black dots for each vertical and horizontal line on the image in order to estimate the frame line. A measurement module 19, a frame line buffer 20 that stores the number of black points for each line, and an end point determination module 21 that determines the four end points of the article part from the value of the number of black points in the frame line buffer 20.
And an image extraction module 2 that cuts out only the image of the article part from the image of the image buffer 18 based on the end point information.
It consists of two.

As the processing of the number removing mechanism 12 in this embodiment, the procedure shown in the flowcharts of FIGS. 4 (a) and 4 (b) is performed. First, as shown in FIG. 4A, the frame line measuring module 19 is used by using the image data after the tilt correction stored in the image buffer 18.
Then, in step 41, the pointers are set to the upper and lower ends of the newspaper image, and the data is taken out one line each in the horizontal direction from the lower and upper parts of the newspaper image. In step 42, the number of points determined to be black dots is measured. , "Upper line buffer" and "lower line buffer" of the frame line buffer 20
The value is stored in each corresponding place of. Next step 4
In step 3, the pointer is set to the left and right edges of the image to extract data line by line in the vertical direction, and the number of points determined to be black dots is measured in step 44, and the "left line" of the frame line buffer 20 is measured. Store the value in the corresponding place of "buffer" and "right line buffer".

Next, at step 45, the end point determination module 21 sets the pointer to the left frame line buffer in the frame line buffer 20, and as step 46, FIG.
The left end is calculated by the processing of the frame line determination described later in (4) and stored in the left point buffer of the image extraction module 22.
Similar operations are performed in steps 47 to 48, using the value of the "right line buffer" to set the right point, in steps 49 to 50 using the value of the "upper line buffer" to set the upper point, and in steps 51 to 52, "down." The lower point is calculated using the value of "line buffer".

Next, in step 53, the image extraction module 22 extracts only the image (layout image) of the article part using the values of the four points above and below and to the right and sends it to the mount shape identifying device 4.

As shown in the flowchart of FIG. 4B, the frame line determination module 19 determines whether to use a vertical frame line threshold value or a horizontal frame line threshold value to determine whether to use the vertical frame line threshold value or the horizontal frame line threshold value. Which end point is to be determined is specified, and in step 54, the number of black dots is sequentially taken out from the line buffer of the frame line buffer 20, and then in step 55, the predetermined frame line threshold value and the number of black dots are compared. When the threshold value is exceeded for the first time, the frame line flag is set to 1 in step 56. Next, at step 57, it is determined whether the frame line threshold is smaller than 1 and the frame line flag is 1,
If it is true (Yes), the current line is determined to be the end line of the article in step 58. Therefore (N
In the case of o), the process further returns to step 54, and the values are sequentially taken out and processed.

FIG. 5 shows the system configuration of the mount shape identifying device 4 in this embodiment. The mount shape identifying device 4 is divided into an image buffer 61 for storing image data of only the article part, an image dividing module 62 for horizontally dividing the image data in order to estimate the width of the step of the mount, and is divided. A step width estimation module 63 that determines the step width by counting the number of black dots for each line in the vertical direction, and a step image division module 64 that cuts the image data into the step width determined by the step width estimation module 63. 1, a line width estimation module 65 that horizontally scans a step image to determine a line width, and a combination of the determined step width, line width, and image data.
The image data construction module 66 for generating data for the composite information storage device 7 shown in FIG.

The processing steps of the mount shape identifying device 4 in this embodiment are carried out according to the procedure shown in the flowchart of FIG. First, with respect to the data of the image buffer 61 which stores the image data only in the article part, in step 71, the image dividing module 62 divides a newspaper image of a predetermined width in the vertical direction and cuts it out vertically to obtain black dots. Substitute -1 for the flag. The cut-out image data is sent to the step width estimation module 63, and in step 72, the number of black dots on each line is measured, and the number of black dots is compared with a predetermined threshold value, and the black dot flag is larger than the threshold value. If -1 is -1, the line number is stored (substituted) in the black dot flag in step 73. If the line number is smaller than the threshold and the black dot flag has a line number, then in step 74, the difference between the current line number and the number in the black dot flag is used as the step width value, and the step width buffer in the step width estimation module 63 is set. Add 1 to the value of the slot of the corresponding step width. Further, in step 75, -1 is substituted for the black spot flag. When the step width estimation is completed for all blocks, the step width error range from the maximum frequency step width is determined according to a predetermined step width error range value from the step width (maximum frequency step width) of the slot having the largest value in step 76. A step width candidate value is determined by performing a weighted average of the step widths in. Further, the height of the layout image is divided by the step width candidate value. In information structured by columns such as newspaper images, the number of columns is always an integer value, and blank bands and ruled lines are inserted between each column, so the column width candidate value is smaller than the actual column width. Therefore, the number after the decimal point is rounded down and only the integer part is extracted as the number of stages. Then, the height of the layout image is divided by the number of steps, and the whole number is rounded off to obtain the integer part, which is taken as the step width. For example, in the case of image data of a newspaper input at a resolution of 400 dpi, if the given step width error range is 10 dots, and the maximum frequency step width is 490 and the weighted average value is 491 dots,
The step width candidate value is 491 dots. If the height of the layout image for identifying the step width is, for example, 7763 dots, the number of steps is identified as 7763/491 = 15 steps,
The identified step width is 7763/15 = 517 dots.

Next, based on the step width calculated by the step width estimation module 63, the step image division module 64
In step 77, the image data in the image buffer 61 is divided into horizontally long images according to the identified step width.
Further, in the line width estimation module 65, step 78
As a substitute, 0 is assigned to the black spot flag and 0 to the candidate line. Next, in step 79, one line is taken out in the vertical direction, the number of black dots in each line is measured, the predetermined threshold value is compared with the number of black points, and if it is larger than the threshold value, step 80 is executed. 1 for the sunspot flag
To store. If it is smaller than the threshold value and the black spot flag is 1, the difference between the current line number and the number in the candidate line is set as the line width in step 81, and the line width estimation module 65 is used.
Add 1 to the value of the slot of the appropriate row width in the row width buffer in. In step 82, the black spot flag is set to 0 and the current line number of the candidate line is substituted. When the line width estimation is completed for all blocks, the line width is within the error range from the maximum frequency line width according to a predetermined line width error range from the line width (maximum frequency line width) of the slot having the largest value in step 83. The line width is determined by weighted averaging the line width frequencies. For example, in the case of newspaper image data input at a resolution of 400 dpi, if the line width error range given in advance is 5 dots and the maximum frequency line width is 62 dots and the weighted average value is 62 dots, The identified line width is 62 dots.

Finally, the image data construction module 66
In step 84, the step width and line width determined in step 84 and the image data are sent to the stored data generation device 6 as structured data.

The text data structuring device 5 in the present embodiment is structured such as a pattern matcher 8 for extracting information such as date and page from the information described in text by pattern matching as shown in FIG. It comprises a structured slot 9 for temporarily storing information, and a pattern matching database 10 for storing knowledge used by the pattern matcher 8. Text data as shown in FIGS. 7A and 7B is sent to the text data structuring device 5 in this embodiment from the text data input device 2 which is a text data input unit.

The text data structuring process of the text data structuring device 5 in this embodiment is performed according to the procedure of the flowchart of FIG. From the input data, for example, in pattern matcher 8, step 85
The information of the character string input as is extracted line by line, and in step 86, "* date *" up to the first blank of each line
Separate the pattern part such as and the text part of the attribute data that follows it. Next, in step 87, the pattern matching database 10 is searched using the pattern,
Get the rules for converting attribute data. Step 88
According to the rule searched for as, for example, “92061
8 ”is converted into a numerical value of (92618) and stored in the structured slot 9. When the data for one article has been structured, it is sent to the stored data generation device 6 in step 89.

As an example of the rule stored in the pattern matching database 10 described above, for example, in the case of "* date *", the character string "920618" is cut out by two characters, and the character string "92" is further converted into characters ""9" is cut out, and numerical data of 9 is obtained from a table of numerical values corresponding to characters between "0, 1, 2, ..., 8, 9",
This is multiplied by 10 and the numerical value 2 is obtained from the character "2" and added to 90 to obtain 92. Since the other data shown in FIG. 7 can be easily analogized, the description thereof will be omitted.

The stored data generating device 6 uses the position of the line in the text data structured by the text data structuring device 5 and the data of the line width and line width obtained by the mount shape identifying device 4. Then, the position where each article exists is converted into a coordinate point on the image data, and the text data,
The image data and the image data are stored in the composite information storage device 7.

The image data input device 1 in this embodiment may be an image scanner or a digital image transmission device by digital facsimile distributed by a newspaper company to a printing factory.

The image information in this embodiment has image data divided into page units as shown in FIG. 7, and at least information on newspaper name, date, and page.

The text data input device 1 in the present embodiment is a newspaper article text data reading device which can be obtained by a database retrieval system from an article text database or a computer data exchange medium such as a magnetic tape. . As another text information input device, there is a CTS (Computer T) which is a typesetting system using a computer system used in a newspaper company or the like, which internally has character string information and position information of a main body.
typed setting) and DTP (Deskto
p Publishing System).

The article text data in this embodiment is
As shown in Figure 7 (c), it is separated into individual articles,
It has information such as at least date, newspaper name, page number, position of article on paper, and text of article.

[Embodiment 2] FIG. 9 is a system configuration diagram of a mount shape identifying apparatus in a second embodiment of the present invention. The article section on the newspaper page can be divided into two areas, the article headquarters and the under-article advertisement, and the layout of the under-article advertisement is almost completely different from the mount structure of the mount, so
In the present embodiment, an example is shown in which the accuracy of identifying the shape of the layout image on the mount image can be increased by adding a means for separating the post-article advertisement as the mount shape identifying device. The configuration other than the mount shape identifying device is the same as in FIG. In general, the article headquarters and the sub-article advertisement are separately produced in advance and finally combined as image information, so the horizontal ruled line that indicates the boundary between them is set to the full width of the layout image. It is always drawn (boundary line L shown in FIG. 7). Therefore, when the number of black dots on each line in the horizontal direction is measured from the newspaper image, the number of black dots is the same as the number of dots in the horizontal width only on the horizontal ruled line at the boundary between the article headquarters and the under-article advertisement except the outer frame.

As an embodiment of the trapezoidal shape identifying device 4 capable of separating the sub-article advertisement, as shown in FIG. 9, the longest horizontal ruled line extraction module 91 and the image division module 92 are used to identify the mount shape shown in the first embodiment. Add to device 4.

The process of separating the post-article advertisement in the mount shape identifying device 4 in this embodiment is performed by the procedure shown in the flowchart of FIG. First, as shown in FIG. 10, the longest horizontal ruled line extraction module 91 sets a white line flag and a black line flag in step 101 as to the image data from which the page number is removed, which is obtained from the image data preprocessing device 2 in FIG. Substitute -1. Step 1
As 02, data is taken line by line from the lower end to the upper end of the newspaper image, and at step 103, the number of data judged as black dots is counted. Then step 104
When the white line flag is -1 and the number of black dots is 0, the line number is stored in the white line flag in step 105. If the white line flag is not -1 in step 106 and the number of black dots is equal to the width of the image, the line number is stored in the black line flag in step 107. In step 108, the line numbers are stored in both the black line flag and the white line flag (-1
If the number of black dots is 0, the line number is sent to the image separation module 92. Step 1
If the number of black dots is 1 or more and less than the width as 09, the process returns to step 101 and the white dot flag and the black dot flag are set to −.
Substitute 1

Next, the image division module 92 cuts out the image of only the article part using the line number of the article part obtained by the longest horizontal ruled line extraction module 91 and stores it in the image buffer 61. Subsequent processing is the first
It is almost the same as the embodiment.

As another embodiment of the longest horizontal ruled line extraction module 91, when a newspaper image printed on paper is read using an image scanner as the image data input device 1 of FIG. 1, the input image is input. There is a possibility that the number of sunspots at the boundary between the article headquarters and the post-article advertisement may be a value of 1 or more and less than the width due to factors such as tilting and noise. Therefore, there are 2
Two thresholds are provided, and when the number of black dots is larger than the advertisement separation threshold black, it is equal to the maximum horizontal line width in FIG. 10, and when the number of black dots is smaller than the advertisement separation threshold white, it is considered as a blank band. . As a result, it is possible to separate the article headquarters from the article advertisement even for data that has errors due to noise or inclination.

[Third Embodiment] Next, a third embodiment of the present invention will be described. In the image correction module 17 of FIG. 2, in the case of a newspaper image, when an image having the same quality as that of a digital facsimile is input by the image scanner, since there are about 10,000 dots in the vertical direction, rotation is performed with the upper left corner of the image as the origin. Then, even if the inclination is about 0.1 °, there is almost no change in the vicinity of the origin.

Therefore, in the image correction module of this embodiment, the center of rotation is located at the center of the newspaper image. For example, if the input image of a newspaper has 9200 dots in the vertical direction and 5820 dots in the horizontal direction, the calculation is performed with the point of 4,600 dots in the vertical direction and 2910 dots in the horizontal direction as the origin. As a result, the formula is
For example, if the horizontal coordinate of the image is x, the horizontal coordinate is y, the calculated horizontal coordinate is U, and the calculated vertical coordinate is V, then U = (x-2910) ) × cos (−0.1 °) − (y−
4600) × sin (−0.1 °) +2910, V = (x−2910) × sin (−0.1 °) + (y−
4600) × cos (−0.1 °) +4600 to calculate a new coordinate point. As a result, the distortion is separated and becomes about 10 dots at most.

Further, as the image data preprocessing device in the present embodiment, in the case of a newspaper image, since an under-article advertisement is attached, when only the article headquarters is required, the article headquarters is detected and the article headquarters is detected. If you rotate using the center,
The average distortion at the article headquarters is even smaller. For example,
The advertisement below the front page of a newspaper occupies 20% of the height of the newspaper. This case can be achieved by incorporating the longest horizontal ruled line extraction module 91 in the image data preprocessing device 3 as shown in the system configuration diagram of the image data preprocessing device in FIG. As shown in FIG. 11, as one embodiment of the image data pre-processing device 3, the inclination correction mechanism 11 and the non-blem removing mechanism 12 are provided in two.
A set and a longest horizontal ruled line extraction module 91.

As the processing process of the image data preprocessing device 3 in the present embodiment, the longest horizontal ruled line extraction module 91 uses the region of the article part for the image information obtained by the processing process of the inclination correction shown in FIG. Then, the image correction module 17 in one of the tilt correction mechanisms 11 rotates about the center coordinates of the article headquarters. As a result, the distortion of the article headquarters is further reduced, and only about 8 dots can be moved in the horizontal direction.

[0052]

[Effects of the Invention] Conventionally, information that uses a laid out image as an information transmission medium, such as a newspaper page or a magazine page, and a character code used by a computer for the same content as the information, and the same is printed on paper. Characters in a character string and information to be used as a transmission medium have been treated separately, but according to the present invention, by adding information on the position on the image information to the information using a character as a transmission medium, It is possible to handle the type of information as one piece of information on a computer, for example.

In particular, the present invention is provided with means for determining the position of each element image in the layout image in the mount image, even if the coordinate points on the mount image are different due to the difference in the input device of the image data. As a result, it is possible to create complex information from the information on the image and the information expressed by the character string by only inputting the position information according to the coordinates on the layout image. The efficiency of creating accurate information is greatly increased.

According to the third aspect of the present invention, means for determining the area of the main body of information such as the article headquarter is particularly provided for information divided into the article headquarter and the advertisement portion such as newspaper articles. With the provision, it is possible to accurately determine the position of the element image of the image of the information body, and it is possible to reduce the distortion of the layout image and the element image.

Further, according to the invention of claim 4, when an image input device such as an image scanner in which an image is likely to be input with inclination, is used, distortion is generated even if the image is input with inclination. A small amount of complex information can be obtained.

[Brief description of drawings]

FIG. 1 is a system configuration diagram showing a complex information construction method according to a first embodiment of the present invention.

FIG. 2 is a system configuration diagram of the image data preprocessing device in the first embodiment.

FIG. 3 is a flowchart of a tilt correction process in the first embodiment.

4 (a) and 4 (b) are flow charts of the process of the pumble removing mechanism in the first embodiment.

FIG. 5 is a system configuration diagram of the mount shape identifying device in the first embodiment.

FIG. 6 is a flowchart of processing of the mount shape identifying device in the first embodiment.

7 (a), (b) and (c) are diagrams showing an example of data in the first embodiment.

FIG. 8 is a flowchart of text data structuring processing in the first embodiment.

FIG. 9 is a system configuration diagram of a mount shape identifying device according to a second embodiment of the present invention.

FIG. 10 is a flowchart of article down advertisement separation processing in the mount shape identifying device in the second embodiment.

FIG. 11 is a system configuration diagram of an image data preprocessing device in a third embodiment of the present invention.

[Explanation of symbols]

 1 ... Image data input device 2 ... Text data input device 3 ... Image data pre-processing device 4 ... Mount shape identification device 5 ... Text data structuring device 6 ... Stored data generation device 7 ... Complex information storage device 8 ... Pattern matcher 9 ... Structured slot 10 ... Pattern match database 11 ... Slope correction mechanism 12 ... French removal mechanism

Claims (4)

[Claims] 1. Information represented by at least one character string and at least one or more image information representing the same content as the information represented by the character string by an image. Image information of a mount image having at least one layout image which is image information composed of element images, and which position on the layout image the information represented by each of the character strings is in accordance with a predetermined rule. Means for determining the area of each of the layout images on the mount image when three types of information, that is, the position information indicating whether or not the input position information is present, are input to the layout image. A means for converting the position information according to the rule on the image, and the position information on the layout image into information on the area where the layout image exists on the mount image. Therefore, it is characterized by having means for converting into position information in accordance with the rule on the mount image, and structured so that the image information and the information expressed by the character string can be handled in association with each other. Complex information construction method. 2. The composite information construction method according to claim 1, wherein the layout image composed of the input element image and one or more element images is image information having regularity of positions by columns or rows. , And the position information input together with the information represented by the character string is represented by the rule of the step or line on the layout image, the position on the layout image and the character string are represented. In order to correlate the position information input together with the information, the means for determining the area of the layout image, in particular the means for identifying a step height from the distribution of black dots on the layout image, It has a means for identifying the line spacing from the distribution of black dots in the columns, and associates the position information represented by the columns and rows of the information represented as the character string with the position on the layout image. A composite information construction method characterized in that the information represented by the character string and the element image are associated with each other according to the rules of the position on the mount image. 3. The composite information construction method according to claim 1, wherein the layout image area is determined particularly when an area representing the main body of information and an additional information area are present. In particular, means for determining a boundary line on the layout image information between the area expressing the main body of the information and the additional information area, and from the mount image information using the position of the determined boundary line. And a means for cutting out image information of an area expressing the main body of the information, and further extracting the structure of steps and rows from the image information of only the main body of the information to determine the positions of the steps and rows in the entire layout image information. A complex information construction method characterized by making a judgment. 4. The composite information construction method according to claim 1, 2 or 3, wherein a means for detecting an outer frame of the input layout image when the layout image is tilted, and the layout image are stored in advance. Means for identifying the position of the boundary line separating the main information and the additional information according to a predetermined standard, and the position of the boundary line between the outer frame information, the main information and the additional information And means for calculating the center point of the area of the image expressing the main information from the information, and inputting the image information after rotating the mount image with the center point of the image having the main information as the center of rotation. A complex information construction method characterized by being performed.