JP2001084258A - Translation information collecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately narrow search range where pages are checked whether they are a pair and to improve processing speed by using a property that a direct link is given to a pair of pages being translation pages or that a different pair of translation pages to which the direct link is given exist in the center of the route of a link connecting the pair. SOLUTION: A translation page candidate accumulating means 13 stores a pair of translation page candidates that a translation page candidate generating means 12 generates and it is formed of a pair of page names. The translation page candidate accumulating means 13 has plural translation page candidate accumulating means 131, 132, etc., according to a distance (m) for searching the translation page candidates from a pair of translation pages. A translation page decision means 14 decides whether a pair of translation page candidates stored in the translation page candidate accumulation means 13 are the translation pages. A translation page accumulating means 15 accumulates a pair of translation pages that the translation page decision means 14 decide as being the translation pages.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a bilingual information collecting apparatus for collecting, as bilingual information, pages describing the same contents in a plurality of languages from data on the WWW (World Wide Web) via a network.

[0002]

2. Description of the Related Art On the WWW, there are pages that provide the same contents in a plurality of languages, and pages having the same contents described in different languages (hereinafter referred to as bilingual pages). ) Is described in the following document.

[0003] Reference 1 "Information Processing Society of Japan Research Report Natural Language Processing 128-18 1998.11.6 Name; Automatic Collection of Bilingual Data from WorldWide Web" A system for searching for a bilingual candidate page from among the text information and creating bilingual data by associating the bilingual candidate pages with each other in units of one sentence is described.

In this system, by utilizing the fact that each bilingual page exists in the same domain, for a certain page on the WWW, a page having a link from that page is collected as a bilingual candidate page, and the bilingual translation page is collected. Find out in detail whether the candidate page is really a bilingual page.

[0005] That is, the Japanese page of content A is A
j, if the English page is Ae, there is a possibility that Aj and Ae are bilingual pages only when Aj has a description of a link to Ae and Ae has a description of a link to Aj. It is determined that there is a translation candidate page. Then, it is checked in more detail by associating the bilingual candidate page with the bilingual page in a sentence unit.

[0006]

[0005] However, Document 1
In the prior art described in (1), the collection of bilingual translation candidate pages is limited to pages directly linked from a certain page, and the link destination is limited to pages having the same domain, so that no direct link is set from a certain page In such a case, there is a problem that the translation page cannot be found.

More specifically, there are a Japanese page Aj and an English page Ae to which a link is provided, and a Japanese page Bj to which a link is provided from Aj, and a link to be provided from Ae. If there is no English page Be and no link to each other is described in Bj and Be, it is determined that Bj and Be are bilingual pages even if Bj and Be are bilingual pages. Can not do.

When describing a plurality of pieces of information on the actual Internet, a page of a table of contents is often described, and a page corresponding to each item in the table of contents is often expressed by a plurality of pages (HTML documents). When publishing information shown on a plurality of pages in a bilingual manner, a link to another language is generally provided only on the page of the table of contents. Therefore, in a system that targets only pages with direct links as described above, pages of items that are not directly linked cannot be collected as bilingual pages.

As an extension of the system described in the prior art, as a candidate for a bilingual page, not only a page linked to a certain page but also a page from that page to a linked page is set. Check if it is a bilingual page ". However, with such a simple extension, the number of candidates for the bilingual page becomes the square of the system of the related art, and the search range is increased. Furthermore, if it is expanded one more stage, the number of candidate translation page candidates will be raised to the third power.

[0010] In view of the above, an object of the present invention is to provide a bilingual information collecting apparatus capable of collecting bilingual pages which will be further increased on the WWW in the future.

[0011]

Means for Solving the Problems In order to solve the above problems, a first invention of the present invention is to form two pages in two pages which are not directly linked but are indirectly linked by a link. A central bilingual page determining unit that determines whether a pair of bilingual page candidates existing at the center of the linking link is a bilingual page that describes a document of the same content in a different language, and A translation page candidate generating means for generating a pair of translation page candidates by combining pages separated by a distance m from each of the translation page pairs determined to be present, and a translation page candidate pair generated by the translation page candidate generation means. And the central bilingual page determining means or the bilingual page determining means determine that the bilingual page is a bilingual page. And having a translation page output means for outputting the pair of the translation page.

A second invention relates to a bilingual page candidate generating means, wherein the central bilingual page determining means separates the bilingual page at a distance m from each pair of bilingual pages determined to be a bilingual page, and in the middle or at the end point. It is characterized in that pages not containing are extracted and all the extracted pages are combined to generate a pair of bilingual page candidates.

According to a third aspect of the present invention, there is provided a search start point input means for inputting a search start point for collecting bilingual pages describing documents having the same contents in different languages, a search start point page and a distance n from the search start point. First parallel translation page candidate generating means for generating a pair with the selected page as a parallel translation page candidate pair, and the central bilingual page determination means includes a translation page candidate pair generated by the first parallel translation page candidate generating means. Is determined to be a bilingual page.

The fourth invention relates to bilingual data, and is characterized by having bilingual data generating means for generating bilingual data and bilingual data outputting means for outputting bilingual data.

The fifth and sixth inventions relate to an untranslated page. The fifth invention has non-translated page storage means for storing pairs of non-translated pages, and when the pair of bilingual page candidates matches the pair of non-translated pages stored in the non-translated page storage means, It is determined that the pair of the translation page candidates is a non-translation page.

According to a sixth aspect of the present invention, the non-translated page storage means stores a value indicating the page name and page content of the pair of non-translated pages. It is determined whether the page name of the pair of bilingual page candidates matches the page name of the pair of non-bilingual pages stored in the non-bilingual page storage means, and if they match, the pair of bilingual page candidates is determined to be a non-bilingual page. Determined, if they do not match, determine a value indicating the page content of the pair of bilingual page candidates, determine again whether it matches the value indicating the page content of the pair of non-translated page stored in the non-translated page storage means, When they match, the pair of bilingual page candidates is determined to be a non-bilingual page.

[0017]

DETAILED DESCRIPTION OF THE INVENTION (Features of the Present Invention) A pair of pages to be a bilingual page is directly linked to each other, or a direct link is set at the center of a link path connecting the pair. In addition, there is a property that a pair of other bilingual pages often exists. The present invention intends to narrow the search range of a bilingual page candidate by utilizing this property.

Here, the center of the link path will be described using an example. For example, when the page A and the page D are not directly linked, but the page A is linked to the page B, the page B is linked to the page C, and the page C is linked to the page D, the link between the page A and the page D is changed. The center of the path indicates a link between page B and page C.

Hereinafter, a bilingual information collecting apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings.

(A) Configuration of the First Embodiment FIG. 1 is a block diagram showing a first embodiment of a bilingual information collecting apparatus according to the present invention, which is realized on an information processing apparatus such as a workstation or a personal computer. Is what is done.

In FIG. 1, the bilingual information collecting apparatus according to the first embodiment includes a search starting point input unit 11, a bilingual page candidate generating unit 12, a bilingual page candidate storing unit 13, a bilingual page determining unit 14, a bilingual page determining unit 14, It has a page storage means 15 and a bilingual page output means 16.

The search start point input means 11 is for inputting a start point (page) at which a search for a bilingual page is started, and is composed of, for example, a keyboard or the like. Search start point is UR
It can be indicated by L (Uniform Resource Locator) or the like.

The bilingual page candidate generation means 12 generates a bilingual page candidate pair that can be traced at a distance n from the search start point input from the search start point input means 11.

Here, the distance between pages will be described. For example, if the distance between two pages is 1,
It means that a link is provided directly from at least one page to the other page. There are three pages, “Page A, Page B, and Page C”, and “Page A and Page B” are at a distance of 1, and “Page B and Page C” are at a distance of 1.
Is linked directly to "Page A and Page C"
Is not directly linked, the distance between “Page A and Page C” is 2. When two pages are linked by two or more routes, the minimum distance is defined as the distance between the two pages.

More specifically, the pair of bilingual page candidates at a distance of 1 generated by the bilingual page candidate generating means 12 is a pair of a page at a distance of 1 from the search start point and the search start point. A pair of bilingual page candidates at a distance of 3 includes a page at a distance of 1 from the search start point and not a bilingual page, and a page at a distance of 1 from the bilingual page at the search start point and not at the search start point. It is a combination of

A page which is a pair of a translation page candidate having a distance of 3 will be described with reference to FIG. FIG. 2 shows a link example of WWW data, in which "Page A is directly linked to Page B, Page C, and Page D", and the distance is 1 for each. Further, “Page D is directly linked to Page E, Page F, Page A”, and the distance is 1 for each.

Here, "page A and page D" are bilingual pages, and the search start point is page A. A pair of bilingual page candidates with a distance of 3 is at a distance of 1 from page A at the search start point, and is not a bilingual page at “Page B, Page C”, and is at a distance of 1 from a bilingual page D, and "Page B and page E", "page B and page F", "page C and page E", and "page C and page F" are combinations of "page E and page F" which are not search start points.

The bilingual page candidate accumulating means 13 stores the pair of bilingual page candidates generated by the bilingual page candidate generating means 12, and comprises a pair of page names. The bilingual page candidate accumulating means 13 has a plurality of bilingual page candidate accumulating means according to a distance m for searching for a bilingual page candidate from a pair of bilingual pages. For example, when a range from a pair of bilingual pages to a distance 2 is set as a bilingual page search range, the bilingual page candidate storage unit 13 includes bilingual page candidate storage units having a distance between pairs of 1, 3, and 5. .

Here, the distance z between pairs is z = 2 × m +
1 (m: distance from the bilingual page)
The bilingual page candidates are accumulated in the bilingual page candidate accumulating means 13 corresponding to the distance between pairs.

In this case, it is assumed that a distance 1 parallel translation page candidate storage unit 131 and a distance 3 parallel translation page candidate storage unit 132 are provided.

Distance 1 bilingual page candidate storage means 131
A page which can be reached at a distance 1 from the search start point generated by the bilingual page candidate generating means 12 is stored as a bilingual page candidate with a distance 1.

The distance 3 bilingual page candidate storage means 132
The bilingual page candidate of distance 3 generated by the bilingual page candidate generating means 12 based on the pair of bilingual pages of distance 1 is stored.

The bilingual page determination means 14 determines whether the pair of bilingual page candidates stored in the bilingual page candidate storage means 13 is a bilingual page. Conventional technology can be used to determine whether the page is a bilingual page.

For example, when the prior art of Document 1 is used, two pages are correlated with a sentence and the likelihood of a translated page is digitized. Thereafter, using an appropriate threshold set in advance, a page having a higher likelihood of a bilingual page than the threshold is determined to be a bilingual page. Further, the result of the two page sentence correspondence performed by the bilingual page determination means 14 is stored in the storage means (not shown).

The bilingual page accumulating means 15 accumulates a pair of a bilingual page determined as a bilingual page by the bilingual page determining means 14, and is composed of a pair of page names. Similar to the bilingual page candidate accumulating means 13, the bilingual page candidate accumulating means 13 includes a plurality of bilingual page accumulating means according to the distance m for searching for a bilingual page candidate from a pair of bilingual pages. are doing.

Here, the distance 1 bilingual page storage means 15
1 and distance 3 bilingual page storage means 152 is provided. Distance 1
The bilingual page storage unit 151 and the distance 3 bilingual page storage unit 152 respectively store the bilingual page candidate storage unit 13 with a distance 1.
1 and distance 3 correspond to the bilingual page candidate storage unit 132.

The distance 1 bilingual page storage means 151 stores the pair of the bilingual page determined as the bilingual page by the bilingual page determining means 14 from the bilingual page candidates having the distance 1 stored in the distance 1 bilingual page candidate storing means 131. accumulate.

The distance 3 bilingual page storage means 152 stores a pair of the bilingual page determined as the bilingual page by the bilingual page determining means 14 from the bilingual page candidates at the distance 3 stored in the distance 3 bilingual page candidate storing means 132. accumulate.

The bilingual page output means 16 outputs the pair of bilingual pages stored in the bilingual page storage means 15. Here, the distance 1 bilingual page storage means 151 is used.
And a distance 3 bilingual page pair stored in the bilingual page storing means 152 is output.

(A-1) Operation of First Embodiment FIG. 3 is a flowchart showing the operation of the first embodiment of the bilingual information collecting apparatus of the present invention.

Here, M is the maximum distance for searching for a translation page candidate from a pair of translation pages. The maximum distance M to be searched is set in advance.

First, a search start point is input from the search start point input means 11 (step 11).

The bilingual page candidate generating means 12 extracts a page at a distance 1 from the search start point, pairs it with the page at the search start point, and then sets the pair as the bilingual page candidate at the distance 1
It is stored in the bilingual page candidate storage means 131 (step 1
2).

Next, the bilingual page determination means 14 determines whether or not the pair of bilingual page candidates having a distance of 1 stored in the distance 1 bilingual page candidate storing means 131 in step 12 is really a bilingual page (step). 13). The pair of the bilingual pages of distance 1 determined to be the bilingual pages in step 13 is stored in the distance 1 bilingual page storage unit 151 (step 14).

Next, it is determined whether or not a bilingual page having a distance of 1 exists in the bilingual page storing unit with distance 1. If a bilingual page having a distance of 1 exists, the process proceeds to step 16.
If there is no bilingual page with distance 1, the process proceeds to step 22 (step 15).

In step 16 where it is determined that a bilingual page with a distance of 1 exists, a variable m indicating the distance from the bilingual page is set to an initial value of 1 (step 16).

Next, the bilingual page candidate generating means 12 generates bilingual page candidates based on the pair of bilingual pages at distance 1 stored in the bilingual page storage means 151 at distance 1. First, a page that is a distance m away from one page of a pair and does not include the other pair of bilingual pages with a distance of 1 at the middle or end point is extracted from each pair. Next, the pages extracted from each pair of the bilingual pages having the distance 1 are combined to form a distance z (z = 2 × m +
The bilingual page candidate of 1) is generated and stored in the bilingual page candidate storage means of the distance z (step 17).

Next, the bilingual page determination means 14 determines whether or not the pair of bilingual page candidates of the distance z stored in the bilingual page candidate storage means of the distance z in step 17 is a bilingual translation (step 18). ). The pair of bilingual pages with the distance z determined to be the bilingual pages in step 18 is stored in the bilingual page storage means with the distance z (step 19).

Next, it is checked whether the variable m indicating the distance from the pair of the bilingual page is smaller than the maximum distance M to be searched. m
Is smaller than M, the process proceeds to step 21, and if m is M or more, the process proceeds to step 22 (step 20).

In step 21, after incrementing m by 1, the process proceeds to step 17 (step 21). In step 22,
The bilingual page output means 16 outputs the pair of bilingual pages stored in the bilingual page storage means 15, and ends the processing (step 22).

Next, the flow of the processing will be specifically described using the link example of the WWW data shown in FIG.

Here, it is assumed that the maximum distance M for searching for a bilingual page candidate from a pair of bilingual pages is 2. Therefore,
The bilingual page candidate storing means 13 has bilingual page candidate storing means of distance 1, distance 3 and distance 5, and the bilingual page storing means 15 has bilingual page storing means of distance 1, distance 3 and distance 5. I do.

First, the URL of the search start point is input. FIG.
Page A as the search start point, and the URL of page A (http: //
www.sample.aa / index / html) (Step 1)
1).

Next, "Page B, Page C, Page D" which is pages at a distance of 1 from page A at the search start point are extracted, and paired with page A at the search start point, respectively.
It is stored in the bilingual page candidate storage means 131. FIG. 4 shows the distance 1 stored in the bilingual page candidate storage unit 131.
The bilingual page candidates are shown (step 12).

Next, distance 1 bilingual page candidate storage means 13
It is determined whether “Page A and Page B, Page A and Page C, Page A and Page D” stored in No. 1 is a bilingual page. Here, the pair of “page A and page D” is determined to be a bilingual page, and a distance 1
It is stored in the bilingual page storage means 151 (step 13,
14).

In step 15, it is determined whether or not a bilingual page having a distance of 1 exists in the bilingual page storage unit with distance 1. Here, since there is a pair of “page A and page D”, the process proceeds to step 16 and the variable m is set to the initial value 1 (steps 15 and 16).

Next, a bilingual page “page A and page D” of distance 1 stored in the distance 1 bilingual page storage means 151
Based on the pair, a candidate for a translation page is generated. First,
A page "Page D" that is a page that is a distance m = 1 away from "Page A", and has a distance of 1 at the middle or end point
"Page B, Page C" that does not include "Page A" and a page that is a distance m = 1 away from "Page D" and does not include a pair "Page A" of a bilingual page with a distance of 1 on the way or at the end point E, page F ".

The “page B and page E, page B and page F, page C and page E, and page C and page F” combinations of the extracted pages are set as distance 3 bilingual page candidate storage means. 132.
FIG. 5 shows the distance 3 bilingual page candidates stored in the distance 3 bilingual page candidate storage means 132 (step 17).

Next, the distance 3 bilingual page candidate storage means 13
It is determined whether or not “Page B and Page E, Page B and Page F, Page C and Page E, Page C and Page F” stored in No. 2 are bilingual pages. Here, the pair of “page B and page E, page C and page F” is determined to be a bilingual page, and is stored as a bilingual page with a distance of 3 in the distance 3 bilingual page storage unit 151 (steps 18 and 19).

Next, the variable m = 1 and the maximum distance to be searched M =
Compare 2. Since M> m, the process proceeds to step 21 and after incrementing m by 1, the process proceeds to step 17 (steps 20 and 21).

Step 17 extracts a page at a distance m = 2 from each of the bilingual pages “Page A and Page D” at a distance 1, and the data in FIG. 2 has a page at a distance m = 2. Absent. Therefore, steps 18, 19
In step 20, m = 2 is compared with the maximum distance M = 2 to be searched. Here, since M> m, the process proceeds to step 22 (step 20).

In step 22, “page A and page D” stored in the distance 1 bilingual page storage means 151 and “page B” stored in the distance 3 bilingual page storage means 152
And page E, page C and page F "are output. FIG.
2 shows a bilingual page to be output (step 22).

(B) Configuration of the Second Embodiment FIG. 7 is a block diagram showing a bilingual information collecting apparatus according to a second embodiment of the present invention. In the second embodiment, the same reference numerals are assigned to blocks indicating the first embodiment,
In the second embodiment, only blocks different from those of the first embodiment will be described in detail.

In FIG. 7, the configuration different from that of the first embodiment is a bilingual data generating means 21, a bilingual data storing means 22, and a bilingual data outputting means 23.

The bilingual data generating means 21 uses the sentence correspondence result created when determining the bilingual page for the pair of bilingual pages determined to be the bilingual page by the bilingual page determining means 14, and associates the sentence in units of one sentence. To generate bilingual data.

If one sentence pair that has been associated is described in a different word in spite of the same language, or is described in a language different from other sentence pairs, it is not generated as bilingual data. As an example in which the paired one sentence is described in the same language, there is a case where a link between bilingual pages such as “English” and “Japanese” in page A and page D in FIG. 2 is shown. As an example in which one sentence pair is described in a language different from other sentence pairs, there is a case where “Japanese” on page D in FIG. 2 is “Japanese”.

In the case of the above example, the bilingual data generating means 21 converts bilingual data “English” and “Japanese”,
Do not generate bilingual data of "English" and "Japanese". As a method of generating the bilingual data, for example, a method described in Document 1 can be used.

The bilingual data storage unit 22 stores the bilingual data generated for the pair of bilingual pages determined by the bilingual data generating unit 21 to be a bilingual page. Similar to the bilingual page candidate accumulating means 13, the bilingual page candidate accumulating means 13 has a plurality of bilingual data accumulating means in accordance with a distance m for searching for a bilingual page candidate from a pair of bilingual pages. are doing.

Here, the distance 1 bilingual data storage means 22
1 and distance 3 bilingual data storage means 222 is provided. Distance 1
The bilingual data storage unit 221 and the distance 3 bilingual data storage unit 222 respectively store the bilingual page candidate storage unit 13.
1 and distance 3 correspond to the bilingual page candidate storage unit 132.

The bilingual data storage unit 221 stores bilingual data generated by the bilingual data generation unit 21 for a pair of bilingual pages determined as bilingual pages by the bilingual page determination unit 14 stored in the bilingual page storage unit 151. To accumulate.

The distance 3 bilingual data storage unit 222 translates a pair of pages determined as a bilingual page by the bilingual page determination unit 14 from the bilingual page candidates of the distance 3 stored in the distance 3 bilingual page candidate storage unit 132. The bilingual data generated by the data generating means 21 is stored.

The bilingual data output means 23 outputs the bilingual data stored in the bilingual data storage means 22. Here, the bilingual pages stored in the distance 1 bilingual data storage unit 221 and the distance 3 bilingual data storage unit 222 are output.

(B-1) Operation of the Second Embodiment FIG. 8 is a flowchart showing the operation of the second embodiment of the bilingual information collecting apparatus of the present invention. The same steps as those in the flowchart of FIG. 3 showing the operation of the first embodiment are given the same numbers, and in the second embodiment, the same steps as those in the bilingual data generation means 21 which are different from those of the first embodiment are performed in step 2.
3, 25, and 24, 26 relating to the bilingual data storage means 22
Will be described in detail.

In step 23, the bilingual data generating means 2
For a pair determined to be a bilingual page stored in the distance 1 bilingual page storage means 151, bilingual data is created using the sentence correspondence result created when the bilingual page is determined (step 23). ).

At step 24, the bilingual data of distance 1 generated at step 23 is stored in the distance 1 bilingual data storage means 2.
21. FIG. 9 shows distance 1 bilingual data storage means 22.
1 shows the bilingual data stored in No. 1. [English] and [Japanese] described on page A and page D in FIG. 2 are not translated data because they are described in the same language (step 24).

At step 25, bilingual data is generated for a pair of bilingual pages having a distance z determined to be a bilingual page at step 18 (step 25).

In step 26, the bilingual data of the distance z generated in the step 25 is stored in the bilingual data storage means 222 of the distance z (step 26).

Next, the flow of the processing will be specifically described using the link example of the WWW data shown in FIG.

Based on the operation shown in the first embodiment, in step 14, the distance 1 is set as the bilingual page of the distance 1.
What is stored in the bilingual page storage means 151 is “page A
And page D ".

Therefore, in step 23, bilingual data is generated for “page A and page D” stored in the distance 1 bilingual page storage means 151, and in step 24, the bilingual data generated in step 23 is converted to distance 1. It is stored in the bilingual data storage means 221 (steps 23 and 2).
4).

Further, based on the operation shown in the first embodiment, it is “page B and page E, page C and page F” that are determined to be the bilingual pages in step 18.

Therefore, in step 25, bilingual data is generated for “page B and page E, page C and page F” determined to be the bilingual page in step 18, and in step 26, the bilingual data generated in step 25 is generated. Is stored in the distance 3 bilingual data storage means 222 (step 2).
5, 26).

(C) Configuration of Third Embodiment FIG. 10 is a block diagram showing a third embodiment of the bilingual information collecting apparatus according to the present invention. In the third embodiment, the same reference numerals are given to the blocks indicating the first embodiment, and only the blocks different from the first embodiment in the third embodiment will be described in detail.

The third embodiment is suitable for the application of the first embodiment, such as finding a bilingual page from a plurality of search start points. Also, since a page having exactly the same content may have a different URL, it is possible to cope with such a case.

In FIG. 10, the configuration different from that of the first embodiment is a bilingual page determining means 31 and a non-bilingual page storage means 32.

The bilingual page determination means 31 converts the pair of bilingual page candidates stored in the bilingual page candidate storage means 13 into
As the first process of determining whether the page is a bilingual page, it is checked whether or not a page having the same name as the pair of the bilingual page candidate is stored in the non-bilingual page storage unit 32. If a page having the same name as the pair of the translation page candidates is stored in the non-translation page storage unit 32, it is immediately determined that the page is a non-translation page.

If a page having the same name as the pair of the translated page candidates is not stored in the non-translated page storage means 32, a hash value of the contents of the translated page candidates is obtained for each of the translated page candidates. The hash value is, for example, M which is frequently used on the Internet.
D5 or the like can be used. The obtained hash value of the bilingual page candidate is compared with the hash value of the non-bilingual page stored in the non-bilingual page storage unit 32 to determine whether the pair is the same page.

If the hash values are the same, it is checked whether or not the contents of the pair of untranslated pages and the pair of translated page candidates having the same hash value stored in the non-translated page storage means 32 are equal. In this case, the page is determined to be a pair of the same page, and is determined to be a non-translated page. As a method of determining whether the contents are the same, for example, there is a method of checking whether or not n characters from the head of two pages are the same character. n = 1
In the case of a page of 300 characters, the character comparison is performed 300 times to determine whether the pair has the same contents.

If the hash values and contents are different,
It is determined whether the page is a bilingual page in detail as in the first embodiment. After making the detailed determination, the translated page candidate determined to be a non-translated page is stored in the non-translated page storage unit 3.
Store in 2.

The non-translated page storage means 32 stores a pair of translated page candidates which the translated page determination means 31 has determined not to be a translated page, and includes a page name and a hash value.

(C-1) Operation of Third Embodiment FIG. 11 is a flowchart showing the operation of the third embodiment of the bilingual information collecting apparatus of the present invention. The same steps as those in the flowchart of FIG. 3 showing the operation of the first embodiment are denoted by the same reference numerals, and in the third embodiment, steps 31, 31 33 and steps 32 and 34 relating to the untranslated page storage means 32 will be described in detail.

FIG. 12 shows the processing (steps 31 and 33) of the bilingual page determination means 31. First, the bilingual page determination unit 31 determines whether a page having the same name as the pair of the bilingual page candidates stored in the bilingual page candidate storage unit 13 is stored in the non-bilingual page storage unit 32. If it is stored, the process proceeds to step 336; otherwise, the process proceeds to step 332 (step 3).
31).

In step 332, the hash value of the contents of the pair of the translation page candidates is calculated (step 332).

Next, it is determined whether or not a pair of pages having a hash value equal to the hash value calculated in step 332 exists in the non-translated page storage means 32. If so, the process proceeds to step 334; Goes to step 335 (step 333).

In step 334, it is determined whether or not the contents of the pair of pages determined to have the same hash value in step 333 are equal to the contents of the pair of bilingual page candidates (step 334). If the contents are the same, the process proceeds to step 336; if the contents are different, the process proceeds to step 335 (step 334).

In step 335, it is checked whether the pair of the translation page candidates is a translation page in detail, as in the first embodiment (step 335). Step 336
Determines that the pair of bilingual page candidates is not a bilingual page (step 336).

In steps 32 and 34, step 31,
At 33, a pair of the page name and the hash value of the pair of the translated page candidates determined to be not the translated page is stored in the non-translated page storage means 32 (steps 32 and 34).

Next, the flow of the processing will be specifically described using the link example of the WWW data shown in FIG. Page B, page C, page E, and page F shown in FIG. 2 have different names, respectively, page K, page O, page H, and page G.
Is assumed. Also, assume that pages having the same content have the same hash value, and pages that do not have the same content have different hash values.

According to the operation of the first embodiment, in step 12, the pair of the bilingual page candidates which are the bilingual page candidates with the distance 1 are “page A and page B, page A and page C, page A and page C” shown in FIG. A and page D ".

In step 331 of FIG. 12, first, the same as the pair of bilingual page candidates “Page A and Page B, Page A and Page C, Page A and Page D” stored in the distance 1 bilingual page candidate storing means 131 It is determined whether the page with the name is stored in the non-translated page storage unit 32.

FIG. 13 shows the non-translated page storage means 32. Since there is no parallel translation page candidate pair having the same name, the process proceeds to step 332, where step 332 obtains hash values of the translation page candidate pairs “Page A and Page B, Page A and Page C, and Page A and Page D” respectively ( Steps 331, 332).

Next, it is determined whether or not a pair of non-translated pages having a hash value equal to the obtained hash value exists in the non-translated page storage means 32. Here, since there is no matching page, the process proceeds to step 335 to check in detail whether the page is a bilingual page, and determines that “page A and page D” are bilingual pages. (Step 33
3, 335).

In step 32, "page A and page B, page A and page C", which are determined to be non-translated pages in step 31, are stored in the non-translated page storage means 32 (step 32).

Further, based on the operation of the first embodiment,
In Step 17, the distance 3 bilingual page candidates stored in the distance 3 bilingual page candidate storage means 132 are “page B and page E, page B and page F, and page C”.
And page E, page C and page F ".

Therefore, in step 331, the distance 3
“Page B” stored in the bilingual page candidate storage unit 132
And page E, page B and page F, page C and page E, and page C and page F ”are stored in the non-translated page storage means 32. Normally, the processing is performed on all the distance 3 bilingual page candidates. Here, the processing of “page B and page F” will be described.

First, in step 331, since the non-translated page storage means 32 does not have a pair of translated page candidates having the same name, the process proceeds to step 332. Find the value for each. Here, the hash value of page B is “3E
4F57 ”and the hash value of page F is“ 2E1F51 ”
(Steps 331 and 332).

Next, it is determined whether or not a non-translated page pair having a hash value equal to the obtained hash value exists in the non-translated page storage unit 32. Here, FIG.
Since the hash values of “Page K and Page G” shown in FIG.
Proceed to step 334 (step 333).

In step 334, it is checked whether or not the contents of “Page K and Page G” whose hash values match each other match the contents of “Page B and Page F”. in this case,
It is determined whether the contents of “page K and page B” and the contents of “page G and page F” match (step 334). Here, since the page contents match each other, the process proceeds to step 336, where it is determined that “page B and page F” are not bilingual pages (step 336).

When the processing of steps 331 to 336 is performed for all the distance 3 bilingual page candidates, the non-translated pages are determined to be “page B and page F, page C and page E”, and “ "Page B and page E, and page C and page F" are determined to be bilingual pages.

In step 34, "page B and page F, page C and page E" determined in the translation page in step 31 are stored in the non-translation page storage means 32 (step 34).

(D) Other Embodiments (D-1) In the third embodiment, when storing a pair of non-translated pages, the non-translated page storage means 32 may store only the page name. FIG. 14 shows an untranslated page storage means 32 consisting only of page names. In this case, the bilingual page determination means 31 does not need to obtain the hash value of the bilingual page candidate, and performs the following processing. First,
The bilingual page determination means 31 is a non-translation page storage means 32
Then, it is checked whether or not a page having the same name as the pair of translation page candidates is stored. If a page with the same name is stored, it is immediately determined to be a non-translation page. If it is not stored in the non-translated page storage means 32, it is determined in detail as in the first embodiment whether the page is a translated page. After making the detailed determination, the bilingual page candidate determined to be a non-translated page is stored in the non-translated page storage unit 32.

(D-2) Bilingual page candidate storage means 13,
As a method of storing a pair of pages in the bilingual page accumulating unit 15, the bilingual data accumulating unit 22, and the non-translated page accumulating unit 32, a name such as a URL that can uniquely specify the page of each pair may be used.

(D-3) Bilingual page determination means 14, 31
The determination of whether the page is a bilingual page is not limited to the technique of Document 1. For example, the tag number of two pages is counted, and when the count numbers match, it is determined to be a bilingual page. Also, various methods can be applied, such as judging a translation page based on the appearance order of tags. When the sentence correspondence is not created in the bilingual page determination means 14 and 31 as described above, there is no need for a storage means for storing the sentence correspondence result.

(D-4) The bilingual data generation means 21 is a means for generating necessary bilingual data, and is not limited to generating bilingual data in sentence units. For example, only words in a table may be generated as bilingual data.

When the translated page determination means 14 and 31 correspond to the sentence in which the tag is inserted, the translated data with the tag removed is output. For example, "This is <Italic> a book </ Italis>" and "This is <Ita
lic> book </ Italis>"is output as bilingual data of" This is a book "and" This is a book "with the tag removed.

Further, in the case where the sentence correspondence result is not created in the bilingual page determination means 14 and 31 and bilingual data in sentence units is required, the bilingual data generation means 21 performs correspondence in sentence units. May be.

(D-5) As described in (D-3) and (D-4) above, the bilingual page determination means 14, 31 and the bilingual data generation means 21 can apply various methods. It is possible to configure by appropriately combining both means.

[0118]

As described above, according to the first embodiment of the present invention, a pair of pages serving as a bilingual page is directly linked to each other, or a path of a link connecting the pair is used. By taking advantage of the fact that there is often a pair of parallel translation pages with a direct link at the center, it is determined whether pages whose bilingual pages are separated by more than 1 are bilingual page pairs. It is possible to appropriately narrow the search range for checking whether or not.

In the second embodiment, when bilingual data is generated, the bilingual page determination means 14 determines whether the bilingual page candidate pair is a bilingual page or not. By using the correspondence result, the generation efficiency is improved as compared with the case where the bilingual data is collectively generated based on the bilingual pages output by the bilingual page output unit 16.

In the third embodiment, a pair of non-translated pages once determined to be non-translated pages is stored in the non-translated page storage means 32, so that bilingual pages are collected from a plurality of search start points. In such a case, it is not necessary to determine again whether the translation page candidate is a translation page, and the processing speed can be improved.

Further, the untranslated page storage means 32 stores the name of the untranslated page and the hash value indicating the content of the untranslated page, and calculates the hash value calculated for the pair of translated pages and the untranslated page storage means 32. By comparing the hash values stored in the "..", even if a page having the same content is represented by a plurality of names, it is not necessary to determine again whether the page is a bilingual page in detail.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of a bilingual information collecting apparatus according to the present invention.

FIG. 2 is a diagram showing a link example of WWW data.

FIG. 3 is a flowchart showing an operation of the first embodiment of the bilingual information collecting apparatus of the present invention.

FIG. 4 is a diagram illustrating an example of a distance 1 bilingual page candidate storage unit.

FIG. 5 is a diagram illustrating an example of a distance 3 bilingual page candidate storage unit.

FIG. 6 is a diagram showing an output example of a bilingual page.

FIG. 7 is a block diagram showing a second embodiment of the bilingual information collecting apparatus of the present invention.

FIG. 8 is a flowchart showing the operation of the second embodiment of the bilingual information collecting apparatus of the present invention.

FIG. 9 is a diagram showing an output example of bilingual data.

FIG. 10 is a block diagram showing a third embodiment of the bilingual information collecting apparatus of the present invention.

FIG. 11 is a flowchart showing an operation of the third embodiment of the bilingual information collecting apparatus of the present invention.

FIG. 12 is a flowchart showing operations of steps 31 and 33 of FIG. 11;

FIG. 13 is a diagram (1) illustrating an example of an untranslated page storage unit;

FIG. 14 is a diagram (2) illustrating an example of an untranslated page storage unit;

[Explanation of symbols]

11 ... search start point input means, 12 ... bilingual page candidate generation means, 13 ... bilingual page candidate storage means, 131 ...
Distance 1 bilingual page candidate storing means, 132... Distance 3 bilingual page candidate storing means, 14, 31... Bilingual page determining means, 15... Bilingual page storing means, 151... Distance 1 bilingual page storing means, 152. Distance 3 bilingual page storage means, 16... About page output means, 21... Bilingual data generating means, 22... Bilingual data storing means, 221... Distance 1 bilingual data storing means, 222. Means, 23..Translation data output means, 32..Untranslated page storage means.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Atsushi Ikeno 1-7-112 Toranomon, Minato-ku, Tokyo Oki Electric Industry Co., Ltd. (72) Inventor Sasara 1-7-112 Toranomon, Minato-ku, Tokyo F term (reference) in Electric Industries, Ltd. 5B075 KK07 ND20 NK02 NK44 PP02 PP03 PQ02 PQ42 QS20 UU06 5B082 EA00

Claims (6)

[Claims] 1. A document having two pages that are not linked by a direct link but are linked indirectly by a link and that have a pair of bilingual page candidates existing at the center of the path of the link connecting the two pages. A central bilingual page determining means for determining whether the page is a bilingual page described in a different language, and extracting a page at a distance m from each of the pair of bilingual pages determined by the central bilingual page determining means to be a bilingual page A bilingual page candidate generating unit that generates a pair of bilingual page candidates by combining the extracted pages; and a bilingual page that determines whether the bilingual page candidate pair generated by the bilingual page candidate generating unit is a bilingual page. Determining means; A bilingual page output means for outputting a bilingual information. 2. The bilingual information collection device according to claim 1, wherein the central bilingual page determination means replaces the bilingual page candidate generating means with a distance m from each of the pair of bilingual pages determined to be a bilingual page. Bilingual information collection characterized by using a bilingual page candidate generating means for extracting a page which is separated and does not include the bilingual page in the middle or at the end point and generates a pair of bilingual page candidates by combining all the extracted pages. apparatus. 3. A bilingual information collecting apparatus according to claim 1, wherein a bibliographical page describing the same document in different languages is collected. A first method of generating a pair of a page and a page at a distance n from the search start point as a pair of bilingual page candidates
A bilingual page candidate generating unit for determining whether the pair of bilingual page candidates generated by the first bilingual page candidate generating unit is a bilingual page instead of the central bilingual page determining unit. A bilingual information collecting apparatus, wherein a bilingual page determining means is used. 4. The bilingual information collecting device according to claim 1, wherein the central bilingual page determining means or the bilingual page determining means determines a bilingual page based on a pair of bilingual pages. A bilingual information collecting apparatus, comprising: a bilingual data generating unit that generates data; and a bilingual data output unit that outputs the bilingual data generated by the bilingual data generating unit. 5. The bilingual information collecting apparatus according to claim 1, further comprising: a non-translated page storing means for storing a pair of bilingual page candidates determined to be not a bilingual page as a pair of non-translated pages. The central bilingual page determining means or the bilingual page determining means, when the pair of bilingual page candidates matches the pair of non-bilingual pages stored in the non-bilingual page storage means, unpairs the bilingual page candidate pair. A bilingual information collection device that determines that the page is a page. 6. The bilingual information collecting apparatus according to claim 1, wherein a pair of a bilingual page candidate determined not to be a bilingual page is set as a non-bilingual page pair, and a page name and a page content of the non-bilingual page pair. The central bilingual page determining means or the bilingual page deciding means further comprises a non-translated page storing means for storing a value indicating the page name of the pair of bilingual page candidates stored in the non-translated page storing means. Judge whether it matches the page name of the bilingual page pair.If they match, determine the bilingual page candidate pair as a non-bilingual page.If they do not match, enter a value indicating the page content of the bilingual page candidate pair. It is determined again whether the value matches the value indicating the page content of the pair of non-translated pages stored in the non-translated page storage means. Parallel translation information collection apparatus characterized by a pair of as non-translation page determination.