JPH11328218A - Contents attribute information normalization method, information collecting/service providing system, attribute information setting device and program storage recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To construct a data base for providing a service without being restricted by the structure/form of a document for the perusal of an information provider by performing normalization processing of an attribute structure as of contents attribute information and performing normalization of a character expression form and normalization processing of numerical expression. SOLUTION: An automatic information collecting part 101 of an automatic information collecting and classifying device 100 patrols a Web site 120 of an information provider on a network 110, collects document files and extracts information. An attribute extraction part 102 normalizes a character code and then extracts only the contents attribute information as of the collected and extracted documents. An attribute normalization part 103 refers to a normalization rule 106 and normalizes the contents attribute information provided with the structure/form coincidently with a perusal document extracted by the attribute extraction part 102 to the form suited to a retrieval service or the like. Further, as of the contents attribute information whose the structure is normalized, a character expression form is normalized and numerical expression is normalized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology for automatically collecting content information distributed on a network and reusing the information for a search service and the like. More specifically, the present invention relates to integrating and organizing content information collected from a plurality of Web sites. A method for extracting and normalizing attributes required for a search service, an information collection and service providing system such as a search engine to which the method is applied, an attribute information setting device for supporting creation and editing of a document file, and And a recording medium storing a program for normalizing content attribute information.

[0002]

2. Description of the Related Art In order to automatically collect content information distributed on a network and reuse it for a search service or the like,
The following are a few common prior arts.

FIG. 24 shows a search engine system that does not consider content attribute information. In the figure, 2400 is a search engine, 2410 is a network, 2420 is a web site distributed and arranged on the network, 243
0 is a user terminal. The web site 2420 holds a browsing document file (generally described in HTML) prepared by an information provider (IP). Search engine 2
400 is the control of the control unit 2403, and the automatic information collection unit 2
Web site 24 on network 2410 at 401
The document file for browsing (HTML document) is collected by circulating through the document 20, and the analysis unit 2402 analyzes the document file, creates summary information, a keyword index, and the like in page units, and stores the information in page units. The content is stored in a content database (content DB) 2404. On the other hand, when there is a search request from the user terminal 2430, the search unit 2405 searches the content DB 2404 and returns the search result to the user terminal 2430. The user of the user terminal 2430 can use the Web site 24 if necessary based on the search result.
Browse 20 document files directly.

FIG. 25 shows a structure of a document file (HTML document) which is a target of the related art. FIG. 26 shows an example of a product catalog described in the HTML language as a specific example. As shown in FIG. 25A and FIG. 26, a target document file (HTML document) stores a plurality of contents, the delimitation is unclear, and the field of the content is unclear. The attribute information of the content is included in the text. The analysis unit 2402 extracts the necessary information by performing a natural language analysis or the like on the document file.
4, each page is miscellaneously stored. FIG.
As shown in (b), a document can be represented by a tree structure, but a document for browsing (HTML document) is structured based on a browsing style or a logical structure of a sentence. It is difficult to extract attribute information suitable for the search service) therefrom.

FIG. 27 shows an automatic collection and classification system in which content attribute information is fixed. In the figure, 2700 is an automatic information collection and classification device, 2710 is a network, 27
Reference numeral 20 denotes a Web site distributed and arranged on the network, and 2730 denotes a user terminal. In this conventional example, We
In b site 2720, content attribute information is prepared separately from the document body (HTML document). FIG. 28 shows a configuration example of a document targeted by the conventional technology. Note that it is also possible to mark up a character string in the browsing document as content attribute information with a tag and include the content attribute information in the browsing document.

[0006] In the automatic information collection and classification device 2700, under the control of the control unit 2704, the automatic information collection unit 2701 circulates the Web site 2720 on the network 2710 and collects corresponding files (document files, attribute information files). The content attribute information is separated by the separation unit 2702, the content attribute information is interpreted by the attribute extraction unit 2703, and the content
B2705. The operation of the service providing unit 2706 is the same as that of the search unit 2405 in FIG. In this conventional example, since the content attribute information is prepared separately from the browsing information, a structure convenient for providing the service can be provided.

FIG. 29 shows an automatic collection and classification system that associates content attribute information with a viewing tag and a character string. In the figure, 2900 is an automatic information collection and classification device, 2
910 is a network, 2920 is a Web site distributed and arranged on the network, and 2930 is a user terminal. In this conventional example, a document file (HTML document) containing only browsing information is prepared on the Web site 2920 as in the search engine system of FIG. 24. In this case, the browsing tag and the correspondence rule 2904 between the character string and the content attribute are stored, and the content attribute can be extracted from the browsed document by referring to the tag. As the correspondence rule 2904, for example, a rule that “price” always appears before the character “yen” is created.

[0008] In the automatic information collection and classification device 2900, under the control of the control unit 2903, the automatic information collection unit 2901 traverses a Web site 2920 on the network 2910 to collect a browsing document file (HTML document), and The extraction unit 2902 determines a character string in the document file and a rule 2904 corresponding to a tag and an attribute indicating a structure for browsing.
, The content attribute information is extracted from the document file and stored in the content DB 2905. The operation of the service providing unit 2906 is basically the same as that of the search unit 2405 in FIG. In this conventional example, attribute information is stored in the content DB 2905 in units of content (for example, in units of products).
, A content field designation search, an attribute search, and an association search can be performed.

Next, the attribute information setting device will be described. The attribute information setting device is a device that sets content attribute information by marking up a character string in text information. A general structured document creation device (for example, S
GML editor) has an equivalent function.

FIG. 30 is a block diagram of a conventional attribute information setting device. The entire menu unit 3001, attribute setting unit 3002, attribute deleting unit 3003, attribute range changing unit 3
005, file input unit 3006, file output unit 30
07, a structure verification unit 3008 and the like. A target document file is input from the file input unit 3006, and the attribute setting unit 3002, the attribute deletion unit 3003, and the attribute change unit 30 are viewed while viewing the editor screen of the overall menu unit 3001.
04, set, delete, change, and change the range of the content attribute information using the functions of the attribute range changing unit 3005 and the like, and output from the file output unit 3007. The document object is managed without distinguishing between the browsing tag and the content attribute tag, and is verified by the structure verification unit 3008.

FIG. 31 shows a document with an attribute tag (generally, X
FIG. 3 is a conceptual diagram of a browsing document structure of an (ML document) and content attribute information. In the figure, white circles indicate browsing tags, and black circles indicate content attribute tags. Conventional attribute information setting device 3
Since the structure verification unit 3008 of the 000 manages the browsing document and the content attribute information in the same manner, for example, when only the parent-child relationship can be verified, there is no grammatical restriction between the white circles and the black circles, and Even if there are restrictions between the black circles, verification may not be possible in some cases. Therefore, the restrictions are loosened and effective grammar verification cannot be performed.

[0012]

In the above prior art, a search engine system that does not consider content attribute information has the following problems. (1) The boundaries of the contents (for example, product information) in a document are unknown, and can be classified in page units and searched only in page units. (2) The field of content cannot be specified, and the search results are noisy. For example, if you think "I want to buy sake" and search with the keyword "Sake", you will find not only mail order of sake, but also unchiku of people who like sake. (3) Content attributes (for example, product price, color, etc.) cannot be recognized. Therefore, search by attribute is not possible. For example, it is impossible to search for “sake of 3000 yen or less”. Also, it is difficult to reuse information.
That is, it cannot be associated with another DB using the attribute.

On the other hand, in an automatic collection and classification system in which content attribute information is fixed, content attribute information is prepared in a document or separately from the document, and the system can acquire attribute information. Possible, search by specifying the field of content is possible, search by attribute of content is possible (eg, search for "sake of 3000 yen or less" is possible, and association with other data using attribute of content is possible This almost eliminates the problem of a search engine system that does not consider content attribute information.

However, in the automatic collection and classification system in which the content attribute information is fixed, the structure and expression format of the document to be browsed and the structure and expression of the attribute information suitable for providing the service.
Since the formats generally do not match, the following new problems arise. (1) When content attribute information is prepared separately from a browsing document (information called so-called metadata), there is a possibility that information obtained by directly browsing a document file does not match information obtained by automatic collection. In particular, in the case of a document file in which a plurality of contents are described (such as a product catalog), the information is described twice, and the possibility is high. (2) When a character string in a browsing document is marked up with a tag as content attribute information (this is a general use of XML), restrict the document structure and description format in the document file, or conversely provide a service Must be the same as the document structure and description format in the document file. In particular, if the XML is used as it is, the method of assigning the attribute to the same content, the name of the attribute, the description format, and the like are different depending on the creator, and the attribute suitable for application to other services is assigned. There is no guarantee. That is, documents collected from a plurality of computers (sites) cannot be stored in a common structure format (such as a DB table) in consideration of provision of other services.

[0015] On the other hand, the content attribute information is read by a viewing tag,
In the system for associating with a character string, only the browsing information is stored in the browsing document file, and the browsing tag and the correspondence rule between the character string and the content attribute in the browsing document file are held in advance by the system and referred to To extract the content attribute from the browsed document. Therefore, under the condition that the corresponding rule is correct, the problem of the search engine system that does not consider the content attribute information can be solved. Further, the problem (1) of the automatic collection and classification system in which the content attribute information is fixed does not exist. However, a restriction is required that the browsing document must obey the corresponding rules, and a strong restriction is imposed on the structure of the browsing document. In the case of a product catalog or the like, the freedom of expression for consumers is limited, which is a major problem. This is a problem equivalent to the above (2) of the automatic collection and classification system in which the content attribute information is fixed.

Next, the conventional attribute information setting device has the following problems. (1) General users can view the results when browsing, but they do not have the function to view the results of the extraction and normalization (structure, value format) processing of attributes by automatic collection / classification devices. However, data creators cannot know how the data will be used. (2) Since the browsing tag and the content attribute tag are not managed separately, the structure verification of the tag becomes complicated. As shown in FIG. 31, generally, the restrictions between the browsing tags and the content attribute tags are relatively strong, but there are often loose restrictions between the browsing tags and the content attribute tags (for example, XML). Degree of limitation as aptitude form in language). In such a case, since the respective components are not managed separately, the structure verification unit that can verify only the parent-child relationship cannot help functioning effectively, as a strict restriction as a whole.

An object of the present invention is to solve the above-mentioned problems of the prior art and to realize a service that allows content information distributed on a network to be freely reusable other than merely viewed.

More specifically, an object of the present invention is to automatically set the information format at the time of providing a service and the structure format of a browsing document prepared by an information provider, and to automatically collect content information distributed on a network. Another object of the present invention is to make it possible to construct a service providing database without being restricted by the structure and format of the information provider's browsing document.

Another object of the present invention is to mark up a character string in text information so that a data creator can check the use result of data in an apparatus for setting content attribute information. An object of the present invention is to enable strict grammar verification of the structure and the content attribute information structure.

[0020]

In order to achieve the above object, according to the first aspect of the present invention, content attribute information having a structure and format adapted to a browsed document is converted into a structure and format independent of the browsed document. Extracting content attribute information from a document file containing content attribute information mixed with browsing information; and performing an attribute structure normalization process on the extracted content attribute information. And performing a normalization process of a character expression format and a numerical expression process on the content attribute information whose structure has been normalized.

According to a second aspect of the present invention, in the content attribute information normalizing method according to the first aspect, in the attribute structure normalizing process, the content development, attribute name normalization, attribute division, and other attribute It is characterized by performing normalization to

Further, the invention of claim 3 provides the above-mentioned claim 1,
2. In the content attribute information normalization method described in 2, the normalization rules of the normalization processing include field-independent / attribute-independent rules, field-dependent / attribute-independent rules, field-independent / attribute-dependent rules, and field-dependent / attribute-dependent. It has rules and is managed by content fields and attribute names.

According to a fourth aspect of the present invention, there is provided an information collection and service providing system in which an information provider's Web site, a host device, and a user terminal are connected via a network.
The host device is a Web device distributed on a network.
Means for automatically collecting the document files of the site, means for extracting the content attribute information contained in the collected document files in a mixture with the browsing information, the extracted content attribute information having a structure suitable for service provision. Means for normalizing to a format, means for storing the normalized content attribute information, and means for providing a service in response to a request from a user terminal using the stored content attribute information. And

According to a fifth aspect of the present invention, there is provided an attribute information setting device for setting content attribute information by marking up a character string in a document for browsing, extracting and normalizing the content attribute information and previewing the extracted attribute information. There is provided a means for separately managing the browsing tag and the content attribute tag, and ignoring the browsing information and the content attribute information for verification.

According to a sixth aspect of the present invention, there is provided a computer-readable recording medium storing a program for normalizing content attribute information having a structure and format adapted to a browsed document into a structure and format independent of the browsed document. A processing process of extracting content attribute information from a document file containing content attribute information mixed with browsing information; a process of performing an attribute structure normalization process on the extracted content attribute information; The content attribute information having the normalized structure includes a processing process for normalizing a character expression format and normalizing a numerical expression.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an information collection and service providing system that automatically collects content information distributed on a network, extracts and normalizes attributes, and reuses them for a search service and the like as an embodiment of a system to which the present invention is applied. It is shown. In the figure, 10
Reference numeral 0 denotes a host device serving as a search engine or the like (herein, referred to as an automatic information collection / classification device); 110, a network such as the Internet; 120, a Web site of an information provider (IP) distributed on the network; It is a user terminal that uses the automatic information collection and classification device 100.

The Web site 120 is a browsing document file (for example, an XML document) containing content attribute information.
Is provided with an attribute information setting device 125 that creates the attribute information. With the attribute information setting device 125, it is possible to include content attribute information in a form for holding the content attribute information in a browsing document (generally, an HTML document) having various structures. The configuration of the attribute information setting device 125 according to the present invention will be described later.

The automatic information collection / classification apparatus 100 traverses a Web site 120 on the network 110 and automatically collects a document file (for example, an XML document) including content attribute information. An attribute extracting unit 102 for extracting content attribute information from a file, and an attribute normalizing unit for normalizing the extracted content attribute information such as a structure conversion and an attribute name / attribute value format conversion for reuse of a search service or the like. 103, a control unit 104 that controls these units, a content database (content DB) 105 that stores the content attribute information normalized by the attribute normalization unit 103 in content units such as product units, and the like. A normalization rule 106 for managing rules for normalization processing by content fields, attribute names, and the like; Using the contents of the content DB 105, it consists of such user service providing unit 108 for providing such search service by the content attribute information to the user. Note that a broken line corresponding rule 107 indicates an option, and, as in the case of the corresponding rule 2904 in FIG. 29, by holding a corresponding rule between a browsing tag and a character string in a viewed document file and a content attribute, attribute extraction is performed. Part 102
Indicates that the content attribute can be extracted from a document file containing only browsing information (generally, an HTML document).

The user accesses the automatic information collection / classification apparatus 100 using the user terminal 130, performs a field designation search, an association search, and the like, and directly browses the document file on the Web site 120 based on the search result.

FIGS. 2 and 3 show the entire processing flow of the automatic information collection / classification apparatus 100, and FIG. 4 shows the structure of the target document and the transition in the processing. An embodiment of the content attribute information extraction / normalization process according to the present invention will be described below with reference to FIGS. The program describing each process of the flowcharts shown in FIGS. 2 and 3 can be provided by being recorded on a computer-readable recording medium, for example, a floppy disk, a memory card, a CD-ROM, or the like. .

The target document in this embodiment is an XML document. XML is a document structure description (markup) language in which tags can be freely defined, and content attributes can be automatically extracted by using the content attribute tags defined in the XML. It is also possible to include attributes that are not used when browsing.

In FIG. 1, an information provider (IP) uses an attribute information setting device 125 to mark up a character string in a browsing document (generally described in HTML) with a tag, thereby obtaining a browsing document. A document file (XML document) including the content attribute information having the structure and format according to is created and prepared on the Web site 120. FIG. 4A shows the structure of an HTML document, and FIG. 4B shows the structure of an XML document. Note that the document file of the Web site 120 may be described in the XML format from the beginning.

The automatic information collection unit 101 of the automatic information collection / classification device 100
A document file (XML document) is automatically collected by circulating through the file 20, and information in IP units and information in page units are extracted from the document file (steps 201 to 203). The information extracted here is an XML document as it is, which includes content attribute information having a structure and format adapted to the browsing document mixed with the browsing information (FIG. 4C).

The attribute extracting unit 102 is provided with the automatic information collecting unit 10
After normalizing the character codes of the XML documents collected and extracted in step 1 (step 204), only the content attribute information is extracted (step 205). Here, the normalization of the character codes is to cope with the fact that document files on the network are often described with various character codes. Extraction of only content attribute information from a document file is performed, for example, by parsing a tag (SG
This can be achieved by operating an ML parser (eg, an ML parser) to interpret only the content attribute information tag (ignoring the browsing tag). Which tag is the tag for the content attribute information is stored as a setting file and is referred to. For example, it is held as a default attribute of DTD in SGML. In this case, it can be reused as a setting file for document creation / editing. The content attribute information extracted here has a structure and
Has a format. FIG. 4D shows this.

Note that the attribute extraction unit 102 sets a rule 10 for browsing tags and corresponding rules between character strings and content attributes.
8 (for example, be sure to precede the "price"
Is stored, the content attribute information can be extracted from the browsed document by referring to the rule 108.

The attribute normalization unit 103 performs the normalization rule 10
Referring to FIG. 4, the content attribute information having the structure and format corresponding to the browsed document extracted by the attribute extraction unit 102 is normalized into a format suitable for providing a service such as a search service. The normalization rules 104 include a field-independent / attribute-independent rule, a field-dependent / attribute-independent rule, a field-independent / attribute-dependent rule, a field-dependent / attribute-dependent rule, and the like, and are managed by content fields and attribute names. doing.

In the attribute normalizing section 103, first, the target content (content attribute information having the same form as the structure of the browsed document)
Is recognized by the value of the category tag of the content attribute information (step 206). This is because the schema of the content DB 105 differs depending on the field, and thus it is necessary to first recognize the schema. Next, with respect to the target content (content attribute information), the structure according to the browsed document is normalized into a structure suitable for providing a service (step 207). Further, the content attribute information having the normalized structure is represented by a character expression. Format normalization (step 208) and numerical expression normalization (step 209) are performed. FIG. 4E shows the concept of normalization. Finally, the normalized content attribute information is stored in the content DB 1
05 is stored for each content (step 210).
Note that the normalization of the character expression form and the normalization of the numerical expression may be performed in the reverse order. In some cases, the attribute structure may be converted, for example, by dividing into a numerical part and a unit part.

The normalization of the attribute structure, the normalization of the character expression form, and the processing of the numerical expression normalization will be described in detail below.

The normalization of the attribute structure can be broadly classified into content expansion processing, normalization of attribute names, division of attributes, and conversion to other attributes. Further, the content development process is divided into two. Here, the content development processing (1),
This will be referred to as content development processing (2).

In order to represent a variant of one content as browsing information, the content may have a structure having a plurality of same attributes, but it may be more convenient to store the content as another content when providing a service. In such a case, the content development processing (1) is applied to develop as another content. For example, if a beverage product has the same contents and the price changes depending on the capacity, the browsed document (product catalog) is expressed as a variant of the same product, and the content attribute information expressed by reuse is also shown. It has a similar shape. In many cases, they are treated as individual products when the service is provided, and in that case, the function of expanding them is indispensable. FIG. 5 shows a conceptual diagram of the content development processing (1). Note that the example of the content expansion processing in FIG. 4E corresponds to this content expansion processing.

On the other hand, contents of the same genre are often described as browsing information at one position. In this case, the content development processing (2) is applied, and one round is considered as an object, and the attribute information set for the object is developed as the default attribute of the child object. For example, if a series of beverage products all belong to a product called Ginjo, the catalog may widely describe it as Ginjo and implicitly write that all subsequent products are Ginjo. At the time of providing a service, it is necessary to add that the manufacturing method is Ginjo to the attribute of each product information in order to handle each product. FIG. 6 shows a conceptual diagram of the content development processing (2).

The normalization of attribute names is performed when the item names of conceptually identical attributes are different. FIG. 7 shows an example of attribute name normalization. Conversely, attribute division is performed when it is easier to handle conceptually by dividing into a plurality of attributes. FIG. 8 shows an example of attribute division.

The conversion to another attribute (normalization) is a process of converting a single or multiple attributes to another single or multiple attributes. For example, the drinking mouth is calculated from the degree of sake and the degree of acidity, and the drinking mouth is determined according to which region in FIG. 9 is included (calculated by an inequality).

The normalization of the character expression form is performed, for example, by classifying the sake brewing method into three types, Daiginjo, Ginjo and ordinary, and providing a service. , "Daiginjo liquor" is also to be normalized with Daiginjo (if only the expression "Daiginjo" is accepted,
Product catalogs for general customers will be limited.) In addition, it normalizes the coding system.
In this case, a conditional statement such as a logical expression can be used.

The normalization of the numerical format is performed, for example, according to the processing procedure shown in FIG. In the read document, the numerical value is expressed in full-width / half-width,
Since there are variations in expressions, such as normal numerical values / kanji numerals and comma-separated / none, these are normalized (step 1001). In a numerical value with a unit, there is a variation in a way of expressing the unit. For example, a price expression of $ 10
There are 00, 1000 yen, 1000 yen, and the like. These are recognized (step 1002). The recognized unit system is converted to a unit system used for service provision (step 1003). Also, a plurality of values or numerical values having a range are recognized and converted (step 1004). For example,
Based on information such as "1000 yen or more and less than 2000 yen, and 3000 yen or more and less than 4000 yen", it is converted into a format suitable for service provision. For service provision, if two numerical ranges can be handled by one content, they are stored as they are,
If there is only one numerical range, it is developed and stored in two contents.

FIGS. 11 to 19 show specific processing examples of content attribute information extraction and normalization according to the present embodiment. FIG. 11 shows a document file (XML document) provided by an information provider, in which content attribute information is set by marking up a document sequence in a browsing document (HTML document) with an arbitrarily defined tag. I have. In FIG. 11, tags shown in bold type (for example, <MEMO>... </ MEM
O>) is the content attribute information.

FIG. 12 shows the document file (XML) shown in FIG.
This is data obtained by extracting the content attribute information from the document as it is, and the content attribute information remains in the structure and format according to the document viewed by the information provider. FIG. 13 shows the content attribute information of FIG. 12 expressed in a tree structure format. As can be seen from FIG. 13, this is not necessarily a structure / form suitable for providing a search service or the like.

FIG. 14 is a content development process (2) for the target content (content attribute information) in FIG.
FIG. FIG.
Content development processing (1)
Is applied, and as a result, the target content in FIG. 12 is expanded into four contents.
FIG. 16 shows the result of the content development process of FIG.
It shows the result of applying normalization in character expression format and normalization in numeric format.

When the normalized content attribute information shown in FIG. 16 is expressed in a tree structure format, it becomes as shown in FIGS. 17 and 18. At a glance, it can be seen that the structure and format are more suitable for providing a search service and the like than in FIG.

FIG. 19 shows the normalized content attribute information of FIG. 16 in a table format. The content DB 105 of FIG. 1 stores the content attribute information in such a format. In this way, the browsing document (HTML
It is possible to construct a database (related database) for providing a search service without being restricted by the structure and format of the document).

Next, an attribute information setting device according to the present invention will be described. FIG. 20 is a block diagram showing an embodiment of the attribute information setting device according to the present invention. The feature of the attribute information setting device 2000 is that an attribute extraction / normalization preview unit 2008 and a structure verification unit 2009 for separately managing and verifying a plurality of tag sets are provided in the conventional device shown in FIG. The attribute extraction / normalization preview unit 2008
Attribute extraction unit 102 in the information automatic collection and classification device 100,
It has basically the same function (equivalent or a simplified version thereof) as that of the attribute normalization unit 103 and the like.
Document), extract and normalize the content attribute information, and preview. In addition, the structure verification unit 2009 manages a plurality of tag sets separately for the browsing document and the content attribute information, and ignores each other to verify the structure.

The attribute information setting device 200 is, as shown by the device 125 in the system of FIG.
Install and use. In this case, according to the attribute information extraction / normalization preview unit 2008, the information provider (IP)
Can automatically preview and see how the content attribute information is extracted and normalized by the automatic information collection and classification device 100 for the created document file (XML document). Settings can be made. FIG. 21 shows an editor screen of the document file (XML document) shown in FIG. 11, and FIG. 22 shows a preview screen of a corresponding attribute extraction / normalization result. The attribute extraction / normalization result preview unit 200 in this example
8 is a simplified version having no unit conversion function, so that the capacity is a mixture of “l” and “ml”. According to the structure verification unit 2009, when handling a plurality of tag sets in which mutual restrictions are weak and restrictions between tags of the same type are strong, for example, the tag sets for the browsing document and the content attribute information are ignored, for example. By performing the grammar verification in this manner, it becomes possible to perform a stricter grammar verification. FIG. 23 shows a conceptual diagram of management / verification of a plurality of tag sets by the structure verification unit 2009.

[0053]

According to the information collection / service providing system to which the present invention is applied, the structure, attribute name, attribute value format, etc. of the attribute information in the document file to be reused are converted into a host device such as a search engine. By having the function, the degree of freedom of expression of a browsing document (for example, a product catalog) can be increased. Conversely, the structure of the existing browsing document
It is possible to build a database for providing services without being restricted by the format.

Also, a device for setting content attribute information by marking up a character string in text information is provided with an attribute extraction / normalization preview function and a function of separately managing and verifying a plurality of tag sets. This allows the data creator to view the results of the host device extracting and normalizing (structure, value format) processing of the attributes, making it possible to set more effective attribute settings, and the mutual restrictions are weak and similar. When a plurality of tag sets having strong restrictions between tags are handled, grammatical verification is performed for each tag set, so that stricter grammatical verification can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an information collection / service providing system to which the present invention is applied.

FIG. 2 is a part of an overall processing flowchart for explaining the operation of FIG. 1;

FIG. 3 is a continuation of the processing flowchart of FIG. 2;

FIG. 4 is a diagram showing an outline of a structure and a processing process of a document which is an object of the present invention.

FIG. 5 is a diagram illustrating content expansion processing (1) of attribute structure normalization according to the present invention.

FIG. 6 is a diagram illustrating a content development process (2).

FIG. 7 is a diagram illustrating normalization of an attribute name.

FIG. 8 is a diagram illustrating attribute division.

FIG. 9 is a diagram illustrating normalization to another attribute.

FIG. 10 is a flowchart illustrating normalization in a numerical format.

FIG. 11 is a specific example of a document file targeted by the present invention.

FIG. 12 shows content attribute information extracted from the document file shown in FIG. 11;

13 is a diagram expressing the content attribute information of FIG. 12 in a tree structure format.

FIG. 14 is a diagram illustrating a result of applying content expansion processing (1) to the content attribute information of FIG. 11;

FIG. 15 is a diagram showing a result of further applying a content development process (2) to the data of FIG. 14;

FIG. 16 is a diagram showing a result of applying a character expression format and a numerical format normalization to the data of FIG. 15;

FIG. 17 is a diagram partially showing the data of FIG. 16 in a tree structure format;

18 is a continuation diagram of the tree structure form expression of FIG. 17;

FIG. 19 is a diagram showing the data of FIG. 16 in a table format.

FIG. 20 is a block diagram of an embodiment of an attribute information setting device according to the present invention.

21 is a diagram showing an editor screen of the document file of FIG. 11 in the attribute information setting device.

FIG. 22 is a diagram showing an editor preview result corresponding to FIG. 19 in the attribute information setting device.

FIG. 23 is a diagram illustrating management and verification of a plurality of tag sets in the attribute information setting device according to the present invention.

FIG. 24 is a block diagram of a conventional search engine system that does not consider content attribute information.

FIG. 25 is a diagram illustrating the structure of a target document in the system of FIG. 24;

26 is a diagram illustrating a specific example of a document targeted by the system of FIG. 24;

FIG. 27 is a block diagram of a conventional automatic collection and classification system in which content attribute information is fixed.

FIG. 28 is a diagram illustrating the structure of a document targeted by the system of FIG. 27;

FIG. 29 is a block diagram of a conventional system for associating content attribute information with a viewing tag and a character string.

FIG. 30 is a block diagram of a conventional attribute information setting device.

FIG. 31 is a diagram illustrating management and verification of a tag set in a conventional attribute information setting device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 Information automatic collection and classification device (host device) 101 Automatic information collection unit 102 Attribute extraction unit 103 Attribute normalization unit 104 Control unit 105 Content database 106 Normalization rule 107 Corresponding rule 108 Service provision unit 110 Network 120 Web site 130 User terminal 120 , 2000 Attribute information setting device 2008 Attribute extraction and normalization preview unit 2009 Structural verification unit

Claims (6)

[Claims] 1. A method for normalizing content attribute information having a structure and format according to a browsing document into a structure and format independent of the browsing document, wherein the content attribute information is mixed with browsing information. Extracting content attribute information from the document file to be extracted; performing a normalization process on an attribute structure for the extracted content attribute information; and for the content attribute information having the normalized structure,
Performing a normalization process of a character expression format and a normalization process of a numerical expression. 2. The content attribute information normalization method according to claim 1, wherein in the attribute structure normalization processing, content expansion, attribute name normalization, attribute division, and normalization to other attributes are performed. Content attribute information normalization method to be characterized. 3. The content attribute information normalization method according to claim 1, wherein the normalization rules for the normalization processing include a field-independent / attribute-independent rule, a field-dependent / attribute-independent rule, and a field-independent / Attribute-dependent rules, field-dependent
A content attribute information normalization method comprising an attribute dependency rule and managing by a content field and an attribute name. 4. An information collection and service providing system in which an information provider's Web site, a host device, and a user terminal are connected via a network, wherein the host device is distributed over the network.
Means for automatically collecting the document files of the site, means for extracting the content attribute information contained in the collected document files together with the browsing information, and means for extracting the extracted content attribute information suitable for providing the service. A means for normalizing to a structure / format, a means for storing the normalized content attribute information, and a means for providing a service to a request from a user terminal using the stored content attribute information. An information collection and service providing system. 5. An apparatus for setting content attribute information by marking up a character string in a document for browsing, extracting and normalizing the content attribute information for previewing, a browsing tag and a content attribute tag. Attribute information setting device characterized by having means for separately managing and browsing the browsing information and the content attribute information. 6. A computer-readable recording medium storing a program for normalizing content attribute information having a structure and format according to a browsing document into a structure and format independent of the browsing document. A processing process of extracting content attribute information from a document file containing content attribute information mixed with application information; a process of performing an attribute structure normalization process on the extracted content attribute information; Content attribute information
A program storage medium, comprising: a process for normalizing a character expression format and a normalization process for a numerical expression.