JP2009026195A - Article classification apparatus, article classification method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily search an article by accurately extracting attribute information on articles from article information. <P>SOLUTION: A feature expansion part 321 expands into features a learning corpus with attribute tags and attribute value tags attached thereto in advance before and after attributes of articles and attribute values representing the contents of the attributes, and an article classification part 322 stores each expanded feature in an attribute learning DB 323 in association with an attribute tag or attribute value tag. When a new article information document is added, a feature expansion part 341 expands the article information document into features, and a tagging part 342 extracts data on attributes and/or attribute values from the article information document feature-expanded by the feature expansion part 341 according to the classification results by the article classification part 322. that is, the attribute learning DB 323. The features are classification information classifying the word containing each character by semantic similarity. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a product classification device, a product classification method, and a program.

  In recent years, opportunities to purchase products have increased due to online auctions and online shopping using communication networks such as the Internet. In an online auction or online shopping, a product is searched according to a product category having a hierarchical structure, or a product is searched using attribute information of the product as a keyword. For example, when the user wants to purchase clothes, the attribute of the desired product (“color”, “size”, etc.) or the attribute value indicating the content of the attribute (“color” is “red”, “yellow”, etc.) If “size”, the product information is searched based on “L”, “M”, etc.).

  The following techniques are known as techniques for searching for products. For example, as an example of performing a search based on the color of a product, an Internet shopping system that stores and manages the color of a product as attribute data of the product and searches for the corresponding product based on the color input by the user on the purchase screen. It has been proposed (see Patent Document 1).

In addition, a product search system is proposed in which an online shop management company reports the product attribute information to the product manufacturer, creates a database in which the attribute information is registered in advance, and searches based on the product attributes (See Patent Document 2).
Japanese Patent Laid-Open No. 2002-92020 JP 2002-259401 A

  By the way, in general, in online auctions and online shopping, categories to which products belong are hierarchically constructed, and product information is classified and managed for each category. Therefore, after the user narrows down the category to which the desired product belongs, the user generally performs a search within that category.

  For example, when a user searches for a red-colored product and searches with a keyword “red”, product information including “red” is searched. At this time, if the product information includes names, store names, or product names (including “Akai”, “Akanishi”, etc.) that contain “red”, there is a risk that even such product information may be searched. The more you have, the more prominent that result. That is, when a product search is performed with attributes such as colors and sizes desired by the user, it is difficult to obtain desired product information because a simple keyword search searches for product information including the keyword.

  Further, in the techniques described in Patent Documents 1 and 2, attribute information is registered in advance for each product information, so that accurate search is possible. However, it is necessary to manually input and register the attribute information in advance, and the work is complicated. In addition, it is not practically practical to adopt the technology for a product search system.

  The present invention has been made in view of the above problems in the prior art, and an object of the present invention is to improve the usability of product search by accurately extracting product attribute information from product information.

  In order to solve the above-mentioned problem, the invention described in claim 1 is a product classification device that classifies a product for each attribute value indicating the attribute and / or the content of the attribute based on the product description, First feature development means for developing a plurality of learning documents to which attribute tags and attribute value tags are assigned to the attributes and attribute values of the product described in the product description; and the first feature development means Classification means for classifying the learning document by associating each feature developed by the feature tag with the attribute tag or the attribute value tag.

  According to a second aspect of the present invention, in the commodity classification apparatus according to the first aspect, a second feature expansion unit that expands the input product information document to be extracted and a classification result by the classification unit. The apparatus further comprises extraction means for extracting attribute and / or attribute value data from the product information document developed by the second feature development means.

  According to a third aspect of the present invention, in the commodity classification device according to the second aspect, the attribute and / or attribute value data extracted by the extracting means is stored in association with the commodity information document from which the data is extracted. Storage means, and search means for searching the product information document associated with the attribute and / or attribute value data received from the client terminal from the storage means and transmitting to the client terminal.

  According to a fourth aspect of the present invention, in the commodity classification apparatus according to any one of the first to third aspects, one of the features used in the feature development includes each development unit in which the feature development is performed. This is classification information in which words are classified according to similarity in meaning.

  According to a fifth aspect of the present invention, in the commodity classification apparatus according to any one of the first to fourth aspects, an SVM (Support Vector Machine) is used as a classification method of the classification means.

  The invention according to claim 6 is a product classification method for classifying the product for each attribute and / or attribute value indicating the content of the attribute based on the product description, which is described in the product description in advance. A first feature development step of developing a plurality of learning documents to which attribute tags and attribute value tags are assigned to product attributes and attribute values, and each feature developed in the first feature development step. A classification step of classifying the learning document by associating with the attribute tag or the attribute value tag.

  The invention according to claim 7 is a computer that classifies the product for each attribute value indicating the attribute and / or the content of the attribute based on the product description, and the attribute of the product described in the product description and A first feature development unit that develops a plurality of learning documents to which an attribute tag and an attribute value tag are attached to an attribute value, and each feature expanded by the first feature development unit is the attribute tag or attribute. This is a program for functioning as classification means for classifying the learning document by associating with a value tag.

  According to the first, sixth, and seventh aspects of the present invention, each feature obtained by developing the feature of the learning document is associated with the attribute tag or the attribute value tag. Extraction can be performed with high accuracy, and usability of product search can be improved using the extracted attribute information.

  According to the second aspect of the present invention, product attribute information can be accurately extracted from the input product information document based on the classification result by the classifying means.

  According to the third aspect of the invention, the attribute and / or attribute value data extracted from the product information document is associated with the product information document from which the data is extracted. Can be extracted with high accuracy and the usability of product search can be improved.

  According to the fourth aspect of the invention, by using the classification information classified based on the similarity of meanings as features, words having similar meanings can be extracted as words indicating the same attribute or attribute value.

  According to the invention described in claim 5, SVM can be used as a classification method.

  Hereinafter, an embodiment of a product classification device according to the present invention will be described with reference to the drawings.

[System configuration]
FIG. 1 shows a system configuration of the auction system 100. As shown in FIG. 1, in an auction system 100, a server device 10 as a product classification device and PCs (Personal Computers) 20a, 20b, 20c,..., 20n (hereinafter referred to as PC 20) are communication networks N. It is connected and configured. The communication network N is a communication network such as the Internet or a telecommunications carrier, and connects devices connected to the communication network N so that data communication is possible.

  The server device 10 is an information processing terminal device such as a PC or WS (Work Station), and classifies the product for each attribute and / or attribute value based on the product description. The server device 10 includes a product information DB (Data Base) 14 that manages product information. The server device 10 has a function as a Web server in a communication session with the PC 20 using HTTP (HyperText Transfer Protocol) or the like. For example, the server device 10 receives product information of products to be exhibited in the online auction from the PC 20 and stores them in the product information DB 14 as product information documents. Further, the server device 10 searches for the product information document stored in the product information DB 14 based on a request from the PC 20 by a user who wishes to purchase the product, and returns a search result.

  The PC 20 is a client terminal used by each user. The PC 20 has a function as a Web browser. When a user places a product for auction, the PC 20 accepts input of product information on an input screen provided by the server device 10, and inputs the input product information to the server device 10. Send. Further, when the user purchases a product, the PC 20 receives an input of a keyword on a search screen provided by the server device 10 and transmits the input keyword to the server device 10.

[Configuration of server device]
FIG. 2 shows the configuration of the server device 10. As shown in FIG. 2, the server apparatus 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a storage unit 13, a product information DB 14, an operation unit 15, a display unit 16, a communication unit 17, and the like. Each unit is connected by a bus 18.

  The CPU 11 comprehensively controls the processing operation of each unit of the server device 10. Specifically, the CPU 11 reads various processing programs stored in the storage unit 13 in response to an operation signal input from the operation unit 15 or an instruction signal received by the communication unit 17, and a work area in the RAM 12. And perform various processes in cooperation with the program.

  The RAM 12 temporarily stores various programs, data, parameters, and the like read from the storage unit 13 in various processes executed by the CPU 11.

  The storage unit 13 includes a hard disk, a nonvolatile semiconductor memory, and the like, and stores various processing programs executed by the CPU 11, various data, and the like. For example, a learning document is stored in the storage unit 13.

  The merchandise information DB 14 is a database for managing merchandise information related to merchandise that the user has submitted for auction. The merchandise information DB 14 stores merchandise information documents, attributes, attribute values, and the like associated with each exhibition merchandise. The merchandise information document is text data including an explanatory text related to the merchandise. An attribute refers to a characteristic of a product, such as “color”, “size”, and the like. The attribute value is a value indicating the content of the attribute. For example, the attribute “color” is “yellow”, the attribute “size” is “M”, and the like. The attribute and attribute value of the merchandise information DB 14 store attribute information (attribute / attribute value) extracted from the merchandise information document in attribute information extraction processing (see FIG. 7) described later. When searching for merchandise, merchandise information documents can be searched using attributes or attribute values as keywords.

  The operation unit 15 includes a keyboard having a cursor key, numeric input keys, various function keys, and the like, and a pointing device such as a mouse, and sends an instruction signal input to the CPU 11 by key operation or mouse operation on the keyboard. Output.

  The display unit 16 is configured by an LCD (Liquid Crystal Display), and performs screen display in accordance with an instruction of a display signal input from the CPU 11.

  The communication unit 17 is a communication interface for connecting to the communication network N under the control of the CPU 11 and transmits / receives data to / from an external device.

  FIG. 3 is a functional block diagram of the server device 10. As illustrated in FIG. 3, the server device 10 includes a learning corpus input unit 31, a machine learner 32, a product information input unit 33, an automatic extractor 34, a product information DB 14, a query reception unit 35, a product information search unit 36, A search result transmission unit 37 is included. The learning corpus input unit 31, machine learner 32, product information input unit 33, automatic extractor 34, and product information search unit 36 are each configured by the CPU 11 and various processing programs stored in the storage unit 13. Realized by collaboration. The query receiving unit 35 and the search result transmitting unit 37 are realized by the communication unit 17.

  The learning corpus input unit 31 is a functional unit that inputs and acquires a learning document (hereinafter referred to as a learning corpus) stored in the storage unit 13. The learning corpus is text data to be used for machine learning, and attribute tags and attribute value tags are assigned before and after the product attributes and attribute values described in advance in the product description.

  Here, the annotation to the product description when creating the learning corpus will be described. Annotation is a process of adding a tag to attribute information (attributes and attribute values) in the learning corpus in advance as a preparation stage for the attribute information learning process (see FIG. 6).

Table 1 shows examples of attributes and attribute values to be annotated.

  As shown in Table 1, as attributes, color, material, size, shape, state, list price, manufacturing place, season / model, design, etc. are adopted. The reasons for selecting these pieces of information include reducing the difference between those who perform annotation when creating a learning corpus, and the fact that users feel that they are necessary for search. In the present embodiment, a fashion category will be described as an example.

  FIG. 4 shows a data example of a learning corpus to which attribute tags (<attr>, </ attr>) and attribute value tags (<val>, </ val>) are assigned. As shown in FIG. 4, for attributes such as “color” and “size”, attribute tags before and after the attribute, such as <attr> color </ attr>, <attr> size </ attr>, etc. Are attached to the "color" attribute value "yellow", the "size" attribute value "M", etc. <val> yellow </ val>, <val> M </ val> As described above, an attribute value tag is added before and after the attribute value.

  When annotating, if an attribute and an attribute value do not appear as a pair, the annotation is performed alone. If the attribute and attribute value are a single compound word, it is decomposed and annotated separately (eg <val> yellow </ val> <attr> color </ attr>). In the case of compound words, they often appear in the order of attribute value-attribute. Also, if the attribute has a hierarchical structure, annotate it individually without considering the hierarchy (for example, <attr> size </ attr>, <attr> shoulder width </ attr>, <attr> length) </ attr>, <attr> Width </ attr>, etc.).

  The machine learning device 32 is a functional unit that performs machine learning based on a plurality of learning corpuses acquired by the learning corpus input unit 31, and includes a feature expansion unit 321, a product classification unit 322, and an attribute learning DB 323.

  The feature development unit 321 performs feature development on a plurality of learning corpora. Specifically, the feature development unit 321 divides the learning corpus in units of characters, and extracts features for each of the divided characters. In the present embodiment, surface characters, character types, parts of speech, and thesaurus classification numbers are used as features.

  FIG. 5 shows an example of feature development. In the example shown in FIG. 5, the character string “material is rayon” is analyzed leftward (from the end of the sentence to the beginning of the sentence) using a total of five characters before and after the target character.

A surface character is a character itself divided in character units.
The character type indicates whether the character is kanji (KANJI), hiragana (HIRAG), katakana (KATAK), or other (OTHER).

  The part of speech indicates the part of speech of the word to which the character belongs, and in the case of “noun”, it includes “common noun”, “proprietary noun” and the like classified in more detail. The feature expansion unit 321 refers to the word-part-of-speech conversion dictionary stored in advance in the storage unit 13 and extracts the part of speech of the word to which the character belongs. The feature development unit 321 also adds a symbol indicating the position of the target character in the word to the head of the feature. “B” is given to the first character in the word, “E” is given to the last character, and “I” is given to the character between them. In addition, “S” is given to a word consisting of one letter.

  The classification number is a type of classification information in which words including each character are classified based on similarity in meaning. The feature expansion unit 321 refers to the word-classification number conversion dictionary stored in advance in the storage unit 13 and extracts the classification number of the word to which the character belongs. In the present embodiment, the number assigned to each word in the Kadokawa Thesaurus new dictionary (Kadokawa Shoten (registered trademark)) is used as the classification number. The vocabulary classification structure of the Kadokawa new dictionary has a decimal classification, and a three-digit number concatenating item numbers in the three levels of large items, medium items, and small items is a classification number. For example, a classification number “143” is assigned to a word related to “color” such as “purple”, “red”, “green”, “color”, and the like. When the classification number is used as a feature, the same classification number is assigned to words having similar meanings, so that even if the surface character is different, it is considered as an example having the same feature.

  The product classification unit 322 classifies the learning corpus by associating each feature developed by the feature development unit 321 with an attribute tag or an attribute value tag. Specifically, the product classification unit 322 is a character in which each character obtained by dividing the learning corpus in character units is included in the attribute or attribute value based on the attribute tag or attribute value tag given to the learning corpus. A classification tag indicating that it exists is assigned. The classification tag is represented by a symbol representing the position of the corresponding character in the chunk and the type of chunk (“attr” for the attribute, “val” for the attribute value) connected with a hyphen. In the IOE2 method, which is one of the chunk encoding methods used in the present embodiment, “E” is assigned to the last character of the chunk, and “I” is assigned to the previous character. “O” is assigned to characters other than elements. In the example shown in FIG. 5, “I-val” is assigned to “ray”, “-”, “yo”, and “E-val” is assigned to “n”. Is done.

  The merchandise classification unit 322 stores each feature obtained for each character by expanding the learning corpus in the attribute learning DB 323 in association with the assigned classification tag. This association is the learning content in the machine learning device 32 and corresponds to the classification of the learning corpus. Specifically, the product classification unit 322 uses SVM (Support Vector Machine) or the like as a classification method, and uses the classification tag “I−” corresponding to the target character “Re” from the information surrounded by the broken line shown in FIG. Generate a classifier that obtains "val". Note that the classification method of the product classification unit 322 is not limited to SVM, and a neural network or the like may be used.

  The attribute learning DB 323 is a database in which each feature obtained based on a plurality of learning corpora is associated with a classification tag. The attribute learning DB 323 is stored in the storage unit 13.

  The merchandise information input unit 33 is a functional unit that inputs and acquires a merchandise information document newly sent in the online auction transmitted from each PC 20 and received by the communication unit 17.

  The automatic extractor 34 is a functional unit for automatically extracting attributes / attribute values from a newly input product information document based on the attribute learning DB 323 obtained by the machine learner 32. 341 and a tagging unit 342.

  The feature development unit 341 develops the feature information document acquired by the product information input unit 33. Specifically, the feature development unit 341 divides the product information document in character units, and extracts features (surface character, character type, part of speech, classification number) for each divided character. The details of the feature development process are the same as those of the feature development unit 321, and are therefore omitted.

  The tagging unit 342 extracts attribute and / or attribute value data from the product information document developed by the feature development unit 341 based on the classification result by the product classification unit 322, that is, the attribute learning DB 323. Specifically, the tagging unit 342 assigns a classification tag associated with each feature to each feature based on each feature and attribute learning DB 323 obtained for each character obtained by developing the feature information document. To do. For example, if the tagging unit 342 estimates that the target character is a character included in the attribute, the tagging unit 342 adds “I-attr” or “E-attr” depending on the position in the chunk, and the target character Is assumed to be a character included in the attribute value, "I-val" and "E-val" are assigned according to the position in the chunk, and the target character is assigned to both the attribute and attribute value. If it is estimated that the character is not applicable, “O” is given. As the tagging unit 342, SVM or the like is used.

  Further, the tagging unit 342 extracts an attribute and / or attribute value from the product information document based on the assigned classification tag. Specifically, the tagging unit 342 uses a word assigned “I-attr” or “E-attr” as an attribute, and a word given “I-val” or “E-val” as an attribute value. The extracted attribute and / or attribute value is associated with the product information document from which the attribute and / or attribute value is extracted and stored in the product information DB 14.

  Note that the process up to adding a classification tag by the tagging unit 342 and making the attribute and attribute value identifiable in the product description may be handled as attribute / attribute value extraction.

  The query receiving unit 35 receives a search query transmitted from the PC 20. The search query includes attributes and / or attribute values as keywords for searching for product information documents.

  The product information search unit 36 searches the product information document stored in the product information DB 14 based on the attribute or attribute value included in the search query received by the query receiving unit 35, and searches the search keyword (attribute or attribute). The product information document associated with (value) is extracted from the product information DB 14, and the search result is output to the search result transmission unit 37.

  The search result transmission unit 37 transmits the search result searched by the product information search unit 36 to the PC 20.

[Operation of server device]
Next, the operation of the server device 10 will be described.
FIG. 6 is a flowchart showing the attribute information learning process. The attribute information learning process is a process that is performed in advance before the server device 10 provides the auction system 100, and is realized by a software process in cooperation with the CPU 11 and a program stored in the storage unit 13.

  First, the learning corpus input unit 31 inputs and acquires the learning corpus stored in the storage unit 13 (step S1). Next, the feature development unit 321 develops the feature corpus for each character (step S2). Specifically, the feature development unit 321 divides the learning corpus in units of characters (step S21), and for each divided character, a surface character, character type, part of speech, and classification number are extracted as features (step S22). .

  Next, the learning corpus is classified by associating each feature-expanded feature with an attribute tag or attribute value tag by the product classification unit 322 (step S3). Specifically, based on the attribute tag or attribute value tag assigned to the learning corpus, the product classification unit 322 assigns a classification tag to each character obtained by dividing the learning corpus in character units.

Next, the feature classification unit 322 stores each feature obtained for each character by developing the feature corpus in association with the classification tag in the attribute learning DB 323 (step S4).
This completes the attribute information learning process.

  Next, attribute information extraction processing will be described with reference to FIG. The attribute information extraction process is a process performed when product information of a product is transmitted from the PC 20 and received by the communication unit 17 by an operation of a user who wishes to exhibit in the auction, and is stored in the CPU 11 and the storage unit 13. This is realized by software processing in cooperation with a program that has been implemented.

  First, a product information document to be extracted is input and acquired by the product information input unit 33 (step S5). Next, the feature development unit 341 develops the feature information document in units of characters (step S6). Specifically, the feature development unit 341 divides the product information document in units of characters (step S61), and the surface character, character type, part of speech, and classification number are extracted as features for each divided character (step S62). .

Next, the tagging unit 342 extracts attribute and / or attribute value data from the product information document that has been developed based on the classification result by the product classification unit 322, that is, the attribute learning DB 323 (step S7). Specifically, the tagging unit 342 assigns a classification tag associated with each feature to each feature based on each feature and attribute learning DB 323 obtained for each character obtained by developing the product information document. Then, attributes and / or attribute values are extracted from the product information document based on the assigned classification tags. The words extracted as attributes or attribute values are associated with the product information document by the tagging unit 342 and stored in the product information DB 14 (step S8).
This completes the attribute information extraction process.

  As described above, the server device 10 assigns each feature obtained by developing the feature corpus to an attribute tag or an attribute value tag (classification tags “I-attr”, “E-attr” corresponding to the attribute tag, or Classification tag corresponding to the attribute value tag "I-val" "E-val"), it can be extracted accurately when extracting product attribute information from the product information document Usability of product search can be improved by using attribute information.

  Further, based on the classification result classified by the machine learner 32 using the learning corpus, the attribute information of the product can be accurately extracted from the product information document by the automatic extractor 34. In addition, it is possible to automatically classify products by applying a classification method to product descriptions written by users in online auctions and online shopping.

  Further, by using the classification number as a feature, words having similar meanings can be extracted as words indicating the same attribute or attribute value. Extraction accuracy when the product information document to be extracted is in a different category by using attribute name, attribute value extraction, and synonym dictionary information based on sequence labeling for character strings by machine learning Reduction can be reduced.

  Further, since the attribute and / or attribute value data extracted from the product information document is associated with the product information document from which the data is extracted, the usability of product search can be improved.

  For example, by extracting attributes and attribute values from a product information document, you can perform attribute information searches across multiple product categories, and register the extracted attributes and attribute values as indexes in the search system. You can search.

  In addition, the attribute information (color, material, size, shape, etc.) of the exhibit determined according to the category is taken as a set of attribute name (“color”, etc.) and attribute value (“red”, etc.) Extract them using machine learning techniques. For this reason, for example, when searching with the search query “red”, product information having the attribute “red” and the attribute value can be searched, and the search queries “red” and “shirt” are used. It is possible to search for product information across a plurality of categories so as to search for red shirts from a plurality of categories.

  The description in the above embodiment is an example of the product classification device according to the present invention, and the present invention is not limited to this. The detailed configuration and detailed operation of each part constituting the apparatus can be changed as appropriate without departing from the spirit of the present invention.

  For example, in the above embodiment, the learning document and the product information document to be extracted are expanded in character units. However, the feature expansion may be performed in morpheme units, which is the minimum grammatical unit.

  Moreover, although what was shown in Table 1 as an attribute which shows the characteristic of goods was mentioned as an example, it is good also as, for example, catching the category to which goods belong as an attribute, and extracting an attribute name and / or an attribute value. Thereby, learning corpus of various categories can be input to learn attributes and attribute values related to the categories, and for example, categories such as attribute name “category” and attribute value “fashion” can be associated with the product information. Accordingly, it is possible to search for a product specifying a category as an attribute, and to search for product information belonging to a plurality of categories by specifying the category.

  Moreover, although the said embodiment demonstrated the case where the goods classification apparatus was applied to the auction system 100, it is good also as applying to the online shopping system which provides goods using a communication network.

[Experimental example]
Experiments 1 to 4 were conducted in order to examine the accuracy of extracting attribute information from the product information document.
For chunking, Yamacha, a general-purpose chunker based on SVM, was used. When developing the feature, the character was used as a unit, and the analysis direction of chunking was performed by left-facing analysis. In addition, the IOE2 method is used for the encoding method of attribute and attribute value chunks, and the context length is set to 5 characters in total, 2 characters before and after the target character.

  As the experiment data, the fashion category of product information of products exhibited at the Yahoo! (registered trademark) auction was used. At this time, in order to eliminate the influence of the description style unique to the exhibitor, consideration was given to not using the product information that the exhibitor duplicated. Details of the data used for this are shown below.

(A) Apparel (for men)-Tops-Shirt-Short sleeves 150 pages Attribute: Total 1422 / Different 149 Attribute value: Total 1794 / Different 512 (B) Apparel (Women)-Tops-Tank top , Camisole 150 pages Attribute: Total number 723 / Different number 91 Attribute value: Total number 1245 / Different number 381

  When used as a learning document (hereinafter referred to as learning data), a document (text data) that has been annotated in accordance with the above-described method is used for the product information (experiment data (A) or (B)). It was.

<Experiment 1>
In Experiment 1, (A) "Apparel (for men)-Tops-Shirt-Short sleeves" was used as experimental data, and experiments were performed using features other than classification numbers, that is, surface characters, character types, and parts of speech. In the evaluation, a 5-fold cross-validation was performed with the product information as a unit, and the average precision and recall were obtained. here,
Relevance rate = number of attribute information successfully extracted / number of attribute information extracted by server apparatus Reproducibility = number of attribute information successfully extracted / number of attribute information in data.

Table 2 shows the extraction accuracy in Experiment 1.

  As shown in Table 2, regarding the attributes, both the relevance ratio and the recall ratio are 80% or more, and it is considered that the attribute information could be extracted with a certain degree of accuracy. In addition, although the attribute value remained in the 70% range, it is considered that the extraction accuracy is suppressed because there are many types of objects to be extracted compared to the attribute. It is considered that this can be solved by increasing the amount of learning data.

<Experiment 2>
In Experiment 2, in order to examine the effect of using surface character as a feature and the effect of using Kadokawa's new dictionary classification number as a feature for extracting attribute information by machine learning, (A) Experiments were conducted using “apparel (for men) —tops—shirts—short sleeves” using features according to the following conditions.

Condition (A): Surface character / character type / part of speech Condition (A): Surface character / character type / part of speech / classification number Condition (U): Character type / part of speech Condition (D): Character type / part of speech / classification number

  In the evaluation, a 5-fold cross-validation was performed with the product information as a unit, and the average precision and recall were obtained.

Table 3 shows the extraction accuracy in Experiment 2.

  Comparing the results of condition (a) and condition (b), we were able to confirm the effects of using classification numbers as features. However, since the dependence on surface characters is high, it can be confirmed at the same time that the effect of the classification number is small.

  Comparing the results of the condition (c) and the condition (d), it can be said that the classification number works very effectively in improving accuracy when the surface character is not used as a feature. In other words, it is possible to maintain a certain level of extraction accuracy with only features that do not depend on the surface representation, so it is possible to extract even new attributes and attribute values that do not appear in the learning data as long as the expression appears in the existing thesaurus. It is expected. In particular, it is considered effective in extracting attributes and attribute values in product information related to products in a new field.

<Experiment 3>
In Experiment 3, an experiment was conducted to examine the relationship between the amount of data used for learning data and the extraction accuracy. As experimental data, (A) "apparel (for men)-tops-shirt-short sleeve" was used, and surface characters, character types, parts of speech, and classification numbers were used as features. When evaluating, use the 5-fold cross-validation method with product information as a unit, gradually extract data from all the data included in the four groups used as learning data, increase the learning data to be used, and average their The precision and recall were calculated.

  FIG. 8 shows the relationship between data amount and accuracy in attribute extraction, and FIG. 9 shows the relationship between data amount and accuracy in attribute value extraction. 8 and 9, the horizontal axis represents the number of product information document pages used as learning data, and the vertical axis represents the relevance ratio or the recall ratio.

  As shown in FIG. 8 and FIG. 9, it is confirmed that the precision can be obtained with a certain degree of accuracy with a small amount of data, but the accuracy can be improved with respect to the recall by increasing the amount of data. did it. Regarding the recall rate, the increase in accuracy is not saturated, and if the data is further increased, the accuracy may be improved.

<Experiment 4>
In Experiment 4, in order to examine the accuracy when automatic extraction was performed using product information belonging to different groups for learning data and a product information document (hereinafter referred to as test data) used as a new extraction target, The average precision and recall were obtained under the following conditions.

Condition (A): Learning data: Experimental data (A)
Test data: Experimental data (B)
Usage features: Surface characters, character types, parts of speech Condition (b): Learning data: Experimental data (A)
Test data: Experimental data (B)
Use feature: Character type, part of speech, classification number Condition (c): Learning data: Experimental data (B)
Test data: Experimental data (A)
Features used: Surface characters, character types, parts of speech Condition (d): Learning data: Experimental data (B)
Test data: Experimental data (A)
Use features: Character type, part of speech, classification number

  Experimental data (A) "Apparel (for men)-Tops-shirt-short sleeve" and experimental data (B) "Apparel (for women)-Tops-Tank top, camisole" are among "Apparel-Tops" , The similarity of the appearing attribute information is low.

Table 4 shows the extraction accuracy in Experiment 4.

  As in Experiment 1 and Experiment 2, etc., the overall accuracy is lower than when automatic extraction is performed using product information (apparel (for men), tops, shirts, short sleeves) in the same group. Recognize. This is because the similarity of the attribute information appearing in the document is lowered.

  Also, as can be seen by comparing the conditions (a) and (b) and the conditions (c) and (d), the extraction accuracy is higher when using surface characters than when using classification numbers. I can say that. However, in the case where the similarity of appearing attribute information is lower because the difference in accuracy between using surface characters and using classification numbers is smaller than when using similar product information It is considered that the effect of the classification number becomes larger.

It is a system configuration figure of an auction system concerning an embodiment of the invention. It is a block diagram which shows the structure of a server apparatus. It is a functional block diagram of a server apparatus. It is a data example of the learning corpus to which the attribute tag and the attribute value tag are given. It is a figure for demonstrating feature expansion. It is a flowchart which shows an attribute information learning process. It is a flowchart which shows an attribute information extraction process. It is a figure which shows the relationship between the data amount and the precision in attribute extraction. It is a figure which shows the relationship between the data amount and precision in extraction of an attribute value.

Explanation of symbols

10 Server device 11 CPU
12 RAM
13 Storage Unit 14 Product Information DB
15 Operation unit 16 Display unit 17 Communication unit 18 Bus 20 PC
31 Learning Corpus Input Unit 32 Machine Learner 321 Feature Expansion Unit 322 Product Classification Unit 323 Attribute Learning DB
33 Product information input unit 34 Automatic extractor 341 Feature development unit 342 Tagging unit 35 Query reception unit 36 Product information search unit 37 Search result transmission unit 100 Auction system N Communication network

Claims (7)

A product classification device that classifies the product for each attribute value indicating the attribute and / or the content of the attribute based on the product description,
First feature development means for developing a plurality of learning documents to which attribute tags and attribute value tags are assigned to the attributes and attribute values of the products described in advance in the product description;
Classification means for classifying the learning document by associating each feature developed by the first feature development means with the attribute tag or attribute value tag;
A product classification apparatus comprising: Second feature expansion means for expanding the input product information document to be extracted;
Extraction means for extracting attribute and / or attribute value data from the product information document that the second feature development means has developed based on the classification result by the classification means;
The product classification device according to claim 1, further comprising: Storage means for storing the attribute and / or attribute value data extracted by the extraction means in association with the product information document from which the data is extracted;
Search means for searching for product information documents associated with attribute and / or attribute value data received from the client terminal from the storage means and transmitting to the client terminal;
The product classification device according to claim 2, further comprising:   One of the features used in the feature expansion is classification information obtained by classifying words including each expansion unit subjected to the feature expansion based on similarity in meaning. Product classification device of description.   The product classification device according to any one of claims 1 to 4, wherein an SVM (Support Vector Machine) is used as a classification method of the classification means. A product classification method for classifying the product into attributes and / or attribute values indicating the contents of the attribute based on the product description,
A first feature development step for feature development of a plurality of learning documents to which an attribute tag and an attribute value tag are attached to an attribute and an attribute value of a product described in advance in a product description;
A classification step of classifying the learning document by associating each feature developed in the first feature development step with the attribute tag or the attribute value tag;
Product classification method including A computer that classifies the product into attributes and / or attribute values indicating the contents of the attribute based on the product description.
First feature expansion means for expanding a plurality of learning documents to which an attribute tag and an attribute value tag are assigned to an attribute and an attribute value of a product described in advance in a product description;
A classifying unit that classifies the learning document by associating each feature developed by the first feature development unit with the attribute tag or the attribute value tag;
Program to function as.

Images