JP2002351875A - Database contents quality analyzing method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for calculating the score of searchable an electronic catalog used in an electronic commerce and an industrial material system. SOLUTION: A catalog is built typically as a database, and the method according to the present invention analyzes the quality of completeness, consistency, or comprehensiveness. This method selects fields in the database to be analyzed, ranks the fields to be analyzed in order of relevancy of quality on which the measurement is based (52), fetches the values of the records in the database from the fields to be analyzed (56), and compares the fetched values with the standard values (58). After the comparison, a score is assigned to each field (62). The scores of the fields are weighted based on the rank of the fields (64) and the weighted scores are combined (66) for calculating the score of the database.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates generally to electronic databases, and more particularly, to comprehensibility of database content, which affects the usefulness of the database in comparison to other databases.
sibility), methods and apparatus for analyzing various qualities, such as completeness and consistency.

[0002]

BACKGROUND OF THE INVENTION Searchable electronic catalogs are commonly used in support of e-commerce and purchasing functions. These electronic catalogs can be created from printed catalogs, spreadsheets, text documents, databases or lists, typically in databases, collections of HTML pages, and other electronic means. The installation of individual buyers or market systems often includes several catalogs from several sources. For example, an office supply installation may include office supply catalogs from several different office supply vendors or manufacturers. While some of the catalogs may describe equal items, such as blue pens, each catalog is more likely to describe similar but different items, such as differently constructed blue pens. These catalogs can differ in their quality and ease of use, as measured by the ability of a user to find and purchase items. Objective measurement of the quality of each catalog allows catalogs to be compared and catalog shortages to be quickly identified. With sufficient support, such analysis can quickly pinpoint the source of the shortfall.

[0003] Three important aspects of catalog use are purchasing, item identification and validation, and discovery.
Sufficient information must be present in the catalog to describe the item so that users or potential buyers can find the item. Catalog suppliers seek to present a catalog that maximizes the likelihood that an item will be found, identified, and then purchased. The information needed for the purchase can be only the part number or can include a very detailed item description with images and interactive applications. Catalogs that support a greater amount of specific information generate greater sales, so they are important in assessing the usefulness of the catalog and as to how easy it is for buyers to find the item they are looking for. Higher scores will be scored in evaluating different attributes.

[0004]

SUMMARY OF THE INVENTION In a preferred embodiment, the present invention provides a database with quality, for example,
Provides a method for scoring for completeness, consistency or comprehension.

[0005]

The method comprises selecting a field to be analyzed in a database, retrieving a value for each record in the database from the field to be analyzed, and comparing the retrieved value to a standard. Including doing. After the comparison, a score is preferably assigned to each field based on the comparison. The fields are ranked in an order that is appropriate for the quality being measured, and a score is weighted for each field based on the rank of each field. The weighted scores are finally combined to obtain a database score.

If the quality being analyzed is complete, the present invention uses the values retrieved for the fields as
Includes comparing to another value retrieved for the same field. Assigning a score includes assigning a point for each null value so that the score for the field corresponds to the number of null values for all records in that field.

If the quality analyzed is consistent, the invention involves comparing the values retrieved for the field with a dictionary of possible values. Assigning a score assigns a point to each retrieved value that does not match the dictionary value, so that the field score corresponds to the number of unmatched values for all records for that field. including.

If the quality being analyzed is comprehensible, the present invention includes comparing the retrieved value for the field with a dictionary of possible values, and assigning a score to the retrieved value. Assigning a point to each value that does not match the dictionary value so that the score for the field corresponds to the number of unmatched values for all records for that field.

[0009] The appended claims set forth the features of the present invention with particularity. The invention, together with its advantages, may best be understood from the following detailed description, taken in conjunction with the accompanying drawings.

[0010]

DETAILED DESCRIPTION In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form.

The present invention includes various steps, which are described below. The steps of the present invention can be performed by hardware components or can be implemented in machine-readable instructions, which can be used to
A general purpose or special purpose processor, or logic programmed with instructions, may cause these steps to be performed. Alternatively, these steps can be performed by a combination of hardware and software.

The present invention may be provided as a computer program product, which includes a machine readable medium having stored thereon instructions, which are used to perform a process according to the present invention (or other computer). Electronic devices) can be programmed. Machine readable media include floppy disks, optical disks,
CD-ROM and magneto-optical disk, ROM, RAM,
Including, but not limited to, EPROM, EEPROM, magnetic or optical card, flash memory, or other types of machine-readable media suitable for storing media / electronic instructions. In addition, the present invention may be downloaded as a computer program product, wherein the program is transmitted from a remote computer to a requesting computer by a data signal embedded in a carrier wave or other propagation medium, such as a modem or modem. Network connection).

Significantly, embodiments of the present invention are described with respect to analyzing the quality of a catalog to allow a user, such as a potential customer, to find and identify items of particular interest. The methods and apparatus described herein are equally applicable to the analysis of any type of database where a particular quality is measured. For example, the techniques described herein may be useful for databases for client or customer management, inventory management, transportation management and scheduling.

The invention is preferably implemented in Java software instructions, but any other computer programming language can be used. Java code can operate on a wide variety of computer systems.
One example of such a computer system in which the invention can be implemented is described below with reference to FIG. The computer system includes a bus or other communication means 1 for communicating information, and processing means for processing information, such as a processor 2 coupled to the bus 1. The computer system further includes a random access memory (RAM) or other dynamic storage device 4 (also referred to as main memory), which is coupled to bus 1 and stores information and instructions executed by processor 2. What to store. Main memory 4 may also be used to store temporary numeric variables or other intermediate information during execution of instructions by processor 2.
The computer system has read-only memory (RO)
M) or other static storage device 6;
It is coupled to bus 1 and stores static information and instructions for processor 2.

[0015] A data storage device 7, such as a magnetic or optical disk, and its corresponding drive may also be coupled to the computer system for storing information and instructions. The computer system is
Via the bus 1, it can also be coupled to a display device 21, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), for displaying information to the end user. For example, graphical and textual instructions for installation status, remaining time in the trial period, and other information are displayed to prospective buyers.
It can be presented on the device 21. Typically, the alphanumeric input device 22 includes alphanumeric and other keys,
Information and command selection can be coupled to bus 1 for communicating to processor 2. Another type of user input device is a cursor control 23, for communicating directional information and command selections such as a mouse, trackball, or cursor direction keys to the processor 2 and for controlling cursor movement on the display 21. belongs to.

A communication device 25 is also coupled to bus 1.
Communication device 25 may include a modem, a network interface card, or other well-known interface devices, such as Ethernet, Token Ring, or, for example,
Other types of physical attachments and the like to configure communication links to support local or wide area networks. In any case, the computer system can thus be coupled to several clients or servers via a conventional network infrastructure, such as, for example, a corporate intranet or the Internet.

It should be appreciated that fewer or more equipped computer systems than the examples described above may be desirable for certain implementations. Thus, the configuration of a computer system will vary from implementation to implementation depending on a number of factors, such as price constraints, performance requirements, technical improvements, and other considerations.

Although the steps described herein may be performed under the control of a programmed processor, such as processor 2, in an alternative embodiment, the steps may be completely or partially performed by any programmable or hard- ware. Coded logic, eg,
Field Programmable Gate Array (FPG)
A), TTL logic, or dedicated integrated circuit (ASI)
It should be noted that this can be performed by C) or the like. In addition, the method of the present invention may be performed by any combination of programmed general-purpose computer components or custom hardware components. Therefore, anything disclosed herein should not be construed as limiting the invention to the particular embodiments in which the recited steps are performed by particular combinations of hardware components. Absent.

The present invention is directed to analyzing a list of data, and in a preferred embodiment is directed to analyzing an electronic catalog. The catalog can exist as a database or in any other electronic format, such as a spreadsheet or text. If there is no electronic format, scan a paper catalog or text document in electronic format,
A standardized list of items can be processed along with their descriptions. This application describes the invention with respect to a database. In the context of the present invention, the term database should not be construed as being limited to any particular type of structure, but rather, in a broader sense, as a list or sequence accompanied by an explanation to an item Should. Such a database can be viewed, for example, as a collection of two-dimensional tables, with each row representing a different record and each column representing a different field. Each record corresponds to a particular item.
In the case of a catalog of office supplies, the record provides catalog information about a particular office supply, such as a particular pen. Different pens have different records. Each record has several fields. Each field describes the attributes of the item corresponding to the record, such as price, color, weight, and size. The present invention analyzes the values entered in the fields of the database.

FIG. 2 shows an application of the present invention, generally for analyzing the quality of a database.
In FIG. 2, the process begins with selecting a field to be analyzed (30). Typically, not all fields are given the same importance. This will be understood in the examples that follow. After the fields are selected, they are ranked in order of importance (32). The present invention examines the deficiencies, excesses, and variability in the values of the fields in the database and provides different scores if different fields are not matched to different levels of importance in scoring. No. Each field is given a weight based on its ranking, and this weight is used to determine the final score. After the fields have been selected and ranked, the value of each selected field in the database is retrieved (34) and then analyzed through a comparison process (36). The particular type of comparison depends on the particular quality being analyzed. After the comparison, a score is assigned based on the comparison (3
8). This score is the basic input to the overall score for the database. As mentioned above, a weighting factor is assigned to each field using the ranking of the fields (40). These weighting factors are preferably recorded in a table used to add weights to each score (42). Preferably, the weights constitute a multiplication factor between all zeros and ones, but numerical scaling can be done in a variety of different ways. Finally, the weighted scores are combined (44) to produce an overall score for the particular quality under analysis for the database. Scores for multiple qualities can be combined to provide a more comprehensive score for the database. This result can also be normalized to facilitate comparison between different databases or electronic sources.

In a preferred embodiment, the present invention can be used to measure the ease with which items can be found in a catalog. Preferably, three components are analyzed. Integrity checks to see if the attributes and field values for the catalog entry are present in the catalog, or whether the important fields for each record contain data entries. The emphasis is
Attributes that are important for discovery and purchasing,
(Stock item identification number), price, supplier name, description, and the like. Catalogs that lack these entries (including null field values) are more difficult to use.
Consistency looks for consistent use of common abbreviations and units of measurement. Comprehension examines how a product is described by assessing the use of words. The words are examined using a dictionary, including units of measurement and common abbreviations in the description field,
The parts of speech are analyzed for appropriateness and count.

FIG. 3 is an example flow chart for analyzing integrity. In the example of analyzing an electronic catalog, the integrity analysis is a test for the presence of all attributes of the product that need to be purchased, as well as the presence of field values that enhance the ability to find the product. Is preferred.

In FIG. 3, the process of analyzing the database for quality of integrity begins with selecting the fields to be analyzed (50). Typically, in the example of an electronic catalog, the fields SKU, Price, Supplier Name, and Description are selected. However, the particular fields selected will depend on the particular database being analyzed and the fields deemed most important.
After the fields are selected, they are ranked 52 in order of importance. Typically, the ranking is
SKU, price, supplier name, and description. The ranking of particular databases, regions of database content, and fields will depend on the particular database and the purpose of the analysis. Next, weights are assigned based on the ranking (54). An example of the weight to be added is S
KU: 1.0, price: 0.75, supplier name: 0.5, and description: 0.25.

After the fields have been selected and ranked, the values in the database for each selected field are retrieved (56) and then compared (5).
Analyzed through 8). In particular, the value of the field is compared to a null value, ie, a determination is made as to whether there is any data entered into the field for a particular field. A count is then performed on all of the null values for each field (6
0). A score is assigned based on the comparison (6
2). Preferably, the score is simply the number of non-null values for each field. Preferably, a weighting factor is assigned to each score (64). Finally, the weighted scores are combined (66) to generate an overall integrity score for the database under analysis.

If three different fields are being analyzed, a mathematical example for determining the integrity score is as follows. Completeness score = (w1 * f1 (n) + w2 * f2
(N) + w3 * f3 (n)) / (w1 + w2 + w3)

However, it becomes as follows. f1 (n) = Σ ([all first group fields] * [count of first group fields with non-null values per record] * [record under evaluation (ie, product in catalog) Count]) / ([all first group fields] *
[Count of records under evaluation]) f2 (n) = Σ ([all second group fields] * [count of second group fields with non-null value per product] * [under evaluation Of products (ie, catalogs)]) / ([all second
F3 (n) = Σ ([all third group fields] * [third group fields that have a non-null value for the product) Count] * [count of product under evaluation (ie, catalog)] / ([all 3rd
Field of the group of *) * [Count of products under evaluation]) where w1, w2 and w3 are the corresponding weights for the first to third fields, respectively.

A detailed report of completeness is typically
Shows the percentage completeness (non-null value) for all selected fields, a list of selected fields, and percentage completeness by category for all fields. In addition, important attributes can indicate the number of items missing in the field, the number of items with rich content (ie, images), and the number of items without categories. Finally, a percentage of completeness by score for all fields can be provided. This can be used to focus on areas where data value improvement efforts are most needed.

It may be desirable to generate a score based on a region, category or attribute. For a catalog that spans several domains, it would be useful to understand which domains have the highest level of completeness and which domains require the greatest improvement. Within a particular area, a catalog user or creator can benefit by understanding which categories of goods or services can benefit most from the modification.
Attributes (descriptors or specifications) that pertain to groups of fields provide another useful basis for reporting to catalog users or creators. If the incomplete field belongs to an attribute that is common across the catalog, such as SKU and price, the incomplete field will have a different modification than if it relates to a category-specific attribute, such as color or power. Effort will be needed.

FIG. 4 presents an application of the present invention for analyzing consistency. Catalog users generally prefer consistency in the way items are described. This allows
As the user searches for product descriptions, the confidence that all items, such as the desired product, will be found and displayed is increased. The first elements of consistency are word usage, units of measurement, and abbreviations, for example, ft. , F. , F
T. Or use foot. The use of unnecessary or inconsistent synonyms, that is, synonyms that do not indicate a difference in the product, is confusing and prevents efficient use of the catalog. Abbreviations with a number of possible meanings (CT = Carton or Crate)
Or Connecticut) can cause ambiguity, which also prevents efficient use of the catalog. The present invention allows a user to score synonym usage using a thesaurus defined as a database of synonyms.

Consistency in abbreviation usage is desirable both for catalog consistency and to avoid ambiguity. A table of abbreviations can be created showing preferred abbreviations. The scoring of this consistency component can be based on the ratio of preferred abbreviations to total abbreviations.
The frequency at which unique combinations occur is the second method. The combination of several methods allows for a weighted score for the overall consistency component.

In FIG. 4, the process of analyzing the database for quality of integrity begins with selecting the fields to be analyzed (70). Typically, in the example of an electronic catalog, fields containing units of measurement and abbreviations are selected. In the catalog, you can select fields for dimensions, colors, types and shipping dates. After the fields are selected, they are ranked in order of importance (72). The particular fields selected, and the ranking of the fields, will depend on the particular database and the purpose of the analysis. Next, weights are assigned based on the ranking (74). Examples of weights to be added are: size: 1.0, weight: 0.75, color: 0.
5, and shipping date: 0.25.

After the fields are selected and ranked, in the database the value for each selected field is retrieved (76) and then analyzed through a process of comparison (80). In particular, the value of each field is compared to the value in the thesaurus (78). The thesaurus is specifically designed for the type of catalog being analyzed. This can be provided by the creator of the catalog or can be based on the needs of the particular user of the catalog. Preferably, the thesaurus contains a complete list of synonyms well understood in the field of units of measurement and abbreviations. Different thesauruses may be needed for different categories or regions.

In the comparison, a determination is made as to whether the unit of measure or abbreviation value from each record matches an entry in the thesaurus. A count is then made on all of the different matching values for each field (82). A score is assigned based on the number of matches (84). Preferably, the score is simply the number of values finding a match in the thesaurus for each field divided by the total number of non-null values. A weighting factor is then assigned to each score (8
6). Another score that can be developed from the comparison (80) is a count (88) of all unique values in each field. For example, the unit of measure value "l
b. "," Lb. "," pd. "And" ound "
Are added together to form four counts, no matter how many times each of these values occurs in the weight field. A score is assigned to this total number of unique values (90), such that a greater number of synonyms produces a lower score. A preferred score is the total number of synonym groups divided by the count of synonyms found for each synonym group. A group of synonyms is, for example, weight in pounds,
Synonyms are various ways of representing pounds above (l
b. , Lb. , Pound, etc.). Then the score is
Weighted in the same way as the total number of matches (9
2). Finally, the weighted scores are combined (9
4) An overall consistency score is generated for the database under analysis. The overall consistency score preferably reflects a ratio of (redundant abbreviations and units of measurement) / (unique abbreviations and units of measurement). A complete mathematical analysis is very similar to that presented above for completeness.

The detailed consistency report for the catalog preferably shows, by category and by attribute: Number of abbreviations Number of unique abbreviations Number of redundant abbreviations Number of units of measurement Number of unique units of measurement Number of units of redundant measurement

A third example of quality to analyze is comprehension. In one method for analyzing complications, the present invention determines whether item descriptions in a catalog use native language words and whether various words match the size of the catalog. Find out. Numeric and alphanumeric strings are
It is excluded from the analysis as it is a known unit of measure and abbreviation. The number is assumed to be the part number or value associated with the descriptor. Alphanumeric strings are assumed to be part numbers. Units of measure and abbreviations are:
It is processed in the consistency evaluation described above.

[0036] Additional analysis can be generated to examine the use of nouns and adjectives in describing items. The invention can also analyze the optimal range of values to describe an item in a given domain, and the relationship between the number of unique nouns and the number of categories. In this case, the grade can be associated with the rate at which unique words are found in the dictionary. Each recurrence of the word is not counted. Other factors to include include the number of words in the catalog, the number of unique words in the catalog, the number of nouns used to distinguish each record or item by having a unique SKU, and
Number of adjectives per record. This last measure can also be taken into account by measuring the percentage of records described by at least one word. In catalogs where users prefer written descriptions, a statistical count of the range of the descriptions is valuable. Preferably, all these measurements are sorted by category and by attribute so that the most effective measurements are provided to catalog users and creators.

In FIG. 5, the process of analyzing the database for the quality of the completion begins with selecting the fields to be analyzed (100). Typically, in the example of an electronic catalog, a field containing a text description is selected. In the database, this information can be spread over several fields associated with the product, and all fields can be examined as a whole. The user can select which fields are appropriate for a particular situation. Further, the part of speech across the entire catalog can be analyzed as an indication of the catalog's ability to distinguish similar items. The text component of the product description can be evaluated for sufficiency as well as consistency. Sufficiency is to provide enough description to effectively describe the product as well as to effectively distinguish the items in the catalog. By examining the number and variance of each part of speech (noun, adjective, etc.) for each item, a high degree of indication of the information carried about that item is obtained. After the fields are selected, they are ranked in order of importance (102). Next, weights are assigned based on the ranking (104). After the fields are selected and ranked, in the database the value for each selected field is retrieved (106) and then analyzed through the process of comparison with the dictionary 110 (108). The dictionary is specifically designed for the type of catalog under analysis.

In the comparison, a determination is made as to whether each word for each record matches an entry in the dictionary (112). A count is then made on all of the different matching values for each field (114). A score is assigned based on the number of matches (114). Preferably, the score is simply the number of values that find a match in the dictionary for each field. A weighting factor is then assigned to each score (116).

Another score that can be developed from the comparison (108) is the count of all nouns in each field (118). Dictionary comparisons can be used to determine the part of speech of a word. The noun count is assigned a score (120), preferably just a count, and then weighted in the same way as the total number of matches (122). In addition, adjectives are counted (12
4) A score for each field is assigned (1
26), appropriately weighted (128). Finally, all weighted scores are combined (130),
An overall completion score is generated for the database under analysis. Scoring involves finding words and total words, the ratio of instances of unique words found to total unique words, and non-verbal texts such as part numbers and non-descriptive Based on filtered ratios of uninteresting text, such as prepositions).

Scoring is based on a value-driven method in which the scores for each component are normalized. As the scoring components are aggregated into a larger representation, each aggregated score is renormalized. Any component consisting of multiple elements has a weight added to reflect that element's relative value in relation to other catalog elements. Weights are added at all levels of the scoring aggregate. The user is allowed to configure the relative weight of scoring (value weighting and normalization). There are additional ways to assess the adequacy of the item description. Some of these include checking the length of the description by counting characters and words, and comparing this to expected values (ranges) or existing calculated distributions. Such an evaluation may be performed on one or more fields by category, catalog, catalog set, or other group defined by the user.

Different applications and domains have different requirements for discovery and purchase. Preferably, the scoring system is configurable to reflect the value of each particular environment. Domain and application experts apply their own assessment of the relative importance of scoring the components of the catalog.

In the above description, three basic quality attributes are scored. The same method can be used to evaluate a number of other qualities of the database. The present invention is not limited to the quality and measurement described above.

After all of the desired qualities have been scored, the resulting report is constructed. The basic report is a summary description of the catalog. This includes the total number of items in the catalog (SKU), the number of unique items (total SKU-duplication), the number of categories, the number of basic and local attributes, and the number of unique local attributes.

The catalog grade is a weighted average of all the individual quality scores mentioned above. All grades use a scale of 0-10,
Preferably, 0 is the highest possible score. The user can define the weight assigned to each component, but standardized weight values are preferred to facilitate catalog comparison.

While the invention has been particularly shown and described with reference to preferred embodiments thereof, modifications, adaptations, and adaptations are possible without departing from the spirit and scope of the invention as defined by the following claims. Those skilled in the art will appreciate that and that modifications can be made therein.

[Brief description of the drawings]

FIG. 1 can implement one embodiment of the present invention;
FIG. 1 is a diagram of an example of a typical computer system.

FIG. 2 is a flowchart illustrating an embodiment of the present invention.

FIG. 3 is a flow chart illustrating an application of the present invention for measuring integrity.

FIG. 4 is a flow chart illustrating an application of the present invention for measuring consistency.

FIG. 5 is a flow chart showing an application example of the present invention for measuring the degree of completion.

[Explanation of symbols]

 1 Bus 2 Processor 4 Main Memory 6 Read Only Memory 7 Data Storage Device 21 Display Device 22 Alphanumeric Input Device 23 Cursor Control 25 Communication Device

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Michael Len Neal, Inventor, United States, 80027, Superior Riverbend, Colorado, Land 1622 (72) Greg Menin, United States, 80020, Bloomfield, Colorado, United States Dog leg lane 13893 F term (reference) 5B075 NK46 NR02 NR12 NR20 PP13 UU38

Claims (3)

[Claims] 1. A method of scoring a database for quality, wherein at least one field of the database is selected for analysis, and a value for each record of the database is analyzed. A method comprising: retrieving from the field; comparing the retrieved value to a standard; and assigning a score for each selected field based on the comparison. 2. A machine-readable medium having stored thereon data representing a sequence of instructions that, when executed by a processor, performs the following steps: selecting at least one field of a database for analysis; A machine comprising: retrieving a value for each record of a database from the analyzed field; comparing the retrieved value to a standard; and assigning a score for each selected field based on the comparison. Readable medium. 3. Ranking the selected fields for relevance to quality; weighting a score for each selected field based on each selected rank; Combining the weighted scores to obtain.