US10489360B2 - Specifying and applying rules to data - Google Patents

Specifying and applying rules to data Download PDF

Info

Publication number
US10489360B2
US10489360B2 US13/653,995 US201213653995A US10489360B2 US 10489360 B2 US10489360 B2 US 10489360B2 US 201213653995 A US201213653995 A US 201213653995A US 10489360 B2 US10489360 B2 US 10489360B2
Authority
US
United States
Prior art keywords
validation
cells
field
validation rule
rule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/653,995
Other languages
English (en)
Other versions
US20140108357A1 (en
Inventor
Roy Leonard Procops
Joel Gould
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ab Initio Technology LLC
Original Assignee
Ab Initio Technology LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ab Initio Technology LLC filed Critical Ab Initio Technology LLC
Priority to US13/653,995 priority Critical patent/US10489360B2/en
Assigned to AB INITIO TECHNOLOGY LLC reassignment AB INITIO TECHNOLOGY LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AB INITIO ORIGINAL WORKS LLC
Assigned to AB INITIO ORIGINAL WORKS LLC reassignment AB INITIO ORIGINAL WORKS LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AB INITIO SOFTWARE LLC
Assigned to AB INITIO SOFTWARE LLC reassignment AB INITIO SOFTWARE LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GOULD, JOEL, PROCOPS, Roy Leonard
Priority to JP2015537766A priority patent/JP6400588B2/ja
Priority to KR1020157011679A priority patent/KR102126099B1/ko
Priority to CA2887141A priority patent/CA2887141C/en
Priority to EP13786013.6A priority patent/EP2909750B1/en
Priority to AU2013331476A priority patent/AU2013331476B2/en
Priority to CN201380054286.5A priority patent/CN104737163A/zh
Priority to PCT/US2013/064979 priority patent/WO2014062638A1/en
Publication of US20140108357A1 publication Critical patent/US20140108357A1/en
Priority to HK15111897.4A priority patent/HK1211115A1/xx
Priority to US15/266,469 priority patent/US20170004151A1/en
Priority to US16/665,413 priority patent/US20200301897A1/en
Publication of US10489360B2 publication Critical patent/US10489360B2/en
Application granted granted Critical
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/248—Presentation of query results
    • G06F17/246—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
    • G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets

Definitions

  • This description relates to specifying and applying rules to data.
  • datasets which may be compiled from various sources.
  • the various sources that provide data to the dataset may have different levels of data quality.
  • an adequate level of data quality in the dataset should be maintained.
  • the dataset can be processed by a data validation system.
  • a data validation system applies validation rules to the dataset before it is provided to the application.
  • the data validation system uses the results of validation rules to calculate a measure of data quality and alert an administrator of the application if the measure of data quality falls below a predetermined threshold.
  • the data validation system includes modules for handling data that fails one or more of the validation rules. For example, the data validation system may discard or repair data that fails one or more of the validation rules.
  • validation rules applied by the data validation system are defined by an administrator of the data validation system.
  • a computing system specifies one or more validation rules for validating data included in one or more fields of each element of a plurality of elements of a dataset.
  • the computing system includes a user interface module configured to render a plurality of cells arranged in a two-dimensional grid having a first axis and a second axis.
  • the two-dimensional grid includes: one or more subsets of the cells extending in a direction along the first axis of the two-dimensional grid, each subset of the one or more subsets associated with a respective field of an element of the plurality of elements of the dataset, and multiple subsets of the cells extending in a direction along the second axis of the two-dimensional grid, one or more of the multiple subsets associated with a respective validation rule.
  • the computing system also includes a processing module configured to apply validation rules to at least one element of the dataset based on user input received from at least some of the cells.
  • at least some cells, associated with a field and a validation rule each include an input element for receiving input determining whether or not the associated validation rule is applied to the associated field.
  • at least some cells, associated with a field and a validation rule each include an indicator for indicating feedback associated with a validation result based on applying the associated validation rule to data included in the associated field of the element.
  • aspects can include one or more of the following features.
  • Applying validation rules to data included in a first field of a first element includes: determining any selected validation rules associated with cells from a subset of cells extending in the direction along the first axis associated with the first field of the first element, based on any input received in the input elements of the cells; and determining validation results for the data included in the first field of the first element based on the selected validation rules.
  • the one or more subsets of the cells extending in a direction along the first axis are rows of cells.
  • the multiple subsets of the cells extending in a direction along the second axis are columns of cells.
  • the input element is configured to receive input specifying one or more validation rule parameters.
  • One or more of the validation rules when evaluated yield a validation result of set of at least two validation results, the validation results including a result of valid and a result of invalid.
  • the indicator for indicating feedback included in at least some of the cells is configured to apply shading to a cell if the validation result is a result of invalid.
  • the input element is further configured to determine a correctness of each of the validation rule parameters.
  • the at least some cells associated with a field and a validation rule each include a second indicator for displaying a result of determining a correctness of the validation rule parameters associated with the cell.
  • the indicator for indicating feedback includes a numeric indicator which is configured to display a number of invalid results, the number of invalid results determined by applying the associated validation rule to data included in the associated field for all of the elements of the dataset.
  • the dataset includes one or more tables of a database and the elements of the dataset include database records.
  • One or more of the validation rules are user defined.
  • One or more of the validation rules are predefined.
  • One or more of the multiple subsets of the cells extending in the direction along the second axis of the two-dimensional grid includes a first cell associated with a first validation rule and a second cell associated with a second validation rule, the second validation rule different from the first validation rule.
  • One or more of the multiple subsets of the cells extending in the direction along the second axis of the two-dimensional grid includes a subset of cells that include an input element for receiving a value to replace an existing value in a corresponding field in response to a result of invalid for one of the validation rules applied to the existing value.
  • One or more of the multiple subsets of the cells extending in the direction along the second axis of the two-dimensional grid includes a subset of cells that include an input element for receiving an excluded value, such that the excluded value appearing in a corresponding field results in preventing validation rules from being applied to the existing value.
  • a computing system specifies one or more validation rules for validating data included in one or more fields of each element of a plurality of elements of a dataset.
  • the computing system includes means for rendering a plurality of cells arranged in a two-dimensional grid having a first axis and a second axis.
  • the two-dimensional grid includes: one or more subsets of the cells extending in a direction along the first axis of the two-dimensional grid, each subset of the one or more subsets associated with a respective field of an element of the plurality of elements of the dataset, and multiple subsets of the cells extending in a direction along the second axis of the two-dimensional grid, one or more of the multiple subsets associated with a respective validation rule.
  • the computing system also includes means for applying validation rules to at least one element of the dataset based on user input received from at least some of the cells.
  • at least some cells, associated with a field and a validation rule each include an input element for receiving input determining whether or not the associated validation rule is applied to the associated field.
  • at least some cells, associated with a field and a validation rule each include an indicator for indicating feedback associated with a validation result based on applying the associated validation rule to data included in the associated field of the element.
  • a method specifies one or more validation rules for validating data included in one or more fields of each element of a plurality of elements of a dataset.
  • the method includes: rendering, by a user interface module, a plurality of cells arranged in a two-dimensional grid having a first axis and a second axis.
  • the two-dimensional grid includes: one or more subsets of the cells extending in a direction along the first axis of the two-dimensional grid, each subset of the one or more subsets associated with a respective field of an element of the plurality of elements of the dataset, and multiple subsets of the cells extending in a direction along the second axis of the two-dimensional grid, one or more of the multiple subsets associated with a respective validation rule.
  • the method also includes applying, by at least one processor, validation rules to at least one element of the dataset based on user input received from at least some of the cells.
  • at least some cells, associated with a field and a validation rule each include an input element for receiving input determining whether or not the associated validation rule is applied to the associated field.
  • at least some cells, associated with a field and a validation rule each include an indicator for indicating feedback associated with a validation result based on applying the associated validation rule to data included in the associated field of the element.
  • a computer program stored on a computer-readable storage medium, specifies one or more validation rules for validating data included in one or more fields of each element of a plurality of elements of a dataset.
  • the computer program includes instructions for causing a computer system to render a plurality of cells arranged in a two-dimensional grid having a first axis and a second axis.
  • the two-dimensional grid includes: one or more subsets of the cells extending in a direction along the first axis of the two-dimensional grid, each subset of the one or more subsets associated with a respective field of an element of the plurality of elements of the dataset, and multiple subsets of the cells extending in a direction along the second axis of the two-dimensional grid, one or more of the multiple subsets associated with a respective validation rule.
  • the computer program also includes instructions for causing the computer system to apply validation rules to at least one element of the dataset based on user input received from at least some of the cells.
  • at least some cells, associated with a field and a validation rule each include an input element for receiving input determining whether or not the associated validation rule is applied to the associated field.
  • at least some cells, associated with a field and a validation rule each include an indicator for indicating feedback associated with a validation result based on applying the associated validation rule to data included in the associated field of the element.
  • the user interface can provide live feedback of the results of applying the rules to a single data element of a dataset as the rules are entered. In this way, the user can test the effectiveness of their rules without having to apply the rules to the entire dataset (a potentially time consuming process).
  • the user interface allows a user to run the specified rules over a dataset and receive feedback regarding the performance of each of the specified rules over the entire dataset. The user then has an opportunity to modify any of the specified rules that do not meet the expectations of the user.
  • the user interface allows a user to quickly and intuitively specify and modify rules, saving time and resources.
  • FIG. 1 is a block diagram of a system for specifying validation rules for validating data.
  • FIG. 2 is a user interface for specifying validation rules for validating data.
  • FIG. 3 is a screen capture of the user interface for specifying validation rules.
  • FIG. 1 shows an exemplary data processing system 100 in which the validation techniques can be used.
  • the system 100 includes a data source 102 that may include one or more sources of data such as storage devices or connections to online data streams, each of which may store data (sometimes referred to as a “dataset”) in any of a variety of storage formats (e.g., database tables, spreadsheet files, flat text files, or a native format used by a mainframe).
  • An execution environment 104 includes a user interface (UI) module 106 and a processing module 108 .
  • UI user interface
  • the UI module 106 manages input received from a user 110 over a user interface 112 (e.g., a graphical view on a display screen) for specifying validation rules to be used by the processing module 108 for processing data from the data source 102 .
  • a user interface 112 e.g., a graphical view on a display screen
  • the execution environment 104 may be hosted on one or more general-purpose computers under the control of a suitable operating system, such as the UNIX operating system.
  • the execution environment 104 can include a multiple-node parallel computing environment including a configuration of computer systems using multiple central processing units (CPUs), either local (e.g., multiprocessor systems such as SMP computers), or locally distributed (e.g., multiple processors coupled as clusters or MPPs), or remote, or remotely distributed (e.g., multiple processors coupled via a local area network (LAN) and/or wide-area network (WAN)), or any combination thereof.
  • CPUs central processing units
  • LAN local area network
  • WAN wide-area network
  • the processing module 108 reads data from the data source 102 and performs validation procedures based on validation information obtained by the UI module 106 .
  • Storage devices providing the data source 102 may be local to the execution environment 104 , for example, being stored on a storage medium connected to a computer running the execution environment 104 (e.g., hard drive 114 ), or may be remote to the execution environment 104 , for example, being hosted on a remote system (e.g., mainframe 116 ) in communication with a computer running the execution environment 104 , over a remote connection.
  • a remote system e.g., mainframe 116
  • a dataset accessed from the data source 102 includes a number of data elements (e.g., records formatted according to a predetermined record structure, or rows in a database table).
  • Each element of the number of data elements can include values for a number of fields (e.g., attributes defined within a record structure, or columns in a database table) (e.g., “first name,” “last name,” “email address,” etc.), possibly including null or empty values.
  • fields e.g., attributes defined within a record structure, or columns in a database table
  • e.g., “first name,” “last name,” “email address,” etc. possibly including null or empty values.
  • Various characteristics of values in the fields e.g., related to content or data type), or the presence or absence of values in certain fields, may be considered valid or invalid. For example, a “last name” field including the string “Smith” may be considered valid, while a “last name” field that is blank may be considered invalid.
  • the performance of an application that utilizes the dataset from the data source 102 may be adversely affected if the dataset includes a significant number of data elements with one or more invalid fields.
  • the processing module 108 performs data validation procedures, including applying data validation rules to the dataset, to ensure that the dataset meets a quality constraint defined by validation rules.
  • the data processing system 100 alerts a system administrator if the quality of the dataset fails to meet the quality constraint.
  • the processing module 108 may be configured to repair invalid data, if possible, or perform various data cleansing procedures to generate a dataset of cleansed data elements.
  • the processing module 108 may be configured to generate a list of fields that include invalid data from which reports can be generated.
  • the reports include a count of records that included invalid data for one or more of the fields in the list of fields.
  • aggregations of invalid fields are calculated from the list of fields.
  • the UI module 106 provides the user interface 112 , which enables a set of validation rules to be specified and used to validate the dataset.
  • the user interface 112 is able to provide a single view including multiple fields of a particular data element structure (in some implementations, all the available fields).
  • the user 110 e.g., a system administrator
  • the user 110 is able to specify appropriate validation rules for the data.
  • one example of the user interface 112 is configured to facilitate the user 110 specifying and verifying one or more validation rules for validating the dataset.
  • the UI module 106 renders the user interface 112 (e.g., on a computer monitor) including a number of cells 224 arranged in a two-dimensional grid 225 having a first axis 226 and a second axis, 228 .
  • One or more subsets 230 of the cells 224 i.e., referred to as rows 230 in the remainder of the detailed description
  • rows 230 extends in a direction along the first axis 226 of the two-dimensional grid 225 .
  • Each of the rows 230 is associated with a field 218 .
  • the first (i.e., leftmost) cell of each of the rows 230 includes the name of the field 218 associated with the row 230 (in this example, the field names are “Field 1 ,” “Field 2 ,” . . . “Field M”).
  • Multiple subsets 232 of the cells 224 extend in a direction along the second axis 228 of the two-dimensional grid 225 .
  • One or more of the columns 232 is associated with a respective validation rule 234 .
  • the first (i.e., the topmost) cell of each of the columns 232 includes the name of the validation rule 234 associated with the column 232 (in this example, the validation rule names are “Validation Rule 1 ,” “Validation Rule 2 ,” . . . “Validation Rule N”). It is noted that in some examples, the directions of the first axis 226 and the second axis 228 can be swapped, causing the rows 230 associated with the fields 218 to become columns and the columns 232 associated with the validation rules 234 to become rows.
  • the user interface 112 includes a list (not shown) of predefined validation rules.
  • the validation rules 234 are added to the two-dimensional grid 225 , for example, by the user 110 dragging one or more of the pre-defined validation rules into the two-dimensional grid 225 , or double-clicking one of the pre-defined validation rules, resulting in one or more new columns 232 being added to the grid 225 .
  • the pre-defined validation rules have a built-in function, which may accept a pre-defined set of parameters as input that can be provided within a corresponding cell. For many situations, the pre-defined list of validation rules is sufficient for the user's 110 needs. However, in some examples, as is described below, the user 110 can define custom validation rules which can also be added as columns 232 to the two-dimensional grid 225 .
  • the user 110 can specify which validation rules 234 should be applied to which fields 218 .
  • the user 110 first selects a cell 224 where the row 230 associated with the given field 218 intersects with the column 232 associated with the given validation rule 234 .
  • the user 110 then enters one or more validation rule parameters 236 in an input element (e.g., a text field or check box) of the selected cell 224 .
  • an input element e.g., a text field or check box
  • the inclusion of a rule parameter 236 in a cell potentially serves two purposes.
  • the first purpose is to provide “configuration input” to configure the validation rule 234
  • the second purpose is to indicate that the given validation rule 234 should be applied to the given field 218 . It follows that if a cell 224 does not include validation rule parameters 236 (i.e., the cell is left blank), the processing module 108 does not apply the validation rule 234 associated with the cell 224 to the field 218 associated with the cell 224 .
  • rule parameters 236 can be entered in to the cells 224 .
  • the rule parameter 236 may simply be a “confirmation input” rule parameter that confirms that a corresponding validation rule is to be applied.
  • an input element for receiving a confirmation input rule parameter is a checkbox which, when checked, indicates that the validation rule 234 associated with a cell 224 should be applied to the field 218 associated with the cell 224 .
  • Examples of various types of validation rules are presented in the following list, which indicates whether or not the validation rule is configured by configuration input:
  • the UI module 106 provides feedback to the user 110 through the user interface 112 by displaying results of the processing module 108 applying the user-specified validation rules 234 to at least some of the elements of the dataset.
  • the user interface 112 shown in FIG. 2 is configured to display the values 242 of the fields 218 for a given element 244 of the dataset.
  • the processing module 108 automatically applies the specified validation rules 234 to the values 242 of the fields 218 of the given data element 244 and provides the results of applying the validation rules 234 to the UI module 106 , which in turn presents the results in the user interface 112 as feedback to the user 110 .
  • the result of applying a validation rule is a pass/fail result.
  • Such a pass/fail result can be indicated to the user 110 by, for example, filling the appropriate cell with a certain color, pattern, or shading.
  • the cell associated with field 1 and validation rule 1 includes gray shading 238 , indicating that the value of field 1 failed validation rule 1 .
  • a pass/fail result can be indicated to the user 100 by the inclusion/exclusion of an indicator icon in the appropriate cell.
  • a failing result can be indicated by including a red exclamation point icon in the cell and a passing result can be indicated by the absence of the red exclamation point icon.
  • an icon such as a green circle can be included in the cell to indicate a passing result.
  • the user interface 112 includes a control 246 which allows the user to select different elements of the dataset (in this example, by entering a sequence number). As the user navigates from one element to the next, the processing module 108 automatically applies the validation rules 234 to the currently selected element.
  • the user interface 112 includes a run control 248 , which permits the processing module 108 to apply the specified validation rules 234 to all of the elements of the dataset.
  • the processing module 108 Upon completion of applying the validation rules 234 to the dataset, the processing module 108 provides the results of applying the validation rules 234 to the dataset to the UI module 106 , which in turn displays the results in the user interface 112 to the user 110 .
  • each cell 234 associated with a validation rule 234 that was applied includes a failed result count indicator 240 .
  • the failed result count indicator 240 displays the number of data elements that failed the validation rule 234 specified by the cell 224 .
  • the user 110 may desire a validation rule with functionality that is not included in any of the pre-defined validation rules.
  • the user interface 112 includes an option for inserting one or more mixed validation rule columns into the two-dimensional grid 225 .
  • a mixed validation rule column allows the user 110 to specify a different validation rule for each cell (associated with a given field 218 ) included in the column.
  • one cell of the mixed validation rule column could include a ‘Valid Values’ test while another cell of the mixed validation rule column could include a ‘Maximum’ test.
  • the user 100 specifies a validation rule for a given cell of the mixed validation rule column by entering the name of the test followed by the rule parameters for the test (if the test accepts rule parameters).
  • any validation rule which can be added to the two-dimensional grid 225 as a column can be entered into a single cell of a mixed validation rule column.
  • Some examples of the contents of cells of the mixed validation rule column are “Not Null,” “Maximum(99),” and “Valid Values(VM,F).”
  • the mixed validation rule column can avoid a situation where a “Valid Email” test applies only to a single field 218 (e.g., an ‘email_addr’ field) but occupies an entire column 232 of the two-dimensional grid 225 , thereby wasting valuable screen real estate.
  • the user 110 can augment the list of pre-defined validation rules with a new, reusable, custom validation rule 234 .
  • the user interface 112 provides a template for the user 110 to define the functionality of the new validation rule 234 .
  • the user 110 defines the desired custom functionality within the bounds of the template using, for example, a programming language or an expression language, for example DML code decorated with structured comments.
  • the validation rule 234 is added to the list of pre-defined validation rules.
  • the user 110 can later use the new custom validation rule 234 , for example, by dragging the validation rule from the list of validation rules into the two-dimensional grid 225 or by double-clicking the validation rule.
  • dragging the new validation rule into the grid 225 or double-clicking the new validation rule causes a new column 232 to be added to the grid 225 , the new column 232 associated with the new validation rule.
  • Validation rules may have an attribute indicating whether the rule should be applied to null values or blank values. If the rules specifies it should not be applied to null values, the value is first tested for null, and then if null the rule is not applied, or if not null the rule is applied. If the rule specifies it should not be applied to blank values, the value is first tested to see if it is blank, and the rule is only applied if the value was found to be not blank.
  • Validation rules may have attributes indicating logic that can be used to determine the whether a set of rule parameters 236 entered in a cell 224 are valid for the validation rule.
  • the user interface 112 uses this logic to determine the correctness of each set of rule parameters 236 entered in a cell 224 , and if the rule parameters are determined to be incorrect (e.g., due to a syntax error), and an indicator (for example a red stop sign) is displayed in the cell, and an error message determined by the logic is displayed (for example in a list of errors, or as a hover tooltip when hovering over the cell).
  • Another example of checking the correctness of a rule parameter is checking semantics, such as checking that a specified lookup file identifier has in fact been made known to the processing module 108 .
  • the user interface 112 may include a pre-processing column, which can be used to apply any initial processing to values in a field, or to specify any particular values to be handled differently by validation rules of other columns.
  • the user interface 112 may also include a post-processing column, which can be used to apply any actions in response to results of a test performed by a validation rule.
  • a pre-processing column can be used, for example, to allow the user 110 to specify values to be excluded from validation, and validation data types for one or more of the fields 218 .
  • a post-processing column can be used, for example, to allow the user 110 to specify replacement values to replace existing values in an element (e.g., to replace different types of invalid values with appropriate replacement values).
  • a replacement value is entered into a single cell of the post-processing column and is associated with a given field 218 .
  • the replacement value replaces the value 242 of the given field 218 when one or more validation rules 236 associated with the given field 218 fails. For example, if a ‘start_date’ field is associated with two validation rules, Minimum(1900-01-01) and Maximum(2011-12-31), one example of a replacement value is 1970-01-01. Thus, if the value of the ‘start_date’ field for a given record is below the minimum (i.e., before 1900-01-01) or above the maximum (i.e., later than 2011-12-31), the value is replaced with the replacement value, 1970-01-01.
  • Other types of replacement values such as strings, date/times, etc. can also be specified in the post-processing column.
  • the user 110 can also specify one or more values to be excluded from validation in an excluded value type pre-processing column.
  • valid data for a field such as ‘end_date’ generally includes only date information (e.g., 1900-01-01).
  • another value such as “ACTIVE” is also valid data for the ‘end_date’ field. This can be done by entering the string “ACTIVE” into the excluded value type pre-processing column, indicating that the value “ACTIVE” is always allowable for the ‘start_date’ field and that the validation rules do not need to be applied to the specified excluded value.
  • a pre-processing column can also include a validation type column that specifies a validation data type for one or more of the fields 218 .
  • the user 110 can enter a DML type declaration which is used to validate a field. For example, if a field 218 includes a string value that represents a date, the user 110 can enter DATE(‘YYYY-MM-DD’) so specify that the string value actually represents a date data type and therefore should be validated as such. Similarly, to validate a string as a decimal number, the user 110 can enter decimal(‘ ’).
  • a screen capture illustrates one implementation of the user interface 112 of FIG. 2 .
  • the user interface 112 is configured to allow a user 110 to specify validation rules 234 for a dataset while receiving validation rule feedback.
  • the user interface 112 includes a two-dimensional grid 225 of cells 224 .
  • the grid 225 includes a number of rows 230 associated with fields 218 of the data elements of the dataset.
  • the first cell of each of the rows 230 includes the name of the field 218 associated with the row 230 and, in parentheses, the value 242 of the field 218 for a currently selected data element 244 of the dataset.
  • Other information about the field can also be displayed visually, to aid in a user specifying validation rules.
  • the first cell also includes an icon 220 that visually indicates a data type of the values of the field 218 .
  • the user 110 has added a number of validation rules 234 to the grid 225 .
  • the validation rules 234 appear in the grid as a number of columns 232 .
  • the name of each validation rule 234 is included at the top of the column 232 associated with the validation rule 234 (e.g., “Maximum Length,” “Not Blank,” “Pattern,” etc.).
  • the user 110 has specified that selected validation rules 234 should be applied to one or more fields 218 of the elements of the dataset. To do so, for each validation rule 234 to be applied, the user 110 has entered a rule parameter 236 at the intersection of the column 232 associated with the validation rule 234 and the row(s) 230 associated with the field(s) 218 to which the validation rule 234 should be applied. For example, the user 110 has entered the rule parameter S“99999” at the intersection of the “Pattern” validation rule and the ‘zipcode’ field.
  • the entered rule parameter configures the “Pattern” validation rule to evaluate the ‘zipcode’ field of each element of the dataset to determine if the value of the ‘zipcode’ field of each of the elements is a string with a pattern of five consecutive numeric characters.
  • the “Pattern” validation rule is configured to evaluate the ‘phonenum’ field of each element of the dataset to determine if the value 242 of the ‘phonenum’ field of each element is a string with a pattern of S“999-999-9999” (i.e., three numeric characters, a dash, three more numeric characters, a dash, and four more numeric characters).
  • validation rules 234 and rule parameters are also illustrated in FIG. 3 .
  • a “Valid Values” validation rule is applied to the ‘statename’ field with a rule parameter of M“StateNames” which identifies the valid values for the ‘statename’ field as the set of state names for the United States of America.
  • M“StateNames” in the rule parameter above indicates that the set of state names is defined (e.g., by the user 110 or a system administrator) as a separate dataset (sometimes referred to as a codeset), which is stored in a metadata reference system that is accessible in the execution environment 104 .
  • the dataset including the state names is referred to by the variable name “StateNames.”
  • a codeset is stored in a lookup table.
  • the rule parameter is entered as, for example, L“StateNames” indicating that a lookup file identified to the system with the name “StateNames” is the source of valid ‘statename’ values.
  • the user 110 can directly enter the set of valid values.
  • the valid set of gender codes can be entered as V“M,F,U”.
  • Another, “Not Blank,” validation rule is applied to a number of the fields.
  • the “Not Blank” validation rule is applied to the ‘street’ field due to the presence of a check mark rule parameter in the cell at the intersection of the “Not Blank” rule parameter column and the ‘street’ field row.
  • the user interface 112 is able to display all of the values 242 of the fields 218 for a given element 244 to the user 110 .
  • the UI module 106 also receives input from the user interface 112 that causes the processing module 108 to execute some or all of the validation rules 234 associated with the fields 218 of the element 244 .
  • the result(s) generated by the processing module 108 are provided to the UI module 106 , which in turn displays feedback based on the result(s) to the user 110 in the user interface 112 .
  • the “Valid Values” validation rule is applied to the ‘statename’ field to test whether the value of the ‘statename’ field is a member of the set of state names.
  • the “Valid Values” validation rule fails for the ‘statename’ field for the given element 244 .
  • the cell associated with the “Valid Values” validation rule and the ‘statename’ field is shaded.
  • the user 110 can navigate through the elements of the dataset using a navigation control 246 .
  • navigation control 246 includes arrows, which allow the user 110 to step through the elements of the dataset one at a time, and a numeric field, which allows the user 110 to enter a dataset element number that they would like to view.
  • the processing module 108 executes the specified validation rules on the values of the new element, and the values 242 and other visual feedback indicating results of the validation tests (for example shading of cells) are refreshed/updated.
  • the user interface 112 also includes a ‘Test’ button 248 which, when actuated, causes the processing module 108 to execute the specified validation rules for all of the elements of the dataset.
  • a ‘Test’ button 248 which, when actuated, causes the processing module 108 to execute the specified validation rules for all of the elements of the dataset.
  • the results of executing the specified validation rules for all of the elements of the dataset are summarized in the user interface 112 by the inclusion of a failed element count indicator 240 in each cell for which one or more elements have failed the specified validation rule.
  • the failed element count indicator 240 is a number that represents the number of elements of the dataset that failed the validation rule specified by the cell.
  • the failed element count indicator for the cell associated with the ‘statename’ field and the “Valid Values” validation rule indicates that 3886 of the elements of the dataset include a state name that is not a member of the set of valid state names. A user can click on that cell to retrieve information about elements that failed.
  • a collection of issue information can be aggregated over the validation issues and stored for later retrieval. For example, a list of fields for which one or more validation rules were specified can be displayed in another view, with counts of number of elements that had a validation issue for that field, including a count of zero elements if there were no validation issues for that field. This enables a user to unambiguously determine that no elements failed that particular validation rule, while also confirming that the validation rules for that field were actually performed.
  • Stored validation issue information can also be used to compute various metrics (e.g., percentages of records that have particular quality issues), or to augment a dataset of data elements with validation issue information.
  • the failed result count indicator 240 is a hyperlink which, when clicked by the user 110 , causes the UI module 106 to display a window that summarizes all of the failed elements to the user 110 .
  • the result of applying data validation rules can be used to determine metrics of the dataset.
  • metrics can include the percentage of records of the dataset which have data quality issues.
  • Other user interfaces which are not described herein can be used to specify and present these metrics to the user 110 .
  • test dataset that has a reduced and more manageable size and is representative of a full dataset can be used.
  • the techniques described above can be implemented using software for execution on a computer.
  • the software forms procedures in one or more computer programs that execute on one or more programmed or programmable computer systems (which may be of various architectures such as distributed, client/server, or grid) each including at least one processor, at least one data storage system (including volatile and non-volatile memory and/or storage elements), at least one input device or port, and at least one output device or port.
  • the software may form one or more modules of a larger program, for example, that provides other services related to the design and configuration of dataflow graphs.
  • the nodes and elements of the graph can be implemented as data structures stored in a computer readable medium or other organized data conforming to a data model stored in a data repository.
  • the software may be provided on a storage medium, such as a CD-ROM, readable by a general or special purpose programmable computer, or delivered (encoded in a propagated signal) over a communication medium of a network to a storage medium of the computer where it is executed. All of the functions may be performed on a special purpose computer, or using special-purpose hardware, such as coprocessors.
  • the software may be implemented in a distributed manner in which different parts of the computation specified by the software are performed by different computers.
  • Each such computer program is preferably stored on or downloaded to a storage media or device (e.g., solid state memory or media, or magnetic or optical media) readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer system to perform the procedures described herein.
  • a storage media or device e.g., solid state memory or media, or magnetic or optical media
  • the inventive system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer system to operate in a specific and predefined manner to perform the functions described herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Artificial Intelligence (AREA)
  • User Interface Of Digital Computer (AREA)
  • Debugging And Monitoring (AREA)
  • Stored Programmes (AREA)
US13/653,995 2012-10-17 2012-10-17 Specifying and applying rules to data Active 2033-12-08 US10489360B2 (en)

Priority Applications (11)

Application Number Priority Date Filing Date Title
US13/653,995 US10489360B2 (en) 2012-10-17 2012-10-17 Specifying and applying rules to data
CN201380054286.5A CN104737163A (zh) 2012-10-17 2013-10-15 指定规则并将规则应用于数据
KR1020157011679A KR102126099B1 (ko) 2012-10-17 2013-10-15 데이터에 규칙 지정 및 적용
JP2015537766A JP6400588B2 (ja) 2012-10-17 2013-10-15 データに対する規則の指定及び適用
PCT/US2013/064979 WO2014062638A1 (en) 2012-10-17 2013-10-15 Specifying and applying rules to data
CA2887141A CA2887141C (en) 2012-10-17 2013-10-15 Specifying and applying rules to data
EP13786013.6A EP2909750B1 (en) 2012-10-17 2013-10-15 Specifying and applying rules to data
AU2013331476A AU2013331476B2 (en) 2012-10-17 2013-10-15 Specifying and applying rules to data
HK15111897.4A HK1211115A1 (en) 2012-10-17 2015-12-03 Specifying and applying rules to data
US15/266,469 US20170004151A1 (en) 2012-10-17 2016-09-15 Specifying and applying rules to data
US16/665,413 US20200301897A1 (en) 2012-10-17 2019-10-28 Specifying and applying rules to data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/653,995 US10489360B2 (en) 2012-10-17 2012-10-17 Specifying and applying rules to data

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US15/266,469 Continuation US20170004151A1 (en) 2012-10-17 2016-09-15 Specifying and applying rules to data
US16/665,413 Continuation US20200301897A1 (en) 2012-10-17 2019-10-28 Specifying and applying rules to data

Publications (2)

Publication Number Publication Date
US20140108357A1 US20140108357A1 (en) 2014-04-17
US10489360B2 true US10489360B2 (en) 2019-11-26

Family

ID=49517666

Family Applications (3)

Application Number Title Priority Date Filing Date
US13/653,995 Active 2033-12-08 US10489360B2 (en) 2012-10-17 2012-10-17 Specifying and applying rules to data
US15/266,469 Abandoned US20170004151A1 (en) 2012-10-17 2016-09-15 Specifying and applying rules to data
US16/665,413 Abandoned US20200301897A1 (en) 2012-10-17 2019-10-28 Specifying and applying rules to data

Family Applications After (2)

Application Number Title Priority Date Filing Date
US15/266,469 Abandoned US20170004151A1 (en) 2012-10-17 2016-09-15 Specifying and applying rules to data
US16/665,413 Abandoned US20200301897A1 (en) 2012-10-17 2019-10-28 Specifying and applying rules to data

Country Status (9)

Country Link
US (3) US10489360B2 (ko)
EP (1) EP2909750B1 (ko)
JP (1) JP6400588B2 (ko)
KR (1) KR102126099B1 (ko)
CN (1) CN104737163A (ko)
AU (1) AU2013331476B2 (ko)
CA (1) CA2887141C (ko)
HK (1) HK1211115A1 (ko)
WO (1) WO2014062638A1 (ko)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10769122B2 (en) 2014-03-13 2020-09-08 Ab Initio Technology Llc Specifying and applying logical validation rules to data
US20200380212A1 (en) * 2019-05-31 2020-12-03 Ab Initio Technology Llc Discovering a semantic meaning of data fields from profile data of the data fields
US11249981B2 (en) 2015-06-12 2022-02-15 Ab Initio Technology Llc Data quality analysis
US11886399B2 (en) 2020-02-26 2024-01-30 Ab Initio Technology Llc Generating rules for data processing values of data fields from semantic labels of the data fields

Families Citing this family (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
AU2009322602B2 (en) 2008-12-02 2015-06-25 Ab Initio Technology Llc Mapping instances of a dataset within a data management system
WO2012061109A1 (en) 2010-10-25 2012-05-10 Ab Initio Technology Llc Managing data set objects in a dataflow graph that represents a computer program
CA2824282C (en) 2011-01-14 2019-05-21 Ab Initio Technology Llc Managing changes to collections of data
US20140101117A1 (en) * 2012-10-10 2014-04-10 Salesforce.Com, Inc. Methods and systems for managing records in an on-demand system
US10489360B2 (en) 2012-10-17 2019-11-26 Ab Initio Technology Llc Specifying and applying rules to data
KR20240132105A (ko) 2013-02-07 2024-09-02 애플 인크. 디지털 어시스턴트를 위한 음성 트리거
GB201409214D0 (en) * 2014-05-23 2014-07-09 Ibm A method and system for processing a data set
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
JP6457622B2 (ja) 2014-07-18 2019-01-23 アビニシオ テクノロジー エルエルシー パラメータの集合の管理
US9626393B2 (en) * 2014-09-10 2017-04-18 Ab Initio Technology Llc Conditional validation rules
US10191924B2 (en) * 2014-10-20 2019-01-29 Ab Initio Technology Llc Specifying and applying rules to data
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
CN106802898B (zh) * 2015-11-26 2020-09-04 北大方正集团有限公司 数据录入方法及装置
KR102432104B1 (ko) 2016-11-09 2022-08-11 아브 이니티오 테크놀로지 엘엘시 데이터 요소 간의 관계를 결정하기 위한 시스템 및 방법
US11126599B2 (en) * 2017-01-24 2021-09-21 Accenture Global Solutions Limited Information validation method and system
DK180048B1 (en) 2017-05-11 2020-02-04 Apple Inc. MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. LOW-LATENCY INTELLIGENT AUTOMATED ASSISTANT
US20180336275A1 (en) 2017-05-16 2018-11-22 Apple Inc. Intelligent automated assistant for media exploration
CN107491430B (zh) * 2017-08-28 2021-01-08 南京农纷期电子商务有限公司 利用pdf工具包和SPEL表达式填充pdf的方法
KR102034679B1 (ko) * 2018-01-17 2019-10-23 (주)비아이매트릭스 그리드 인터페이스 기반 데이터 입출력 시스템
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. VIRTUAL ASSISTANT OPERATION IN MULTI-DEVICE ENVIRONMENTS
CN108897728B (zh) * 2018-06-27 2023-04-07 平安科技（深圳）有限公司 短信拼接处理方法、装置、计算机设备和存储介质
CA3106682A1 (en) * 2018-07-19 2020-01-23 Ab Initio Technology Llc Publishing to a data warehouse
CN110889106B (zh) * 2018-09-11 2024-08-16 京东科技控股股份有限公司 配置方法、装置、系统、计算机可读存储介质
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11243972B1 (en) * 2018-12-28 2022-02-08 Lumeris Solutions Company, LLC Data validation system
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
US20200380389A1 (en) * 2019-05-31 2020-12-03 Apple Inc. Sentiment and intent analysis for customizing suggestions using user-specific information
US11227599B2 (en) 2019-06-01 2022-01-18 Apple Inc. Methods and user interfaces for voice-based control of electronic devices
KR102240496B1 (ko) * 2020-04-17 2021-04-15 주식회사 한국정보기술단 데이터 품질 관리 시스템 및 그 방법
US11061543B1 (en) 2020-05-11 2021-07-13 Apple Inc. Providing relevant data items based on context
US11490204B2 (en) 2020-07-20 2022-11-01 Apple Inc. Multi-device audio adjustment coordination
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones
WO2022165321A1 (en) 2021-01-31 2022-08-04 Ab Initio Technology Llc Data processing system with manipulation of logical dataset groups
CN113190829B (zh) * 2021-05-18 2024-04-09 京东科技信息技术有限公司 一种权限数据的识别方法、装置、设备和介质
CN113468037A (zh) * 2021-07-26 2021-10-01 浙江大华技术股份有限公司 一种数据质量评估方法、装置、介质和电子设备
CN113642311B (zh) * 2021-08-12 2023-09-05 北京奇艺世纪科技有限公司 一种数据对比方法、装置、电子设备及存储介质

Citations (117)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05507376A (ja) 1990-05-30 1993-10-21 アレン　―　ブラッドリィ　カンパニー，インコーポレーテッド 機械コントローラ／プロセスコントローラ用グラフィック・プログラム・インタフェース
JPH0744368A (ja) 1993-07-29 1995-02-14 Hitachi Ltd 組合せモデルの編集システム
US5630127A (en) 1992-05-15 1997-05-13 International Business Machines Corporation Program storage device and computer program product for managing an event driven management information system with rule-based application structure stored in a relational database
US5758351A (en) 1995-03-01 1998-05-26 Sterling Software, Inc. System and method for the creation and use of surrogate information system objects
JPH11143755A (ja) 1997-09-17 1999-05-28 Hewlett Packard Co <Hp> 階層構成情報のコピーおよび共有方法
US5966072A (en) 1996-07-02 1999-10-12 Ab Initio Software Corporation Executing computations expressed as graphs
US6088702A (en) 1998-02-25 2000-07-11 Plantz; Scott H. Group publishing system
US20010007959A1 (en) * 2000-01-07 2001-07-12 Samuel Abdalla Tabular range editing mechanism
US20010014890A1 (en) 1998-02-06 2001-08-16 Gwoho Liu Methods for mapping data fields from one data set to another in a data processing environment
JP2002279147A (ja) 2001-03-22 2002-09-27 Sharp Corp 内製化判断支援装置、内製化判断支援方法および内製化判断支援プログラムを記録した機械読取可能な記録媒体および内製化判断支援プログラム
US20020161799A1 (en) * 2001-02-27 2002-10-31 Microsoft Corporation Spreadsheet error checker
US6494159B2 (en) 2001-05-11 2002-12-17 The United States Of America As Represented By The Secretary Of The Navy Submarine launched unmanned combat vehicle replenishment
US20030016246A1 (en) 2001-07-18 2003-01-23 Sanjai Singh Graphical subclassing
US20030163597A1 (en) 2001-05-25 2003-08-28 Hellman Ziv Zalman Method and system for collaborative ontology modeling
US20030163441A1 (en) * 2002-02-22 2003-08-28 Niels Godfredsen Method and system for navigating a large amount of data
US6633875B2 (en) 1999-12-30 2003-10-14 Shaun Michael Brady Computer database system and method for collecting and reporting real estate property and loan performance information over a computer driven network
US20040015783A1 (en) 2002-06-20 2004-01-22 Canon Kabushiki Kaisha Methods for interactively defining transforms and for generating queries by manipulating existing query data
US20040024740A1 (en) 2001-01-16 2004-02-05 Mcgeorge Vernon E. Method and system for validating data submitted to a database application
US6708186B1 (en) 2000-08-14 2004-03-16 Oracle International Corporation Aggregating and manipulating dictionary metadata in a database system
US20040056908A1 (en) 2001-03-22 2004-03-25 Turbo Worx, Inc. Method and system for dataflow creation and execution
US20040225632A1 (en) 2003-05-08 2004-11-11 Microsoft Corporation Automated information management and related methods
US20040239681A1 (en) 2000-08-07 2004-12-02 Zframe, Inc. Visual content browsing using rasterized representations
US6832366B2 (en) 2001-05-17 2004-12-14 Simdesk Technologies, Inc. Application generator
US20050010896A1 (en) 2003-07-07 2005-01-13 International Business Machines Corporation Universal format transformation between relational database management systems and extensible markup language using XML relational transformation
US20050060317A1 (en) * 2003-09-12 2005-03-17 Lott Christopher Martin Method and system for the specification of interface definitions and business rules and automatic generation of message validation and transformation software
US20050060313A1 (en) * 2003-09-15 2005-03-17 Oracle International Corporation A California Corporation Data quality analyzer
US20050114369A1 (en) 2003-09-15 2005-05-26 Joel Gould Data profiling
US20050144189A1 (en) 2002-07-19 2005-06-30 Keay Edwards Electronic item management and archival system and method of operating the same
US20050178833A1 (en) 2001-12-20 2005-08-18 Canon Information Systems Research Australia Pty Microprocessor card defining a custom user interface
US20050187984A1 (en) 2004-02-20 2005-08-25 Tianlong Chen Data driven database management system and method
US6948154B1 (en) * 1999-03-22 2005-09-20 Oregon State University Methodology for testing spreadsheets
US20050234762A1 (en) 2004-04-16 2005-10-20 Pinto Stephen K Dimension reduction in predictive model development
US20050262121A1 (en) * 1999-09-21 2005-11-24 International Business Machines Corporation Method, system, program, and data structure for cleaning a database table
US20060007464A1 (en) 2004-06-30 2006-01-12 Percey Michael F Structured data update and transformation system
US20060020570A1 (en) 2004-07-23 2006-01-26 Yuh-Cherng Wu Conflict resolution engine
JP2006113810A (ja) 2004-10-14 2006-04-27 Fujitsu Ltd 書誌情報入力支援プログラム
US20060095466A1 (en) 2004-11-02 2006-05-04 Daniell Stevens Managing related data objects
US7080088B1 (en) 2002-01-30 2006-07-18 Oracle International Corporation Automatic reconciliation of bindable objects
US20060200739A1 (en) 2005-03-07 2006-09-07 Rishi Bhatia System and method for data manipulation
US7110924B2 (en) 2002-05-15 2006-09-19 Caterpillar Inc. Method for controlling the performance of a target system
JP2006277624A (ja) 2005-03-30 2006-10-12 Ns Engineering Corp データ照合装置及び方法
US7143076B2 (en) 2000-12-12 2006-11-28 Sap Aktiengesellschaft Method and apparatus for transforming data
US20070011208A1 (en) 2005-07-06 2007-01-11 Smith Alan R Apparatus, system, and method for performing semi-automatic dataset maintenance
US7164422B1 (en) 2000-07-28 2007-01-16 Ab Initio Software Corporation Parameterized graphs with conditional components
US7167850B2 (en) 2002-10-10 2007-01-23 Ab Initio Software Corporation Startup and control of graph-based computation
US20070027858A1 (en) 2005-07-29 2007-02-01 Paul Weinberg Method for generating properly formed expressions
US7185317B2 (en) 2002-02-14 2007-02-27 Hubbard & Wells Logical data modeling and integrated application framework
US20070050750A1 (en) 2005-08-31 2007-03-01 Microsoft Corporation Extensible data-driven setup application for operating system
US20070080088A1 (en) 2005-10-06 2007-04-12 Trotter Jason K Preassembly of cooperating parts
US20070094060A1 (en) 2005-10-25 2007-04-26 Angoss Software Corporation Strategy trees for data mining
US20070136692A1 (en) 2005-12-09 2007-06-14 Eric Seymour Enhanced visual feedback of interactions with user interface
US20070179956A1 (en) 2006-01-18 2007-08-02 Whitmyer Wesley W Jr Record protection system for networked databases
US7257603B2 (en) 2003-05-08 2007-08-14 Microsoft Corporation Preview mode
US20070198457A1 (en) 2006-02-06 2007-08-23 Microsoft Corporation Accessing and manipulating data in a data flow graph
US20070226203A1 (en) 2006-03-23 2007-09-27 Microsoft Corporation Generation of query and update views for object relational mapping
US20070239751A1 (en) 2006-03-31 2007-10-11 Sap Ag Generic database manipulator
US20070271381A1 (en) 2006-05-16 2007-11-22 Joseph Skeffington Wholey Managing computing resources in graph-based computations
US20070276787A1 (en) 2006-05-15 2007-11-29 Piedmonte Christopher M Systems and Methods for Data Model Mapping
US20070294119A1 (en) 2006-03-30 2007-12-20 Adaptive Alpha, Llc System, method and computer program product for evaluating and rating an asset management business and associate investment funds using experiential business process and performance data, and applications thereof
US7328428B2 (en) 2003-09-23 2008-02-05 Trivergent Technologies, Inc. System and method for generating data validation rules
US20080049022A1 (en) 2006-08-10 2008-02-28 Ab Initio Software Corporation Distributing Services in Graph-Based Computations
US20080126988A1 (en) 2006-11-24 2008-05-29 Jayprakash Mudaliar Application management tool
US20080162384A1 (en) 2006-12-28 2008-07-03 Privacy Networks, Inc. Statistical Heuristic Classification
US20080228697A1 (en) 2007-03-16 2008-09-18 Microsoft Corporation View maintenance rules for an update pipeline of an object-relational mapping (ORM) platform
US20080243772A1 (en) 2007-03-29 2008-10-02 Ariel Fuxman Method and sytsem for generating nested mapping specifications in a schema mapping formalism and for generating transformation queries based thereon
US20080243891A1 (en) 2007-03-30 2008-10-02 Fmr Corp. Mapping Data on a Network
US20080256014A1 (en) 2007-04-10 2008-10-16 Joel Gould Editing and Compiling Business Rules
US20080313204A1 (en) 2007-06-14 2008-12-18 Colorquick, L.L.C. Method and apparatus for database mapping
US20080312979A1 (en) 2007-06-13 2008-12-18 International Business Machines Corporation Method and system for estimating financial benefits of packaged application service projects
JP2008547134A (ja) 2005-06-27 2008-12-25 エービー イニティオ ソフトウェア コーポレーション グラフ型計算のためのパラメータ管理
US20090083313A1 (en) 2007-09-20 2009-03-26 Stanfill Craig W Managing Data Flows in Graph-Based Computations
US20090094291A1 (en) 2007-09-14 2009-04-09 Oracle International Corporation Support for compensation aware data types in relational database systems
US7536406B2 (en) 2004-06-23 2009-05-19 Microsoft Corporation Impact analysis in an object model
US7577724B1 (en) 2006-03-28 2009-08-18 Emc Corporation Methods and apparatus associated with advisory generation
US20090234623A1 (en) 2008-03-12 2009-09-17 Schlumberger Technology Corporation Validating field data
US20090319494A1 (en) 2008-06-20 2009-12-24 Microsoft Corporation Field mapping for data stream output
US20090327196A1 (en) 2008-06-30 2009-12-31 Ab Initio Software Llc Data Logging in Graph-Based Computations
US7661067B2 (en) 2006-02-21 2010-02-09 International Business Machines Corporation Method for providing quick responses in instant messaging conversations
US7689565B1 (en) 2006-06-28 2010-03-30 Emc Corporation Methods and apparatus for synchronizing network management data
US20100100220A1 (en) 2005-06-09 2010-04-22 Belanger David G Arrangement for guiding user design of comprehensive product solution using on-the-fly data validation
US20100114833A1 (en) 2008-10-31 2010-05-06 Netapp, Inc. Remote office duplication
US20100121890A1 (en) 2008-11-12 2010-05-13 Ab Initio Software Llc Managing and automatically linking data objects
US20100138388A1 (en) 2008-12-02 2010-06-03 Ab Initio Software Llc Mapping instances of a dataset within a data management system
US20100145914A1 (en) 2008-06-09 2010-06-10 Panasonic Corporation Database management server apparatus, database management system, database management method and database management program
US7765529B1 (en) 2003-10-31 2010-07-27 The Mathworks, Inc. Transforming graphical objects in a graphical modeling environment
US20100198769A1 (en) 2009-01-30 2010-08-05 Ab Initio Technology Llc Processing data using vector fields
EP2221733A1 (en) 2009-02-17 2010-08-25 AMADEUS sas Method allowing validation in a production database of new entered data prior to their release
US20100223218A1 (en) 2007-01-10 2010-09-02 Radiation Watch Limited Data processing apparatus and method for automatically generating a classification component
US7805474B2 (en) 2001-04-13 2010-09-28 Siebel Systems, Inc. Method and apparatus for mapping between XML and relational representations
US7840949B2 (en) 2003-11-03 2010-11-23 Ramal Acquisition Corp. System and method for data transformation using dataflow graphs
US7853553B2 (en) 2001-03-26 2010-12-14 Siebel Systems, Inc. Engine for converting data from a source format to a destination format using user defined mappings
US7860863B2 (en) 2007-09-05 2010-12-28 International Business Machines Corporation Optimization model for processing hierarchical data in stream systems
US7890509B1 (en) 2006-12-05 2011-02-15 First American Real Estate Solutions Llc Parcel data acquisition and processing
US7895586B2 (en) 2004-06-21 2011-02-22 Sanyo Electric Co., Ltd. Data flow graph processing method, reconfigurable circuit and processing apparatus
US20110061057A1 (en) 2009-09-04 2011-03-10 International Business Machines Corporation Resource Optimization for Parallel Data Integration
US20110066602A1 (en) 2009-09-16 2011-03-17 Ab Initio Software Llc Mapping dataset elements
US7912264B2 (en) 2007-08-03 2011-03-22 Siemens Medical Solutions Usa, Inc. Multi-volume rendering of single mode data in medical diagnostic imaging
JP2011090598A (ja) 2009-10-26 2011-05-06 Hitachi Solutions Ltd 文書検証装置および文書検証方法
WO2011060257A1 (en) 2009-11-13 2011-05-19 Ab Initio Technology Llc Managing record format information
US20110145297A1 (en) 2009-12-16 2011-06-16 Teradata Us, Inc. System and method for enhanced user interactions with a grid
US7970746B2 (en) 2006-06-13 2011-06-28 Microsoft Corporation Declarative management framework
US7979646B2 (en) 2002-11-22 2011-07-12 Qst Holdings, Inc. External memory controller node
US20110295863A1 (en) 2010-05-26 2011-12-01 Microsoft Corporation Exposing metadata relationships through filter interplay
US8103704B2 (en) 2007-07-31 2012-01-24 ePrentise, LLC Method for database consolidation and database separation
US20120054164A1 (en) 2010-08-27 2012-03-01 Microsoft Corporation Reducing locking during database transactions
US20120102029A1 (en) 2010-10-25 2012-04-26 Ab Initio Technology Llc Managing data set objects
US20120158625A1 (en) 2010-12-16 2012-06-21 International Business Machines Corporation Creating and Processing a Data Rule
US20120185449A1 (en) 2011-01-14 2012-07-19 Ab Initio Technology Llc Managing changes to collections of data
US8255363B2 (en) 2005-06-08 2012-08-28 rPath Methods, systems, and computer program products for provisioning software using dynamic tags to identify and process files
US8423564B1 (en) 2006-10-31 2013-04-16 Ncr Corporation Methods and apparatus for managing and updating stored information
US20130166515A1 (en) * 2011-12-22 2013-06-27 David Kung Generating validation rules for a data report based on profiling the data report in a data processing tool
US8484159B2 (en) 2005-06-27 2013-07-09 Ab Initio Technology Llc Managing metadata for graph-based computations
US8516008B1 (en) * 2012-05-18 2013-08-20 Splunk Inc. Flexible schema column store
US8555265B2 (en) 2010-05-04 2013-10-08 Google Inc. Parallel processing of data
US20140108357A1 (en) 2012-10-17 2014-04-17 Ab Initio Technology Llc Specifying and applying rules to data
US9092639B2 (en) 2008-12-02 2015-07-28 Ab Initio Technology Llc Processing data updates within a data maintenance system
US9367586B2 (en) 2011-06-01 2016-06-14 International Business Machines Corporation Data validation and service

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06162075A (ja) * 1992-11-20 1994-06-10 Sharp Corp 自動リライト装置
JPH1131042A (ja) * 1997-07-11 1999-02-02 Hitachi Ltd コード表データ作成時の属性即時チェック方法及び装置
CN101576823A (zh) * 2009-06-11 2009-11-11 阿里巴巴集团控股有限公司 一种表单控件的验证方法及装置

Patent Citations (127)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05507376A (ja) 1990-05-30 1993-10-21 アレン　―　ブラッドリィ　カンパニー，インコーポレーテッド 機械コントローラ／プロセスコントローラ用グラフィック・プログラム・インタフェース
US5630127A (en) 1992-05-15 1997-05-13 International Business Machines Corporation Program storage device and computer program product for managing an event driven management information system with rule-based application structure stored in a relational database
JPH0744368A (ja) 1993-07-29 1995-02-14 Hitachi Ltd 組合せモデルの編集システム
US5758351A (en) 1995-03-01 1998-05-26 Sterling Software, Inc. System and method for the creation and use of surrogate information system objects
US5966072A (en) 1996-07-02 1999-10-12 Ab Initio Software Corporation Executing computations expressed as graphs
JPH11143755A (ja) 1997-09-17 1999-05-28 Hewlett Packard Co <Hp> 階層構成情報のコピーおよび共有方法
US20010014890A1 (en) 1998-02-06 2001-08-16 Gwoho Liu Methods for mapping data fields from one data set to another in a data processing environment
US6088702A (en) 1998-02-25 2000-07-11 Plantz; Scott H. Group publishing system
US6948154B1 (en) * 1999-03-22 2005-09-20 Oregon State University Methodology for testing spreadsheets
US20050262121A1 (en) * 1999-09-21 2005-11-24 International Business Machines Corporation Method, system, program, and data structure for cleaning a database table
US6633875B2 (en) 1999-12-30 2003-10-14 Shaun Michael Brady Computer database system and method for collecting and reporting real estate property and loan performance information over a computer driven network
US20040088318A1 (en) 1999-12-30 2004-05-06 Brady Shaun Michael Computer database system and method for collecting and reporting real estate property and loan performance information over a computer driven network
US20010007959A1 (en) * 2000-01-07 2001-07-12 Samuel Abdalla Tabular range editing mechanism
US7164422B1 (en) 2000-07-28 2007-01-16 Ab Initio Software Corporation Parameterized graphs with conditional components
US20040239681A1 (en) 2000-08-07 2004-12-02 Zframe, Inc. Visual content browsing using rasterized representations
US6708186B1 (en) 2000-08-14 2004-03-16 Oracle International Corporation Aggregating and manipulating dictionary metadata in a database system
US7143076B2 (en) 2000-12-12 2006-11-28 Sap Aktiengesellschaft Method and apparatus for transforming data
US20040024740A1 (en) 2001-01-16 2004-02-05 Mcgeorge Vernon E. Method and system for validating data submitted to a database application
US20020161799A1 (en) * 2001-02-27 2002-10-31 Microsoft Corporation Spreadsheet error checker
JP2002279147A (ja) 2001-03-22 2002-09-27 Sharp Corp 内製化判断支援装置、内製化判断支援方法および内製化判断支援プログラムを記録した機械読取可能な記録媒体および内製化判断支援プログラム
US20040056908A1 (en) 2001-03-22 2004-03-25 Turbo Worx, Inc. Method and system for dataflow creation and execution
US7853553B2 (en) 2001-03-26 2010-12-14 Siebel Systems, Inc. Engine for converting data from a source format to a destination format using user defined mappings
US7805474B2 (en) 2001-04-13 2010-09-28 Siebel Systems, Inc. Method and apparatus for mapping between XML and relational representations
US6494159B2 (en) 2001-05-11 2002-12-17 The United States Of America As Represented By The Secretary Of The Navy Submarine launched unmanned combat vehicle replenishment
US6832366B2 (en) 2001-05-17 2004-12-14 Simdesk Technologies, Inc. Application generator
US20030163597A1 (en) 2001-05-25 2003-08-28 Hellman Ziv Zalman Method and system for collaborative ontology modeling
US20030016246A1 (en) 2001-07-18 2003-01-23 Sanjai Singh Graphical subclassing
US20050178833A1 (en) 2001-12-20 2005-08-18 Canon Information Systems Research Australia Pty Microprocessor card defining a custom user interface
US7080088B1 (en) 2002-01-30 2006-07-18 Oracle International Corporation Automatic reconciliation of bindable objects
US7185317B2 (en) 2002-02-14 2007-02-27 Hubbard & Wells Logical data modeling and integrated application framework
US20030163441A1 (en) * 2002-02-22 2003-08-28 Niels Godfredsen Method and system for navigating a large amount of data
US7110924B2 (en) 2002-05-15 2006-09-19 Caterpillar Inc. Method for controlling the performance of a target system
US20040015783A1 (en) 2002-06-20 2004-01-22 Canon Kabushiki Kaisha Methods for interactively defining transforms and for generating queries by manipulating existing query data
US20050144189A1 (en) 2002-07-19 2005-06-30 Keay Edwards Electronic item management and archival system and method of operating the same
US7167850B2 (en) 2002-10-10 2007-01-23 Ab Initio Software Corporation Startup and control of graph-based computation
US7979646B2 (en) 2002-11-22 2011-07-12 Qst Holdings, Inc. External memory controller node
US7257603B2 (en) 2003-05-08 2007-08-14 Microsoft Corporation Preview mode
US20040225632A1 (en) 2003-05-08 2004-11-11 Microsoft Corporation Automated information management and related methods
US20050010896A1 (en) 2003-07-07 2005-01-13 International Business Machines Corporation Universal format transformation between relational database management systems and extensible markup language using XML relational transformation
US20050060317A1 (en) * 2003-09-12 2005-03-17 Lott Christopher Martin Method and system for the specification of interface definitions and business rules and automatic generation of message validation and transformation software
US20050114369A1 (en) 2003-09-15 2005-05-26 Joel Gould Data profiling
US20050060313A1 (en) * 2003-09-15 2005-03-17 Oracle International Corporation A California Corporation Data quality analyzer
US7756873B2 (en) 2003-09-15 2010-07-13 Ab Initio Technology Llc Functional dependency data profiling
US7328428B2 (en) 2003-09-23 2008-02-05 Trivergent Technologies, Inc. System and method for generating data validation rules
US7765529B1 (en) 2003-10-31 2010-07-27 The Mathworks, Inc. Transforming graphical objects in a graphical modeling environment
US7840949B2 (en) 2003-11-03 2010-11-23 Ramal Acquisition Corp. System and method for data transformation using dataflow graphs
US20050187984A1 (en) 2004-02-20 2005-08-25 Tianlong Chen Data driven database management system and method
US20050234762A1 (en) 2004-04-16 2005-10-20 Pinto Stephen K Dimension reduction in predictive model development
US7895586B2 (en) 2004-06-21 2011-02-22 Sanyo Electric Co., Ltd. Data flow graph processing method, reconfigurable circuit and processing apparatus
US7536406B2 (en) 2004-06-23 2009-05-19 Microsoft Corporation Impact analysis in an object model
US20060007464A1 (en) 2004-06-30 2006-01-12 Percey Michael F Structured data update and transformation system
US20060020570A1 (en) 2004-07-23 2006-01-26 Yuh-Cherng Wu Conflict resolution engine
JP2006113810A (ja) 2004-10-14 2006-04-27 Fujitsu Ltd 書誌情報入力支援プログラム
US20060095466A1 (en) 2004-11-02 2006-05-04 Daniell Stevens Managing related data objects
JP2008524671A (ja) 2004-11-02 2008-07-10 エービー イニティオ ソフトウェア コーポレーション 関係データオブジェクトの管理
US20060200739A1 (en) 2005-03-07 2006-09-07 Rishi Bhatia System and method for data manipulation
JP2006277624A (ja) 2005-03-30 2006-10-12 Ns Engineering Corp データ照合装置及び方法
US8255363B2 (en) 2005-06-08 2012-08-28 rPath Methods, systems, and computer program products for provisioning software using dynamic tags to identify and process files
US20100100220A1 (en) 2005-06-09 2010-04-22 Belanger David G Arrangement for guiding user design of comprehensive product solution using on-the-fly data validation
JP2008547134A (ja) 2005-06-27 2008-12-25 エービー イニティオ ソフトウェア コーポレーション グラフ型計算のためのパラメータ管理
US8484159B2 (en) 2005-06-27 2013-07-09 Ab Initio Technology Llc Managing metadata for graph-based computations
US7716630B2 (en) 2005-06-27 2010-05-11 Ab Initio Technology Llc Managing parameters for graph-based computations
US20070011208A1 (en) 2005-07-06 2007-01-11 Smith Alan R Apparatus, system, and method for performing semi-automatic dataset maintenance
US20070027858A1 (en) 2005-07-29 2007-02-01 Paul Weinberg Method for generating properly formed expressions
US20070050750A1 (en) 2005-08-31 2007-03-01 Microsoft Corporation Extensible data-driven setup application for operating system
US20070080088A1 (en) 2005-10-06 2007-04-12 Trotter Jason K Preassembly of cooperating parts
US20070094060A1 (en) 2005-10-25 2007-04-26 Angoss Software Corporation Strategy trees for data mining
US20070136692A1 (en) 2005-12-09 2007-06-14 Eric Seymour Enhanced visual feedback of interactions with user interface
US20070179956A1 (en) 2006-01-18 2007-08-02 Whitmyer Wesley W Jr Record protection system for networked databases
US20070198457A1 (en) 2006-02-06 2007-08-23 Microsoft Corporation Accessing and manipulating data in a data flow graph
US7661067B2 (en) 2006-02-21 2010-02-09 International Business Machines Corporation Method for providing quick responses in instant messaging conversations
US20070226203A1 (en) 2006-03-23 2007-09-27 Microsoft Corporation Generation of query and update views for object relational mapping
US7577724B1 (en) 2006-03-28 2009-08-18 Emc Corporation Methods and apparatus associated with advisory generation
US20070294119A1 (en) 2006-03-30 2007-12-20 Adaptive Alpha, Llc System, method and computer program product for evaluating and rating an asset management business and associate investment funds using experiential business process and performance data, and applications thereof
US20070239751A1 (en) 2006-03-31 2007-10-11 Sap Ag Generic database manipulator
US20070276787A1 (en) 2006-05-15 2007-11-29 Piedmonte Christopher M Systems and Methods for Data Model Mapping
US20070271381A1 (en) 2006-05-16 2007-11-22 Joseph Skeffington Wholey Managing computing resources in graph-based computations
US7970746B2 (en) 2006-06-13 2011-06-28 Microsoft Corporation Declarative management framework
US7689565B1 (en) 2006-06-28 2010-03-30 Emc Corporation Methods and apparatus for synchronizing network management data
US20080049022A1 (en) 2006-08-10 2008-02-28 Ab Initio Software Corporation Distributing Services in Graph-Based Computations
US8423564B1 (en) 2006-10-31 2013-04-16 Ncr Corporation Methods and apparatus for managing and updating stored information
US20080126988A1 (en) 2006-11-24 2008-05-29 Jayprakash Mudaliar Application management tool
US7890509B1 (en) 2006-12-05 2011-02-15 First American Real Estate Solutions Llc Parcel data acquisition and processing
US20080162384A1 (en) 2006-12-28 2008-07-03 Privacy Networks, Inc. Statistical Heuristic Classification
US20100223218A1 (en) 2007-01-10 2010-09-02 Radiation Watch Limited Data processing apparatus and method for automatically generating a classification component
US20080228697A1 (en) 2007-03-16 2008-09-18 Microsoft Corporation View maintenance rules for an update pipeline of an object-relational mapping (ORM) platform
US20080243772A1 (en) 2007-03-29 2008-10-02 Ariel Fuxman Method and sytsem for generating nested mapping specifications in a schema mapping formalism and for generating transformation queries based thereon
US20080243891A1 (en) 2007-03-30 2008-10-02 Fmr Corp. Mapping Data on a Network
US20080256014A1 (en) 2007-04-10 2008-10-16 Joel Gould Editing and Compiling Business Rules
US20080312979A1 (en) 2007-06-13 2008-12-18 International Business Machines Corporation Method and system for estimating financial benefits of packaged application service projects
US20080313204A1 (en) 2007-06-14 2008-12-18 Colorquick, L.L.C. Method and apparatus for database mapping
US8103704B2 (en) 2007-07-31 2012-01-24 ePrentise, LLC Method for database consolidation and database separation
US7912264B2 (en) 2007-08-03 2011-03-22 Siemens Medical Solutions Usa, Inc. Multi-volume rendering of single mode data in medical diagnostic imaging
US7860863B2 (en) 2007-09-05 2010-12-28 International Business Machines Corporation Optimization model for processing hierarchical data in stream systems
US20090094291A1 (en) 2007-09-14 2009-04-09 Oracle International Corporation Support for compensation aware data types in relational database systems
US20090083313A1 (en) 2007-09-20 2009-03-26 Stanfill Craig W Managing Data Flows in Graph-Based Computations
US20090234623A1 (en) 2008-03-12 2009-09-17 Schlumberger Technology Corporation Validating field data
US20100145914A1 (en) 2008-06-09 2010-06-10 Panasonic Corporation Database management server apparatus, database management system, database management method and database management program
US20090319494A1 (en) 2008-06-20 2009-12-24 Microsoft Corporation Field mapping for data stream output
US20090327196A1 (en) 2008-06-30 2009-12-31 Ab Initio Software Llc Data Logging in Graph-Based Computations
US20100114833A1 (en) 2008-10-31 2010-05-06 Netapp, Inc. Remote office duplication
WO2010056867A1 (en) 2008-11-12 2010-05-20 Ab Initio Software Llc Managing and automatically linking data objects
US20100121890A1 (en) 2008-11-12 2010-05-13 Ab Initio Software Llc Managing and automatically linking data objects
US9092639B2 (en) 2008-12-02 2015-07-28 Ab Initio Technology Llc Processing data updates within a data maintenance system
WO2010065511A1 (en) 2008-12-02 2010-06-10 Ab Initio Software Llc Mapping instances of a dataset within a data management system
US20100138388A1 (en) 2008-12-02 2010-06-03 Ab Initio Software Llc Mapping instances of a dataset within a data management system
US20100198769A1 (en) 2009-01-30 2010-08-05 Ab Initio Technology Llc Processing data using vector fields
EP2221733A1 (en) 2009-02-17 2010-08-25 AMADEUS sas Method allowing validation in a production database of new entered data prior to their release
US20120167112A1 (en) 2009-09-04 2012-06-28 International Business Machines Corporation Method for Resource Optimization for Parallel Data Integration
US20110061057A1 (en) 2009-09-04 2011-03-10 International Business Machines Corporation Resource Optimization for Parallel Data Integration
WO2011035039A1 (en) 2009-09-16 2011-03-24 Ab Initio Technology Llc Mapping dataset elements
US20110066602A1 (en) 2009-09-16 2011-03-17 Ab Initio Software Llc Mapping dataset elements
JP2011090598A (ja) 2009-10-26 2011-05-06 Hitachi Solutions Ltd 文書検証装置および文書検証方法
US20110153667A1 (en) 2009-11-13 2011-06-23 Ab Initio Technology Llc Managing record format information
WO2011060257A1 (en) 2009-11-13 2011-05-19 Ab Initio Technology Llc Managing record format information
US20110145297A1 (en) 2009-12-16 2011-06-16 Teradata Us, Inc. System and method for enhanced user interactions with a grid
US8555265B2 (en) 2010-05-04 2013-10-08 Google Inc. Parallel processing of data
US20110295863A1 (en) 2010-05-26 2011-12-01 Microsoft Corporation Exposing metadata relationships through filter interplay
US20120054164A1 (en) 2010-08-27 2012-03-01 Microsoft Corporation Reducing locking during database transactions
US20120102029A1 (en) 2010-10-25 2012-04-26 Ab Initio Technology Llc Managing data set objects
US20120158625A1 (en) 2010-12-16 2012-06-21 International Business Machines Corporation Creating and Processing a Data Rule
WO2012097278A1 (en) 2011-01-14 2012-07-19 Ab Initio Technology Llc Managing changes to collections of data
US20120185449A1 (en) 2011-01-14 2012-07-19 Ab Initio Technology Llc Managing changes to collections of data
US9367586B2 (en) 2011-06-01 2016-06-14 International Business Machines Corporation Data validation and service
US20130166515A1 (en) * 2011-12-22 2013-06-27 David Kung Generating validation rules for a data report based on profiling the data report in a data processing tool
US8516008B1 (en) * 2012-05-18 2013-08-20 Splunk Inc. Flexible schema column store
US20140108357A1 (en) 2012-10-17 2014-04-17 Ab Initio Technology Llc Specifying and applying rules to data

Non-Patent Citations (12)

* Cited by examiner, † Cited by third party
Title
Chaiken et al., "Xope: easy and efficient parallel processing of massive data sets," J. Proc. of the VLDB Endowment VLDB Endowment Hompagearchive, vol. 1, No. 2, (2008), pp. 1265-1276.
English Translation of Notice of Reasons for Rejection, Japanese Application No. 2011-539631, dated Oct. 28, 2013 (3 pages).
Harkins, Susan; "Use Excel's conditional formatting to find errors"; Feb. 16, 2008; TechRepublic; http://www.techrepublic.com/blog/microsoft-office/use-excels-conditional-formatting-to-find-errors/ ; p. 1-3. *
Liskin, Miriam "Microsoft Access 97 for Windows SuperGuide" Ziff-Davis Press, Jan. 1, 1997, ch. 4 & 15, pp. 117-157 and 687-739.
Melia, Mark et al., "Constraint-Based Validation of Adaptive e-Learning Courseware," IEEE Transactions on Learning Technologies, vol. 2, No. 1, Jan.-Mar. 2009, pp. 37-49.
Pinheiro et al., "Mobile agents for aggregation of network management data," Agent Systems and Applications, (1999) pp. 130-140.
Rull, Guillem et al., "MVT: A Schema Mapping Validation Tool," EDBT'09, Mar. 24-26, 2009, pp. 1120-1123.
U.S. Appl. No. 12/628,521, filed Dec. 1, 2009, Mapping Instances of a Dataset Within a Data Management System.
U.S. Appl. No. 12/883,721, filed Sep. 16, 2010, Mapping Dataset Elements.
U.S. Appl. No. 13/281,039, filed Oct. 25, 2011, Managing Data Set Objects.
U.S. Appl. No. 13/350,191, filed Jan. 13, 2012, Managing Changes to Collections of Data.
Van Megen, Rudolf et al., "Costs and benefits of early defect detection: experiences from developing client server and host applications." Software Quality Journal 4, 247-256 (1995).

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10769122B2 (en) 2014-03-13 2020-09-08 Ab Initio Technology Llc Specifying and applying logical validation rules to data
US11249981B2 (en) 2015-06-12 2022-02-15 Ab Initio Technology Llc Data quality analysis
US20200380212A1 (en) * 2019-05-31 2020-12-03 Ab Initio Technology Llc Discovering a semantic meaning of data fields from profile data of the data fields
US11704494B2 (en) * 2019-05-31 2023-07-18 Ab Initio Technology Llc Discovering a semantic meaning of data fields from profile data of the data fields
US11886399B2 (en) 2020-02-26 2024-01-30 Ab Initio Technology Llc Generating rules for data processing values of data fields from semantic labels of the data fields

Also Published As

Publication number Publication date
AU2013331476A1 (en) 2015-04-30
JP6400588B2 (ja) 2018-10-03
EP2909750B1 (en) 2019-12-04
AU2013331476B2 (en) 2018-06-14
EP2909750A1 (en) 2015-08-26
KR102126099B1 (ko) 2020-06-23
JP2016504642A (ja) 2016-02-12
CN104737163A (zh) 2015-06-24
KR20150074020A (ko) 2015-07-01
CA2887141C (en) 2021-05-04
CA2887141A1 (en) 2014-04-24
US20200301897A1 (en) 2020-09-24
US20140108357A1 (en) 2014-04-17
US20170004151A1 (en) 2017-01-05
HK1211115A1 (en) 2016-05-13
WO2014062638A1 (en) 2014-04-24

Similar Documents

Publication Publication Date Title
US20200301897A1 (en) Specifying and applying rules to data
JP6916403B2 (ja) データへの論理的妥当性検査規則の指定および適用
AU2015315203B2 (en) Conditional validation rules
US20180260375A1 (en) Evaluating rules applied to data
US9021440B1 (en) System and method for automated test script generation
AU2017327823B2 (en) Test case generator built into data-integration workflow editor
US9940182B1 (en) Business rule engine validation systems and related methods
JPWO2012144257A1 (ja) ポリシー記述支援システム及びポリシー記述支援方法

Legal Events

Date Code Title Description
AS Assignment

Owner name: AB INITIO SOFTWARE LLC, MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PROCOPS, ROY LEONARD;GOULD, JOEL;REEL/FRAME:029622/0727

Effective date: 20121019

Owner name: AB INITIO ORIGINAL WORKS LLC, MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AB INITIO SOFTWARE LLC;REEL/FRAME:029622/0745

Effective date: 20121019

Owner name: AB INITIO TECHNOLOGY LLC, MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AB INITIO ORIGINAL WORKS LLC;REEL/FRAME:029622/0760

Effective date: 20121019

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STPP Information on status: patent application and granting procedure in general

Free format text: WITHDRAW FROM ISSUE AWAITING ACTION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4