WO2016040445A1 - Règles de validation conditionnelles - Google Patents

Règles de validation conditionnelles Download PDF

Info

Publication number
WO2016040445A1
WO2016040445A1 PCT/US2015/049131 US2015049131W WO2016040445A1 WO 2016040445 A1 WO2016040445 A1 WO 2016040445A1 US 2015049131 W US2015049131 W US 2015049131W WO 2016040445 A1 WO2016040445 A1 WO 2016040445A1
Authority
WO
WIPO (PCT)
Prior art keywords
field
condition
axis
validation
cells
Prior art date
Application number
PCT/US2015/049131
Other languages
English (en)
Inventor
Joel Gould
Roy Procops
Original Assignee
Ab Initio Technology Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ab Initio Technology Llc filed Critical Ab Initio Technology Llc
Priority to CN201580048962.7A priority Critical patent/CN106796585B/zh
Priority to KR1020177009727A priority patent/KR102214297B1/ko
Priority to AU2015315203A priority patent/AU2015315203B2/en
Priority to SG11201702139QA priority patent/SG11201702139QA/en
Priority to CA2960270A priority patent/CA2960270C/fr
Priority to JP2017511602A priority patent/JP6720144B2/ja
Priority to EP15771811.5A priority patent/EP3191982A1/fr
Publication of WO2016040445A1 publication Critical patent/WO2016040445A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/174—Form filling; Merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
    • G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/226—Validation

Definitions

  • This description relates to specifying and applying rules to data.
  • datasets Many modern applications, including business applications, process large sets of data (i.e., "datasets") which can be compiled from various sources.
  • the various sources that provide data to the dataset can have different levels of data quality.
  • an adequate level of data quality in the dataset should be maintained.
  • the dataset can be processed by a data validation system.
  • a data validation system applies validation rules to the dataset before it is provided to the application.
  • the data validation system uses the results of validation rules to calculate a measure of data quality and alert an administrator of the application if the measure of data quality falls below a
  • the data validation system includes modules for handling data that fails one or more of the validation rules. For example, the data validation system can discard or repair data that fails one or more of the validation rules.
  • validation rules applied by the data validation system are defined by an administrator of the data validation system.
  • an aspect 1 that include a system that includes processors and processing instructions, including a user interface module configured to render a plurality of cells arranged in a two-dimensional grid having a first axis and a second axis, the two-dimensional grid including one or more subsets of the cells extending in a direction along the first axis of the two-dimensional grid, each subset of the one or more subsets associated with a respective field of an element of the plurality of elements of the dataset, and multiple subsets of the cells extending in a direction along the second axis of the two-dimensional grid, one or more of the multiple subsets associated with a respective validation rule.
  • a user interface module configured to render a plurality of cells arranged in a two-dimensional grid having a first axis and a second axis, the two-dimensional grid including one or more subsets of the cells extending in a direction along the first axis of the two-dimensional grid, each subset of the one or more subsets associated with a respective field
  • the method includes rendering a subset of the cells extending in a direction along the second axis of the two-dimensional grids, each cell in the subset being associated with the respective field at the corresponding position in the grid identified by the second axis, and being capable of being used as a conditional cell.
  • a condition cell associated with a field includes an input element, for receiving input in the form of a conditional expression.
  • a processing module configured to evaluate a conditional expression from a condition cell and, based on that evaluation, apply one or more validation rules to at least one element of the dataset.
  • inventions of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • a system of one or more computers can be configured to perform particular actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or causes the system to perform the actions.
  • One or more computer programs can be configured to perform particular actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.
  • An aspect 2 according to aspect 1 includes that the input element may receive a condition such that at least one validation rule associated with the second axis is applied to the associated field when the condition is satisfied.
  • An aspect 3 according to aspects 1 or 2 include that the condition may be dependent on a value of a second field different from the associated field.
  • An aspect 4 includes applying validation rules to data in a first field of a first element of the dataset including determining selected validation rules associated with cells from a subset of cells extending in the direction along the second axis associated with the first field of the first element, based on any input received in the input elements of the cells; determining a condition associated with the selected validation rules based on input received in an input element of a condition cell associated with the first field; determining that the condition is satisfied; and applying the selected validation rules to the first field.
  • An aspect 5 according to aspects 1, 2, 3, or 4 includes that the input element of the condition cell may be configured to receive an operator and a value associated with the operator.
  • An aspect 6 according to aspects 1, 2, 3, 4 or 5 includes that determining that the condition is satisfied may include applying the operator to the value associated with the operator and a value in a second field of the first element of the dataset, the second field being identified by the field identifier.
  • aspects of the disclosed system can have one or more of the following
  • a user interface can enable a business user to identify validation rules.
  • the application of a validation rule can be conditional on the value of a field in a data element. Validating a dataset can increase the performance of application using that dataset.
  • Conditional validation rules can improve system performance by bypassing the unnecessary application of validation rules.
  • Conditional validation rules can reduce system complexity, but allowing specific validation rules to be easily applied for uncommon test cases.
  • FIG. 1 shows an exemplary data processing system in which the conditional techniques can be used.
  • FIG. 2 is an example of a user interface for specifying validation rules for conditionally validating data elements in a data set.
  • FIG. 3 illustrates an example of a conditional validation rule.
  • Data processing systems accept data from different sources and produce outputs based on the data.
  • the data processed by the system can include anomalies, errors, or issues.
  • These data quality problems can be due to, for example, errors caused by inaccurate entry, such as typographical errors, machine failures, such as data corruption, and errors caused by translation errors, such as an incorrect identification of the encoding of the data. Errors can also be due to missing or unreported data, improperly formatted data, change in specification at data producer that results in misinterpreted data at data consumer, errors in reference code sets resulting in mismatches with data,
  • synchronization problems e.g. new reference data fails to appear on time and source data does not match older reference data
  • introduction of new code values etc.
  • a goal of data quality system is to detect issues in the data that need to be measured, usually as part of a data improvement program (number of issues should go down over time).
  • Data validation is the process of measuring the quality of the data. In general, one or more data validation tests are executed against a data source. Quality of the data is quantified and, if necessary, action can be taken to improve the quality of the data.
  • a user interface can simplify the process of creating a validation rule.
  • a validation rule is a set of one or more validation tests that can be applied to data to determine the data quality.
  • a validation rule may include a validation test that determines whether a field of data is of a particular data type and a validation test that determines whether a field of data is of a particular size.
  • the systems, methods, and mediums described herein have at least some of the following advantages over the prior art.
  • a user interface can enable a business user to identify validation rules. Logical rules can be associated with business terms. Fields in a dataset can also be associated with business terms. Default validation rules can be determined based on the associations. Logical rules can decrease the time spent validating a data set.
  • a business term represents a concept that is embodied in the data.
  • a relational database table named "cust tbl” may include a field named “phone no.”
  • a row in the "cust tbl” with a value in the "phone no” field represents that a customer exists with that particular phone number.
  • "customer” is a business term embodied in, at least, the "cust tbl” and "phone number” is a business term embodied in the "phone no” field.
  • FIG. 1 shows an exemplary data processing system 100 in which the conditional techniques can be used.
  • the system 100 includes a data source 102.
  • the data source 102 can include one or more sources of data such as storage devices or connections to online data streams, each of which can store data (sometimes referred to as a "dataset") in any of a variety of storage formats (e.g., database tables, spreadsheet files, flat text files, or a native format used by a mainframe).
  • data sometimes referred to as a "dataset”
  • storage formats e.g., database tables, spreadsheet files, flat text files, or a native format used by a mainframe.
  • the execution environment 104 can be hosted on one or more general-purpose computers under the control of a suitable operating system, such as UNIX.
  • the execution environment 104 can include a multiple-node parallel computing environment: this can include configuration of computer systems using multiple central processing units (CPUs); either local (e.g., multiprocessor systems such as SMP computers) or locally distributed processors (e.g., multiple processors coupled as clusters or MPPs); remote or remotely distributed processors (e.g., multiple processors coupled via a local area network (LAN) and/or wide-area network (WAN)); or any combination thereof.
  • CPUs central processing units
  • local e.g., multiprocessor systems such as SMP computers
  • locally distributed processors e.g., multiple processors coupled as clusters or MPPs
  • remote or remotely distributed processors e.g., multiple processors coupled via a local area network (LAN) and/or wide-area network (WAN)
  • LAN local area network
  • WAN wide-area network
  • the execution environment includes a user interface (UI) module 106 and a processing module 108.
  • the UI module 106 manages input received from a user 110 over a user interface 112 (e.g., a graphical view on a display screen) which is utilized in specifying validation rules to be used by the processing module 108 for processing data from the data source 102.
  • a user interface 112 e.g., a graphical view on a display screen
  • the processing module 108 reads data from the data source 102 and performs validation procedures based on validation information obtained by the UI module 106.
  • Storage devices providing the data source 102 can be local to the execution environment 104, for example, being stored on a storage medium connected to a computer running the execution environment 104 (e.g., hard drive 114). Alternatively or additionally, the storage devices can be remote to the execution environment 104, for example, being hosted on a remote system (e.g., mainframe 116) in communication with a computer running the execution environment 104, over a remote connection.
  • a remote system e.g., mainframe 116
  • a dataset accessed from the data source 102 includes a number of data elements, which can be records formatted according to a predetermined record structure, or rows in a database table.
  • Each data element can include values for a number of fields, for example, attributes defined within a record structure or columns in a database table, possibly including null or empty values.
  • Various characteristics of values in the fields or the presence or absence of values in certain fields can be considered valid or invalid.
  • a "last_name” field including the string "Smith” may be considered valid
  • a "last name” field that is blank may be considered invalid.
  • the conditions that determine whether a field is valid or invalid can be different for different fields.
  • a "middle name” field including the string "Randol ' may be considered valid and a "middle name” field that is blank may also be considered valid.
  • fields in a dataset can represent a physical representation of a business rule.
  • a "first name” field (which may for example, be a variable-length character string stored in a relational database) in a "customer" data set may include data that is the first name of a customer and therefore is associated with a business term "first name". That is, the character string "John" in the "first name” field of a data element of the customer data set represents the existence of a business customer with the given name "John.”
  • Other datasets may represent the same business concept in different ways.
  • a second dataset can include the field "fname" which may also correspond to the business term "first name.”
  • FIG. 2 is an example of a user interface 112 for specifying validation rules for conditionally validating data elements in a data set.
  • the user interface 1 12 is configured to facilitate the user 110 specifying and verifying one or more conditional validation rules.
  • a validation rule may include one or more validation tests, each of the validation tests may require the application of another validation rule. Some validation tests may be conditional on values in the data elements of the data set.
  • the UI module 106 renders the user interface 112.
  • the user interface 112 may be rendered, for example, on a computer monitor.
  • the user interface 112 includes a number of cells 224 arranged in a two-dimensional grid 225 with a first axis 226 and a second axis 228.
  • One or more subsets 230 of the cells 224 (referred to as rows 230 in the remainder of the detailed description) extends in a direction along the first axis 226 of the two-dimensional grid 225.
  • Each of the rows 230 is associated with a field 218.
  • the first (i.e., leftmost) cell of each of the rows 230 includes the name of the data element 218 associated with the row 230.
  • the first cells of each row include the data element names "Field 1 ,” “Field 2,” ... “Field M”.
  • Multiple subsets 232 of the cells 224 extend along the second axis 228.
  • One or more of the columns 232 is associated with a validation test 234.
  • the columns 232 can include a "conditions" column 250 that identifies circumstances under which the validations rules associated with the data element (identified by the row) are applied.
  • the first (i.e., the topmost) cell of each of the remaining columns 232 includes the name of the validation test 234 associated with the column, or a name such as "Conditional Test” or "Conditions" for the conditions column 250.
  • the validation rule names are "Validation Test 1," “Validation Test 2,” ... “Validation Test N”. It is noted that in some examples, the directions of the first axis 226 and the second axis 228 can be swapped, causing the rows 230 associated with the fields 218 to become columns and the columns 232 associated with the validation test 234 to become rows.
  • validation test columns 232 are added to the two-dimensional grid 225, the user 110 can specify which validation test 234 should be applied to which data elements 218.
  • An example, of different kinds of validation tests that can be specified is described in U.S. Patent 8,069,129 entitled “EDITING AND COMPILING BUSINESS RULES", incorporated herein by reference.
  • the user interface 1 12 includes a condition column 250 that allows the user to establish a pre-condition for testing the validation rules for a given row.
  • the user may specify that "Condition 1" 252 should be met before applying "Validation Test 1" with Test Parameters 236 to the data element "Field 1" 218.
  • the condition can refer to a data element that may or may not be the subject of the validation.
  • the condition may require that a field in the data element have a value that matches a predetermined value, is included in a range or set of possible values, or matches a provided regular expression.
  • the field may or may not be the field that is the subject of the validation test.
  • condition 1 252 may test the value of Field 3 254 before applying Validation Test 1 234 to the value of Field 1 218.
  • the condition can require that a data element have a value (for example, the data element does not include a null value indicator.)
  • conditions can be data driven.
  • data driven conditions can include record of a certain type, accounts in a certain range, countries in a set of values. The conditions may involve lookup files or referencing data in another input file or database table.
  • establishing a condition can include identifying a field, an operation, and a value.
  • the operation can be, for example, a Boolean function.
  • the field identifies which data element is going to be evaluated by the condition, the operation identifies how the field is going to be evaluated, and a value is used to determine whether the condition is satisfied based on the evaluation. For example, for the condition:
  • the field is Country
  • the value is 'US'.
  • the condition is satisfied is a value in a country field for the data element is equal to 'US', and the condition is not satisfied if the value is not 'US'.
  • the user interface when a user adds a condition to a field the user interface automatically creates a new validation test row for that field. For example, in response to the addition of the condition 252 to Field 1 218 the user interface inserts a new row 256. Multiple rows in the user interface that each refer to the same Field may be evaluated sequentially, that is from top to bottom.
  • a field in the data element being validated fails the validation rule if it fails any validation test that the system determines should be applied. For example, if the system determines that Condition 1 252 is satisfied and applies Validation Test 1 234 with Test Parametersij 236 to the value in the data element and that value does not meet the validation test then the field of the data element fails and the failure counter 240 is incremented.
  • validation test 256 would not be applied.
  • each validation rule for which the conditions are met is applied.
  • the system may designate a final validation rule as an catchall validation rule that is only executed if no other validation rule is applied to the field.
  • a user may designate a validation rule as an catchall validation rule by supplying a predetermined condition, for example, by using a keyword (such as "else") in the condition test column.
  • a catchall validation rule is a validation rule that executes on a field if and only if no other validation rule has executed for that field.
  • a catchall is provided for that field automatically.
  • a validation rule set may include multiple different condition tests that are each associated with one or more validation rules.
  • a business rule may have one set of validation rules that are applied when country code is 'US' and another set of one or more validation rules that are applied when the state code is "MA”. And other set of validation rules may be applied when the last name begins with a "W.”
  • only a single condition will be triggered for a given validation rule set, for example, after a single condition is true, the associated one or more validation rules are applied and then the validation rule set ends.
  • each condition is tested and all of the validation rules associated with any triggered condition are applied.
  • a validation rule set can include conditional validation rules and non-conditional validation rules.
  • the non-conditional validation rules can be applied whether or not any condition is triggered.
  • FIG. 3 illustrates an example of a conditional validation rule.
  • a user interface 300 which can be, for example, the user interface 212 of FIG. 2, include a table 306 defines the validation rules for the "Postal Code" field.
  • the first validation 308 indicates that if the country is "US” then the postal code must be of the format "(99999)1(99999-9999).”
  • the second validation 310 indicates that if the country is "CA” then the postal code must be of the format "A9A 9A9”.
  • the third validation 312 indicates that if the country is "UK” then the postal code must be of the format " ⁇ (( ⁇ 9)
  • a fourth validation 314 is a catchall provision as designated by the condition "Else.” In this example, the fourth validation rule tests that the postal code is of the format "(99999)1(99999-9999).”
  • the user interface 300 also includes a fifth validation 316 that determines whether the country field contains one of the values "US", "CA”, UK".
  • the techniques described above can be implemented using software for execution on a computer.
  • the software forms procedures in one or more computer programs that execute on one or more programmed or programmable computer systems (which can be of various architectures such as distributed, client/server, or grid) each including at least one processor, at least one data storage system (including volatile and non-volatile memory and/or storage elements), at least one input device or port, and at least one output device or port.
  • the software can form one or more modules of a larger program, for example, that provides other services related to the design and configuration of dataflow graphs.
  • the nodes and elements of the graph can be implemented as data structures stored in a computer readable medium or other organized data conforming to a data model stored in a data repository.
  • the software can be provided on a storage medium, such as a CD-ROM, readable by a general or special purpose programmable computer, or delivered (encoded in a propagated signal) over a communication medium of a network to a storage medium of the computer where it is executed. All of the functions can be performed on a special purpose computer, or using special-purpose hardware, such as coprocessors.
  • the software can be implemented in a distributed manner in which different parts of the computation specified by the software are performed by different computers.
  • Each such computer program is preferably stored on or downloaded to a storage media or device (e.g., solid state memory or media, or magnetic or optical media) readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer system to perform the procedures described herein.
  • a storage media or device e.g., solid state memory or media, or magnetic or optical media
  • the inventive system can also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer system to operate in a specific and predefined manner to perform the functions described herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Quality & Reliability (AREA)
  • User Interface Of Digital Computer (AREA)
  • Stored Programmes (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Generation (AREA)
  • Image Analysis (AREA)
  • Debugging And Monitoring (AREA)

Abstract

L'invention concerne des procédés, des systèmes et des appareils, incluant des programmes informatiques codés sur des supports d'informations informatiques, pour la génération de règles de validation conditionnelles. Un des procédés consiste à représenter une pluralité de cellules agencées dans une grille bidimensionnelle comportant un premier axe et un second axe, la grille bidimensionnelle incluant un ou plusieurs sous-ensembles des cellules, chaque sous-ensemble étant associé à un champ respectif d'un élément de l'ensemble de données, et plusieurs sous-ensembles des cellules s'étendant dans une direction le long du second axe de la grille bidimensionnelle, un ou plusieurs des sous-ensembles étant associés à une règle de validation respective. Le procédé consiste à appliquer une ou plusieurs règles de validation à un élément de l'ensemble de données sur la base d'une entrée d'utilisateur reçue d'au moins certaines des cellules. Une cellule de condition associée à un champ inclut un élément d'entrée pour la réception d'entrée.
PCT/US2015/049131 2014-09-10 2015-09-09 Règles de validation conditionnelles WO2016040445A1 (fr)

Priority Applications (7)

Application Number Priority Date Filing Date Title
CN201580048962.7A CN106796585B (zh) 2014-09-10 2015-09-09 条件验证规则
KR1020177009727A KR102214297B1 (ko) 2014-09-10 2015-09-09 조건부 검증 규칙
AU2015315203A AU2015315203B2 (en) 2014-09-10 2015-09-09 Conditional validation rules
SG11201702139QA SG11201702139QA (en) 2014-09-10 2015-09-09 Conditional validation rules
CA2960270A CA2960270C (fr) 2014-09-10 2015-09-09 Regles de validation conditionnelles
JP2017511602A JP6720144B2 (ja) 2014-09-10 2015-09-09 条件付き検証規則
EP15771811.5A EP3191982A1 (fr) 2014-09-10 2015-09-09 Règles de validation conditionnelles

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14/482,374 2014-09-10
US14/482,374 US9626393B2 (en) 2014-09-10 2014-09-10 Conditional validation rules

Publications (1)

Publication Number Publication Date
WO2016040445A1 true WO2016040445A1 (fr) 2016-03-17

Family

ID=54207733

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/049131 WO2016040445A1 (fr) 2014-09-10 2015-09-09 Règles de validation conditionnelles

Country Status (9)

Country Link
US (1) US9626393B2 (fr)
EP (1) EP3191982A1 (fr)
JP (1) JP6720144B2 (fr)
KR (1) KR102214297B1 (fr)
CN (1) CN106796585B (fr)
AU (1) AU2015315203B2 (fr)
CA (1) CA2960270C (fr)
SG (1) SG11201702139QA (fr)
WO (1) WO2016040445A1 (fr)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11157689B2 (en) 2015-11-02 2021-10-26 Microsoft Technology Licensing, Llc Operations on dynamic data associated with cells in spreadsheets
US11023668B2 (en) 2015-11-02 2021-06-01 Microsoft Technology Licensing, Llc Enriched compound data objects
US20170124043A1 (en) 2015-11-02 2017-05-04 Microsoft Technology Licensing, Llc Sound associated with cells in spreadsheets
US10963635B2 (en) 2015-11-02 2021-03-30 Microsoft Technology Licensing, Llc Extensibility of compound data objects
US10515145B2 (en) 2015-11-02 2019-12-24 Microsoft Technology Licensing, Llc Parameterizing and working with math equations in a spreadsheet application
US10268833B2 (en) * 2016-04-07 2019-04-23 International Business Machines Corporation Method for conditional permission control in a digital data sheet based on a formula
US11210459B1 (en) 2016-09-23 2021-12-28 Massachusetts Mutual Life Insurance Company Systems, devices, and methods for software coding
US10540152B1 (en) 2016-09-23 2020-01-21 Massachusetts Mutual Life Insurance Company Systems, devices, and methods for software coding
US11138370B1 (en) 2016-09-23 2021-10-05 Massachusetts Mututal Life Insurance Company Modifying and using spreadsheets to create a GUI on another device
US10657120B2 (en) * 2016-10-03 2020-05-19 Bank Of America Corporation Cross-platform digital data movement control utility and method of use thereof
US11151100B2 (en) * 2016-10-17 2021-10-19 Sap Se Performing data quality functions using annotations
US10496737B1 (en) 2017-01-05 2019-12-03 Massachusetts Mutual Life Insurance Company Systems, devices, and methods for software coding
US11106643B1 (en) * 2017-08-02 2021-08-31 Synchrony Bank System and method for integrating systems to implement data quality processing
US10977240B1 (en) * 2017-10-21 2021-04-13 Palantir Technologies Inc. Approaches for validating data
EP3743820A1 (fr) * 2018-01-25 2020-12-02 Ab Initio Technology LLC Techniques d'intégration de résultats de validation en profilage de données et systèmes et procédés associés
CN109359277B (zh) * 2018-10-22 2023-04-28 众安在线财产保险股份有限公司 数据监控方法、设备及计算机存储介质
US11249978B2 (en) 2018-11-29 2022-02-15 Kyndryl, Inc. Multiple parameter based composite rule data validation
US11194793B1 (en) 2019-06-25 2021-12-07 Amazon Technologies, Inc. Dynamically materialized views for sheets based data
US11086894B1 (en) * 2019-06-25 2021-08-10 Amazon Technologies, Inc. Dynamically updated data sheets using row links
JP7382642B2 (ja) 2020-07-03 2023-11-17 株式会社岩崎電機製作所 データ群識別装置、データ群識別方法、および、データ群識別プログラム
US11797541B1 (en) 2020-10-23 2023-10-24 State Farm Mutual Automobile Insurance Company Systems and methods for enhanced rules conflict checking with data validation
US20230100418A1 (en) * 2021-09-17 2023-03-30 Ab Initio Technology Llc Metadata-driven data ingestion
WO2023200456A1 (fr) * 2022-04-15 2023-10-19 Rakuten Symphony Singapore Pte. Ltd. Validation conditionnelle pour un générateur de vues
WO2023200455A1 (fr) * 2022-04-15 2023-10-19 Rakuten Symphony Singapore Pte. Ltd. Validation conditionnelle pour constructeur de vues

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8069129B2 (en) 2007-04-10 2011-11-29 Ab Initio Technology Llc Editing and compiling business rules
US20140108357A1 (en) * 2012-10-17 2014-04-17 Ab Initio Technology Llc Specifying and applying rules to data

Family Cites Families (108)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5168441A (en) 1990-05-30 1992-12-01 Allen-Bradley Company, Inc. Methods for set up and programming of machine and process controllers
US5446885A (en) * 1992-05-15 1995-08-29 International Business Machines Corporation Event driven management information system with rule-based applications structure stored in a relational database
JPH0744368A (ja) 1993-07-29 1995-02-14 Hitachi Ltd 組合せモデルの編集システム
US5758351A (en) 1995-03-01 1998-05-26 Sterling Software, Inc. System and method for the creation and use of surrogate information system objects
US5966072A (en) 1996-07-02 1999-10-12 Ab Initio Software Corporation Executing computations expressed as graphs
US6216140B1 (en) 1997-09-17 2001-04-10 Hewlett-Packard Company Methodology for the efficient management of hierarchically organized information
US6216131B1 (en) 1998-02-06 2001-04-10 Starfish Software, Inc. Methods for mapping data fields from one data set to another in a data processing environment
US6088702A (en) * 1998-02-25 2000-07-11 Plantz; Scott H. Group publishing system
US6948154B1 (en) 1999-03-22 2005-09-20 Oregon State University Methodology for testing spreadsheets
US7120638B1 (en) 1999-09-21 2006-10-10 International Business Machines Corporation Method, system, program, and data structure for cleaning a database table
US6633875B2 (en) 1999-12-30 2003-10-14 Shaun Michael Brady Computer database system and method for collecting and reporting real estate property and loan performance information over a computer driven network
GB2358072B (en) 2000-01-07 2004-01-28 Mitel Corp Tabular range editing mechanism
US7164422B1 (en) 2000-07-28 2007-01-16 Ab Initio Software Corporation Parameterized graphs with conditional components
US6704024B2 (en) 2000-08-07 2004-03-09 Zframe, Inc. Visual content browsing using rasterized representations
US6708186B1 (en) 2000-08-14 2004-03-16 Oracle International Corporation Aggregating and manipulating dictionary metadata in a database system
US7143076B2 (en) 2000-12-12 2006-11-28 Sap Aktiengesellschaft Method and apparatus for transforming data
US6629098B2 (en) * 2001-01-16 2003-09-30 Hewlett-Packard Development Company, L.P. Method and system for validating data submitted to a database application
US7117430B2 (en) 2001-02-27 2006-10-03 Microsoft Corporation Spreadsheet error checker
US7614036B2 (en) 2001-03-22 2009-11-03 Robert D Bjornson Method and system for dataflow creation and execution
US7853553B2 (en) 2001-03-26 2010-12-14 Siebel Systems, Inc. Engine for converting data from a source format to a destination format using user defined mappings
US6732095B1 (en) 2001-04-13 2004-05-04 Siebel Systems, Inc. Method and apparatus for mapping between XML and relational representations
US6494159B2 (en) 2001-05-11 2002-12-17 The United States Of America As Represented By The Secretary Of The Navy Submarine launched unmanned combat vehicle replenishment
US6832366B2 (en) * 2001-05-17 2004-12-14 Simdesk Technologies, Inc. Application generator
US7099885B2 (en) 2001-05-25 2006-08-29 Unicorn Solutions Method and system for collaborative ontology modeling
US6868526B2 (en) 2001-07-18 2005-03-15 The Mathworks, Inc. Graphical subclassing
AUPR966001A0 (en) 2001-12-20 2002-01-24 Canon Information Systems Research Australia Pty Ltd A microprocessor card defining a custom user interface
US7080088B1 (en) 2002-01-30 2006-07-18 Oracle International Corporation Automatic reconciliation of bindable objects
US7185317B2 (en) 2002-02-14 2007-02-27 Hubbard & Wells Logical data modeling and integrated application framework
US6820077B2 (en) 2002-02-22 2004-11-16 Informatica Corporation Method and system for navigating a large amount of data
US7110924B2 (en) 2002-05-15 2006-09-19 Caterpillar Inc. Method for controlling the performance of a target system
US7574652B2 (en) 2002-06-20 2009-08-11 Canon Kabushiki Kaisha Methods for interactively defining transforms and for generating queries by manipulating existing query data
US7167850B2 (en) 2002-10-10 2007-01-23 Ab Initio Software Corporation Startup and control of graph-based computation
US7225301B2 (en) 2002-11-22 2007-05-29 Quicksilver Technologies External memory controller node
US7266763B2 (en) * 2002-11-26 2007-09-04 Microsoft Corporation User defined spreadsheet functions
US20040225632A1 (en) 2003-05-08 2004-11-11 Microsoft Corporation Automated information management and related methods
US7257603B2 (en) 2003-05-08 2007-08-14 Microsoft Corporation Preview mode
US20050010896A1 (en) 2003-07-07 2005-01-13 International Business Machines Corporation Universal format transformation between relational database management systems and extensible markup language using XML relational transformation
US20050060317A1 (en) 2003-09-12 2005-03-17 Lott Christopher Martin Method and system for the specification of interface definitions and business rules and automatic generation of message validation and transformation software
CA2655735C (fr) 2003-09-15 2011-01-18 Ab Initio Software Corporation Interconnexion de donnees
US7865507B2 (en) 2003-09-15 2011-01-04 Oracle International Corporation Data quality analyzer
US7765529B1 (en) 2003-10-31 2010-07-27 The Mathworks, Inc. Transforming graphical objects in a graphical modeling environment
US7743382B2 (en) 2003-11-03 2010-06-22 Ramal Acquisition Corp. System for deadlock condition detection and correction by allowing a queue limit of a number of data tokens on the queue to increase
US20050187984A1 (en) 2004-02-20 2005-08-25 Tianlong Chen Data driven database management system and method
US8165853B2 (en) 2004-04-16 2012-04-24 Knowledgebase Marketing, Inc. Dimension reduction in predictive model development
JP4275013B2 (ja) 2004-06-21 2009-06-10 三洋電機株式会社 データフローグラフ処理装置、処理装置、リコンフィギュラブル回路。
US7536406B2 (en) 2004-06-23 2009-05-19 Microsoft Corporation Impact analysis in an object model
US20060007464A1 (en) 2004-06-30 2006-01-12 Percey Michael F Structured data update and transformation system
US7353227B2 (en) 2004-07-23 2008-04-01 Sap Aktiengesellschaft Conflict resolution engine
US7899833B2 (en) 2004-11-02 2011-03-01 Ab Initio Technology Llc Managing related data objects
WO2006096681A1 (fr) 2005-03-07 2006-09-14 Computer Associates Think, Inc. Systeme et procede de manipulation de donnees
JP4550641B2 (ja) 2005-03-30 2010-09-22 大陽日酸エンジニアリング株式会社 データ照合装置及び方法
US8255363B2 (en) 2005-06-08 2012-08-28 rPath Methods, systems, and computer program products for provisioning software using dynamic tags to identify and process files
US8630917B2 (en) 2005-06-09 2014-01-14 At&T Intellectual Property Ii, L.P. Arrangement for guiding user design of comprehensive product solution using on-the-fly data validation
US7877350B2 (en) 2005-06-27 2011-01-25 Ab Initio Technology Llc Managing metadata for graph-based computations
US7716630B2 (en) 2005-06-27 2010-05-11 Ab Initio Technology Llc Managing parameters for graph-based computations
US7464105B2 (en) 2005-07-06 2008-12-09 International Business Machines Corporation Method for performing semi-automatic dataset maintenance
US7870162B2 (en) 2005-07-29 2011-01-11 Sap Ag Method for generating properly formed expressions
US20070050750A1 (en) 2005-08-31 2007-03-01 Microsoft Corporation Extensible data-driven setup application for operating system
WO2007048229A1 (fr) 2005-10-25 2007-05-03 Angoss Software Corporation Arborescences de stratégie pour une exploration de données
US8060821B2 (en) 2005-12-09 2011-11-15 Apple Inc. Enhanced visual feedback of interactions with user interface
US20070179956A1 (en) 2006-01-18 2007-08-02 Whitmyer Wesley W Jr Record protection system for networked databases
US7761586B2 (en) 2006-02-06 2010-07-20 Microsoft Corporation Accessing and manipulating data in a data flow graph
US7661067B2 (en) 2006-02-21 2010-02-09 International Business Machines Corporation Method for providing quick responses in instant messaging conversations
US7647298B2 (en) 2006-03-23 2010-01-12 Microsoft Corporation Generation of query and update views for object relational mapping
US20070294119A1 (en) 2006-03-30 2007-12-20 Adaptive Alpha, Llc System, method and computer program product for evaluating and rating an asset management business and associate investment funds using experiential business process and performance data, and applications thereof
US8868577B2 (en) 2006-03-31 2014-10-21 Sap Se Generic database manipulator
US7797319B2 (en) 2006-05-15 2010-09-14 Algebraix Data Corporation Systems and methods for data model mapping
US7870556B2 (en) 2006-05-16 2011-01-11 Ab Initio Technology Llc Managing computing resources in graph-based computations
US7689565B1 (en) 2006-06-28 2010-03-30 Emc Corporation Methods and apparatus for synchronizing network management data
CN101501678B (zh) 2006-08-10 2013-10-16 起元科技有限公司 在基于图的计算中分配服务
US8423564B1 (en) 2006-10-31 2013-04-16 Ncr Corporation Methods and apparatus for managing and updating stored information
US20080126988A1 (en) * 2006-11-24 2008-05-29 Jayprakash Mudaliar Application management tool
US7890509B1 (en) 2006-12-05 2011-02-15 First American Real Estate Solutions Llc Parcel data acquisition and processing
GB0700450D0 (en) 2007-01-10 2007-02-21 Radiation Watch Ltd The RWL threat engine
US9430552B2 (en) 2007-03-16 2016-08-30 Microsoft Technology Licensing, Llc View maintenance rules for an update pipeline of an object-relational mapping (ORM) platform
US20080243765A1 (en) 2007-03-29 2008-10-02 Ariel Fuxman Method for generating nested mapping specifications in a schema mapping formalism
US7603368B2 (en) 2007-03-30 2009-10-13 Fmr Llc Mapping data on a network
US8032404B2 (en) 2007-06-13 2011-10-04 International Business Machines Corporation Method and system for estimating financial benefits of packaged application service projects
US8694518B2 (en) 2007-06-14 2014-04-08 Colorquick, L.L.C. Method and apparatus for database mapping
US8103704B2 (en) 2007-07-31 2012-01-24 ePrentise, LLC Method for database consolidation and database separation
US7860863B2 (en) 2007-09-05 2010-12-28 International Business Machines Corporation Optimization model for processing hierarchical data in stream systems
US8566296B2 (en) 2007-09-14 2013-10-22 Oracle International Corporation Support for compensation aware data types in relational database systems
KR101572599B1 (ko) 2007-09-20 2015-11-27 아브 이니티오 테크놀로지 엘엘시 그래프에 기초한 연산에서 데이터 흐름을 관리하는 방법 및 시스템
US20090234623A1 (en) * 2008-03-12 2009-09-17 Schlumberger Technology Corporation Validating field data
JP4224126B1 (ja) 2008-06-09 2009-02-12 パナソニック株式会社 データベース管理サーバ装置、データベース管理システム、データベース管理方法およびデータベース管理プログラム
US20090319494A1 (en) 2008-06-20 2009-12-24 Microsoft Corporation Field mapping for data stream output
EP2324434A4 (fr) 2008-06-30 2013-10-30 Ab Initio Technology Llc Enregistrement chronologique de données dans des calculs sur la base de graphique
US8082228B2 (en) 2008-10-31 2011-12-20 Netapp, Inc. Remote office duplication
KR101627594B1 (ko) 2008-11-12 2016-06-07 아브 이니티오 테크놀로지 엘엘시 데이터 객체의 관리 및 자동 링킹
KR101661532B1 (ko) 2008-12-02 2016-09-30 아브 이니티오 테크놀로지 엘엘시 데이터 관리 시스템 내의 데이터 집합의 맵핑 인스턴스
CN101452441A (zh) * 2008-12-05 2009-06-10 山东浪潮齐鲁软件产业股份有限公司 一种电子表格通用数据解析导入方法
AU2010208112B2 (en) 2009-01-30 2015-05-28 Ab Initio Technology Llc Processing data using vector fields
EP2221733A1 (fr) 2009-02-17 2010-08-25 AMADEUS sas Procédé permettant la validation dans une base de données de production de nouvelles données entrées avant leur diffusion
US8935702B2 (en) 2009-09-04 2015-01-13 International Business Machines Corporation Resource optimization for parallel data integration
US8825695B2 (en) * 2009-09-16 2014-09-02 Ab Initio Technology Llc Mapping dataset elements
JP5401279B2 (ja) * 2009-11-26 2014-01-29 株式会社日立製作所 チェックルール設計支援方法、チェックルール設計支援システム、およびチェックルール設計支援プログラム
US9805015B2 (en) * 2009-12-16 2017-10-31 Teradata Us, Inc. System and method for enhanced user interactions with a grid
CN101794280B (zh) * 2010-03-11 2011-07-27 北京中科辅龙计算机技术股份有限公司 一种基于表格模板集的表格自动生成方法及系统
US8555265B2 (en) 2010-05-04 2013-10-08 Google Inc. Parallel processing of data
US8583664B2 (en) 2010-05-26 2013-11-12 Microsoft Corporation Exposing metadata relationships through filter interplay
US8732143B2 (en) 2010-08-27 2014-05-20 Microsoft Corporation Reducing locking during database transactions
CA2814835C (fr) 2010-10-25 2019-01-08 Ab Initio Technology Llc Gestion d'objets ensembles de donnees dans un graphe de flux de donnees qui represente un programme d'ordinateur
CN102044016A (zh) * 2010-12-13 2011-05-04 沈阳东大中新科技有限公司 以电子表格定义规则实现自动化产品配置的方法及装置
US8949166B2 (en) 2010-12-16 2015-02-03 International Business Machines Corporation Creating and processing a data rule for data quality
US9418095B2 (en) 2011-01-14 2016-08-16 Ab Initio Technology Llc Managing changes to collections of data
US20120310904A1 (en) * 2011-06-01 2012-12-06 International Business Machine Corporation Data validation and service
US20130166515A1 (en) 2011-12-22 2013-06-27 David Kung Generating validation rules for a data report based on profiling the data report in a data processing tool
US8516008B1 (en) 2012-05-18 2013-08-20 Splunk Inc. Flexible schema column store

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8069129B2 (en) 2007-04-10 2011-11-29 Ab Initio Technology Llc Editing and compiling business rules
US20140108357A1 (en) * 2012-10-17 2014-04-17 Ab Initio Technology Llc Specifying and applying rules to data

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Microsoft Access 97 for Windows SuperGuide - Chapters 4 and 15", 1 January 1997, ZIFF-DAVIS PRESS, Emeryville, CA 94608, ISBN: 978-1-56-276437-1, article MIRIAM LISKIN: "Microsoft Access 97 for Windows SuperGuide - Chapters 4 and 15", XP055089566 *

Also Published As

Publication number Publication date
KR102214297B1 (ko) 2021-02-08
CN106796585B (zh) 2021-06-01
US9626393B2 (en) 2017-04-18
CA2960270C (fr) 2022-01-04
AU2015315203B2 (en) 2020-09-03
EP3191982A1 (fr) 2017-07-19
CN106796585A (zh) 2017-05-31
SG11201702139QA (en) 2017-04-27
CA2960270A1 (fr) 2016-03-17
KR20170052672A (ko) 2017-05-12
JP6720144B2 (ja) 2020-07-08
AU2015315203A1 (en) 2017-03-16
JP2017534944A (ja) 2017-11-24
US20160070733A1 (en) 2016-03-10

Similar Documents

Publication Publication Date Title
AU2015315203B2 (en) Conditional validation rules
JP6916403B2 (ja) データへの論理的妥当性検査規則の指定および適用
US20200301897A1 (en) Specifying and applying rules to data
AU2010319344B2 (en) Managing record format information
KR20170031749A (ko) 파라미터 세트의 관리
US20240104235A1 (en) Techniques for agentless detection of sensitive data on managed databases
US20240104118A1 (en) System and method for agentless detection of sensitive data on managed databases
CN117370160A (zh) 代码审核方法、装置、计算机设备、存储介质和程序产品
CN116362230A (zh) 参数校验方法、装置、计算机设备可存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15771811

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017511602

Country of ref document: JP

Kind code of ref document: A

REEP Request for entry into the european phase

Ref document number: 2015771811

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015771811

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2960270

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2015315203

Country of ref document: AU

Date of ref document: 20150909

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20177009727

Country of ref document: KR

Kind code of ref document: A