US20090271351A1 - Rules engine test harness - Google Patents

Rules engine test harness Download PDF

Info

Publication number
US20090271351A1
US20090271351A1 US12/323,707 US32370708A US2009271351A1 US 20090271351 A1 US20090271351 A1 US 20090271351A1 US 32370708 A US32370708 A US 32370708A US 2009271351 A1 US2009271351 A1 US 2009271351A1
Authority
US
United States
Prior art keywords
test
rules
results
database
test cases
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/323,707
Inventor
Uma Maheswari Kandasamy
Krishnam Raju B. Datla
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Conduent Business Services LLC
Original Assignee
Affiliated Computer Services LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Affiliated Computer Services LLC filed Critical Affiliated Computer Services LLC
Priority to US12/323,707 priority Critical patent/US20090271351A1/en
Assigned to AFFILIATED COMPUTER SERVICES, INC. reassignment AFFILIATED COMPUTER SERVICES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DATLA, KRISHNAM RAJU B., KANDASAMY, UMA MAHESWARI
Publication of US20090271351A1 publication Critical patent/US20090271351A1/en
Assigned to AFFILIATED COMPUTER SERVICES, LLC reassignment AFFILIATED COMPUTER SERVICES, LLC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: AFFILIATED COMPUTER SERVICES, INC.
Assigned to JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT reassignment JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT SECURITY AGREEMENT Assignors: BUCK CONSULTANTS, LLC, Healthy Communities Institute Corporation, RSA MEDICAL LLC, STRATACARE, LLC, XEROX BUSINESS SERVICES, LLC (F/K/A AFFILIATED COMPUTER SERVICES, INC.), XEROX COMMERCIAL SOLUTIONS, LLC (F/K/A ACS COMMERCIAL SOLUTIONS, INC.), XEROX HR SOLUTIONS, LLC (F/K/A ACS HR SOLUTIONS, LLC), XEROX MORTGAGE SERVICES, INC. (F/K/A ADVECTIS, INC.), XEROX STATE & LOCAL SOLUTIONS, INC. (F/K/A ACS STATE AND LOCAL SOLUTIONS, INC.), XEROX TRANSPORT SOLUTIONS, INC. (F/K/A ACS TRANSPORT SOLUTIONS, INC.)
Assigned to CONDUENT BUSINESS SOLUTIONS, LLC, CONDUENT BUSINESS SERVICES, LLC, CONDUENT STATE & LOCAL SOLUTIONS, INC., CONDUENT HEALTH ASSESSMENTS, LLC, CONDUENT COMMERCIAL SOLUTIONS, LLC, CONDUENT CASUALTY CLAIMS SOLUTIONS, LLC, CONDUENT TRANSPORT SOLUTIONS, INC., ADVECTIS, INC. reassignment CONDUENT BUSINESS SOLUTIONS, LLC RELEASE BY SECURED PARTY (SEE DOCUMENT FOR DETAILS). Assignors: JPMORGAN CHASE BANK, N.A.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0637Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/08Insurance

Definitions

  • the United States Medicaid program was enacted in 1965 to provide a medical assistance program for individuals and families with low incomes.
  • the Medicaid program comprises three main entities—the patients, the healthcare providers, and the agency administering the plan (i.e., the payer).
  • the Medicaid program is financed through joint federal and state funding.
  • the Medicaid program is administered by each state according to an approved state plan.
  • the specifics of the Medicaid program differ from state to state. Differences may include covered healthcare procedures, allowable procedure costs, and patient eligibility criteria.
  • the state administrators of the Medicaid program are required to have a Medicaid management information system (MMIS) that provides for mechanized and/or computerized Medicaid claims processing.
  • MMIS Medicaid management information system
  • Recently, the Medicaid information technology architecture (MITA) has been promulgated by the U.S. government to provide a blueprint and a set of standards that individual states are to follow in administering the Medicaid program and for developing the next generation MMIS.
  • Rules engines may be pluggable software components that execute rules that have been externalized from application code.
  • the rules define business rules and/or business logic that may change frequently.
  • rules may be defined by nonprogrammers and may be provided to the rules engine in the form of data or data files.
  • Using a rules engine to provide business rules to an application may reduce time to market and reduce total cost of ownership, with reference to the alternative of encoding the business logic in high level programming language code.
  • Automation designs for example business rules to be executed by a rules engine and/or applications encoded in a high level programming language, are typically tested extensively before deployment in customer facing products and/or systems. Testing may take the form of executing a number of test cases on the target computer systems, where each test case comprises a single set of inputs to the subject rules engine and/or application. In addition to defining the inputs, the test cases may define the expected output and/or results of executing the rules engine and/or the application on the inputs. Because testing every possible combination of inputs is typically infeasible, an attempt is often made to identify a representative selection of test cases that may provide sufficient confidence in the automation design under test. Testing automation designs may be an expensive process that consumes substantial business resources including personnel, equipment, and schedule time. Insufficient testing can result in deployment of a flawed automation design that may fail inopportunely, possibly damaging the enterprise brand and/or incurring liability.
  • a system comprising a computer system, a builder component, and a test execution component.
  • the builder component promotes defining a plurality of test cases and a plurality of test scenarios.
  • the text execution component simulates at least one service application, invokes execution of a plurality of rules on a rules engine with an input based on one of the test cases defined using the builder component, wherein the rules engine interacts with the simulated service application, and stores the result of the rule execution in a database.
  • a method comprises providing a builder interface for building a plurality of test cases and a plurality of test scenarios, the interface comprising a list of rules and a plurality of input controls for defining a plurality of inputs for the test cases.
  • the method also comprises storing the test cases and the test scenarios in a database, invoking the rules on a rules engine based on at least one of the test cases and the test scenarios, and simulating at least one service application, wherein the rules engine interacts with the service application when executing the rules.
  • the method also comprises collecting the results of the invocation of the rules on the rules engine and storing the results in the database.
  • an apparatus comprising a test case editor and a rule test manager.
  • the test case editor When executed on a first computer system, the test case editor provides an environment for building a plurality of rule test cases by selecting an entry point from a list of available entry points, by defining a plurality of inputs for the rule, and by defining expected results for one of the test cases.
  • the rule test manager executes the rule test cases by simulating at least one service program, invoking a plurality of rules on a rules engine based on the rule test cases built using the test case editor, receives results from the rules engine, and stores the results, wherein the rules engine interacts with the simulated service program when executing the rules.
  • FIG. 1 is a schematic diagram of a rules engine test harness according to an embodiment of the disclosure.
  • FIG. 2 is a flowchart of a method of testing rules according to an embodiment of the disclosure.
  • FIG. 3 is a schematic diagram of an exemplary general-purpose computer system suitable for implementing some aspects of the several embodiments of the disclosure.
  • a rules engine test harness for testing rules may promote testing rules without using a fully configured and/or duplicate execution environment.
  • a rules engine framework is used to create and optionally to compile rules that may be executed by a rules engine. Further details about a rules engine framework are provided in U.S. patent application Ser. No. 12/257,782 filed Oct. 24, 2008 entitled “Rules Engine Framework” by Uma Kandasamy et al., which is hereby incorporated by reference in its entirety.
  • the rules When deployed as a released product, the rules may be executed by a rules engine on a process server computer system in cooperation and coordination with a business process.
  • the business process and rules engine cooperatively provide at least a portion of a healthcare management information system.
  • a user of the healthcare management information system may interact with the business process and rules engine by entering information and selecting actions from a user interface presented at a workstation communicatively coupled to the process server computer system.
  • the business process may promote enrolling healthcare providers into the healthcare system, for example enrolling physicians, therapists, hospitals, minor emergency medical centers, and other healthcare providers.
  • the business process may promote enrolling healthcare recipients and/or patients.
  • the business process may promote receiving and processing claims from enrolled healthcare providers for services provided to enrolled healthcare recipients.
  • a function of the business process may be invoked by the user of the workstation, for example processing of a claim for healthcare services.
  • the business process may process this claim by invoking a series of rules and/or a rule flow on the rules engine.
  • the use of rules to provide automated processing may have the advantage of allowing for the rapid creation, modification, and deployment of rules with respect to processing based on specifically designed computer programs.
  • the processing of the claim may involve the rules engine executing a sequence of rules that validate the enrollment of the healthcare provider, validate the enrollment of the healthcare recipient, determine a coverage status of the subject procedure, determines a maximum coverage amount, and other such processes.
  • the rules engine may return a rule result object to the business process that invoked the rules engine, wherein the rule result object may identify one or more exceptions if something went wrong, or no exceptions if the rule processed successfully.
  • the system 100 comprises a computer system 102 , a workstation 108 , a network 110 , and a database 112 .
  • the computer system 102 comprises a builder component 104 and a test execution component 106 .
  • the workstation 108 may execute a builder interface 126 and a test control interface 128 .
  • the database 112 may also be referred to as a test case repository, and may comprise a plurality of test cases 114 , a plurality of test scenarios 116 , a plurality of test results 122 , and a plurality of success/fail reports 124 .
  • the database 112 may be substantially empty and the test cases 114 , the test scenarios 116 , the test results 122 , and the success/fail reports 124 may accumulate in number over time. While the computer system 102 in FIG. 1 is shown to be in direct communication with the database 112 , in some embodiments the computer system 102 may communicate with the database 112 via the network 110 .
  • the computer system 102 , the workstation 108 , and the database 112 along with their contained components, applications, and data, may be referred to in some contexts as a rules engine test harness.
  • the network 110 may or may not be considered as a part of the rules engine test harness.
  • the system 100 also comprises a server 118 and a rules engine 120 .
  • the test execution component 106 then invokes the server 118 to execute the test cases.
  • the computer system 102 , the workstation 108 , and the server 118 may each be implemented as a general-purpose computer system, which is discussed in greater detail hereinafter.
  • the builder component 104 may comprise one or more components.
  • the builder component 104 promotes defining a plurality of test cases and a plurality of test scenarios for testing rules, for example using the builder interface 126 on the workstation 108 .
  • the builder interface 126 may be a web interface that is provided by the builder component 104 , for example as a hypertext mark-up language (HTML) page transmitted from the builder component 104 to the workstation 108 .
  • the test cases and test scenarios may be stored in the database 112 .
  • a test case 114 identifies a set of inputs, a rule, and a set of expected results from executing the rule based on the set of inputs.
  • a rule may comprise a condition portion and an action portion.
  • the condition may comprise a left hand variable associated with a right hand value by a generic template.
  • the condition may comprise a left hand operand associated with a right hand operand by an operator.
  • a rule may comprise a condition that evaluates the enrollment of an individual in a healthcare management information system and an action that indicates the rule succeeded if the condition evaluates to TRUE and sets an exception code if the condition evaluated to FALSE.
  • a test scenario 116 comprises an ordered sequence of test cases 114 . Each test scenario 116 may be used to determine the correct operability of one or more rules.
  • the set of inputs associated with a test case 114 may comprise a number of different input parameters and the values associated with those different input parameters.
  • the set of expected results may comprise an output parameter and the values associated with the output parameter.
  • as many as 6,000 or more rules may be deployed to be executed by a rules engine in an enterprise function, for example a portion of a healthcare management information system.
  • a representative set of test cases 114 adequate to provide an acceptable level of confidence in the rules may exceed 100,000 test cases 114 .
  • the builder component 104 provides drop down menus for composing test cases 114 , for example in a web interface provided to the workstation 108 .
  • the builder interface 126 may be a web interface that is provided by the builder component 104 , for example as an HTML page transmitted from the builder component 104 to the workstation 108 .
  • the drop down menus may comprise drop down menus for selecting invocation points and/or entry points, for defining inputs, and for defining results, for example rule return object contents and/or values.
  • the builder component 104 may provide a way to begin defining a new test case 114 by first cloning or copying a previously defined test case 114 .
  • the new test case 114 is then changed in some manner, for example by specifying a different value for an input parameter and a different expected value of the rule return object.
  • the builder component 104 also provides means for composing a test scenario 116 as an ordered sequence of test cases 114 .
  • the builder component 104 writes test cases 114 to the test cases 114 in the database 112 and writes test scenarios 116 to the test scenarios 116 in the database 112 .
  • the builder component 104 writes test cases 114 and test scenarios 116 to the database 112 by using a test case data access object (DAO).
  • DAO test case data access object
  • the builder component 104 may be referred to as a test case editor.
  • the test execution component 106 simulates one or more service applications that the rules engine 120 interacts with when executing rules.
  • the test execution component 106 may simulate a portal server processor, a plurality of databases, a plurality of database management applications, a messaging application, and other applications and/or services.
  • the test execution component 106 also invokes execution of rules on the rules engine 120 according to the test cases 114 and/or the test scenarios 116 stored in the database 112 , for example under the control of a web interface on the workstation 108 .
  • the test control interface 128 may be a web interface that is provided by the builder component 104 , for example as a HTML page transmitted from the builder component 104 to the workstation 108 .
  • the test execution component 106 may read the test cases 114 and/or test scenarios 116 from the database 112 and, based on the content of the test cases 114 and/or test scenarios 116 , invoke the execution of specific rules with appropriate input parameter values specified by the test cases 114 and/or test scenarios 116 .
  • the test execution component 106 receives the results of the execution of the rules by the rules engine 120 , for example receiving a rules return object containing SUCCESS and/or exception code values.
  • the test execution component 106 saves the results of executing the test cases 114 and/or the test scenarios 116 in the test results 122 of the database 112 .
  • the test execution component 106 compares the rule return object received from the rules engine 120 to the expected rule return object defined in the test cases 114 and generates a report based on this comparison.
  • the test execution component 106 may compare the rule return object from the current execution of a test case 114 with the expected result that was optionally provided as input to the test case. If the expected result is not given as an input, the result will be marked as not available (N/A) since there is no data for the comparison.
  • the report may comprise a summary of numbers of test cases 114 that passed, numbers of test cases 114 that failed, numbers of test scenarios 116 that passed, numbers of test scenarios 116 that failed.
  • the report may comprise trend analysis of pass/fail statistics over a plurality of testing sessions.
  • the test execution component 106 stores the report in the success/fail report 124 in the database 112 .
  • the test execution component 106 may communicate results information to the database 112 at least in part using a reporting data access object.
  • the rule return object may be referred to as results and/or test results.
  • the test execution component 106 may optionally perform timing of the execution of one or more of the test cases 114 , for example test cases 114 that include an elapsed time of execution expected value.
  • a test case 114 may be directed to testing the timeliness of a response to an operator command input.
  • the test execution component 106 may store the timing results along with other test results in the test results 122 of the database 112 .
  • comparisons among timing results for executing the same or similar test cases 114 stored in the test results 122 of the database 112 may be performed to determine a trend of performance of the rules engine 120 .
  • the network 110 may comprise any combination of private networks and public networks.
  • the network 110 may comprise electrical or optical wired links as well as wireless links.
  • the network 110 may be confined to a single facility or campus, but in another embodiment, the network 110 may be distributed across a wide geographical area, including across the entire globe.
  • the rules engine 120 may include some portions of a commercial off the shelf (COTS) rules engine tool.
  • COTS rules engine tool may be customized and/or encapsulated by a wrapper or a rules engine framework to provide functionality and convenience otherwise not provided by the COTS rules engine tool.
  • a Fair Isaac BLAZE ADVISOR COTS rules engine package may be encapsulated in the rules engine 120 .
  • the rules engine 120 may extend the COTS rules engine tool, for example, by providing automated input value validation between a left hand variable and a right hand input.
  • the rules engine 120 also may feature other extensions of the COTS rules engine tool.
  • a test builder interface comprising a list of rules to be tested.
  • the tested rules may be all of the rules or only some of the rules.
  • the test builder interface 126 may be provided as a web interface that may execute on the workstation 108 .
  • the list of rules and/or entry points may be provided as one or more drop down menus in the test builder interface 126 .
  • a plurality of test cases 114 are defined using the test builder interface. Defining test cases 114 may comprise identifying a specific rule, an entry point, one or more input parameter values associated with the rule, and one or more expected output parameter values associated with executing the rule based on the input parameter values.
  • the rule test cases 114 and test scenarios 116 defined at block 208 above are stored, e.g., in the database 112 .
  • a plurality of test case designers may be concurrently defining test cases 114 and test scenarios 116 by executing the test builder interface on separate workstations 108 , and may be concurrently storing the test cases 114 and the test scenarios 116 in the database 112 .
  • the rules are invoked on the rules engine 120 based on the test cases 114 and based on the test scenarios 116 .
  • a single invocation point may be invoked at one time, providing appropriate input parameter values based on the test case 114 .
  • a plurality of invocation points may be serially invoked by a test scenario 116 , thereby providing appropriate input parameter values based on the test cases 114 composing the test scenario 116 .
  • the rules engine 120 may access the rule and/or rules out of a rule repository database (not shown), or the rule definition may be provided along with the input parameter values.
  • the rules are precompiled and/or preprocessed into a binary file format that is stored on the server 118 , for example in Fair Isaac binary (ADB) file format.
  • ADB Fair Isaac binary
  • one or more service applications are simulated by the test execution component 106 .
  • the rules engine 120 may interact with the service applications during the course of executing the rule under test.
  • the service applications may comprise a portal server processor, a database management system, a messaging service, and other services.
  • the elapsed time of execution of the rule processing by the rules engine 120 is optionally timed by the test execution component 106 .
  • the results of executing the rule and/or rules by the rules engine 120 for example output parameter values and optionally elapsed time information is collected.
  • the results of executing the rule and/or rules are stored in the database 112 .
  • the results may also be displayed by an interface, for example a web interface that may execute on the workstation 108 .
  • a success/fail report is optionally generated based on the results of executing the rule and/or rules.
  • the expected output parameter values defined in the test cases 114 are compared to the results to determine whether the subject test cases 114 succeeded or failed.
  • Summary statistics for example total numbers of successes and failures, may be aggregated and placed in the success/fail report 124 .
  • the success/fail report 124 may also include information comparing the current test session results with earlier test results to report a test success trend.
  • the success/fail report 124 may be stored in the database 112 and may be displayed by an interface, for example a web interface that may execute on the workstation 108 . The process 200 then exits.
  • FIG. 3 illustrates a typical, general-purpose computer system suitable for implementing one or more embodiments disclosed herein.
  • the computer system 380 includes a processor 382 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage 384 , read only memory (ROM) 386 , random access memory (RAM) 388 , input/output (I/O) devices 390 , and network connectivity devices 392 .
  • the processor 382 may be implemented as one or more CPU chips.
  • the secondary storage 384 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 388 is not large enough to hold all working data. Secondary storage 384 may be used to store programs that are loaded into RAM 388 when such programs are selected for execution.
  • the ROM 386 is used to store instructions and perhaps data that are read during program execution. ROM 386 is a non-volatile memory device, which typically has a small memory capacity relative to the larger memory capacity of secondary storage 384 .
  • the RAM 388 is used to store volatile data and perhaps to store instructions. Access to both ROM 386 and RAM 388 is typically faster than to secondary storage 384 .
  • I/O devices 390 may include printers, video monitors, liquid crystal displays (LCDs), touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
  • LCDs liquid crystal displays
  • touch screen displays keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
  • the network connectivity devices 392 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards such as code division multiple access (CDMA), global system for mobile communications (GSM), and/or worldwide interoperability for microwave access (WiMAX) radio transceiver cards, and other well-known network devices.
  • These network connectivity devices 392 may enable the processor 382 to communicate with an Internet or one or more intranets. With such a network connection, it is contemplated that the processor 382 might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed using processor 382 , may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.
  • Such information may be received from and outputted to the network, for example, in the form of a computer data baseband signal or signal embodied in a carrier wave.
  • the baseband signal or signal embodied in the carrier wave generated by the network connectivity devices 392 may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media, for example optical fiber, or in the air or free space.
  • the information contained in the baseband signal or signal embedded in the carrier wave may be ordered according to different sequences, as may be desirable for either processing or generating the information or transmitting or receiving the information.
  • the baseband signal or signal embedded in the carrier wave, or other types of signals currently used or hereafter developed, referred to herein as the transmission medium may be generated according to several methods well known to one skilled in the art.
  • the processor 382 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 384 ), ROM 386 , RAM 388 , or the network connectivity devices 392 . While only one processor 382 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor 382 , the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors 382 .

Abstract

A system is provided. The system comprises a computer system, a builder component, and a test execution component. When executed by the computer system, the builder component promotes defining a plurality of test cases and a plurality of test scenarios. When executed by the computer system, the text execution component simulates at least one service application, invokes execution of plurality of rules on a rules engine with an input based on one of the test cases defined using the builder component, wherein the rules engine interacts with the simulated service application, and stores the result of the rule execution in a database.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application claims priority to U.S. Provisional Application Ser. No. 61/048,810 filed Apr. 29, 2008, and entitled “MMIS Health Enterprise Solution,” by Jack Devos, et al., which is incorporated herein by reference for all purposes.
  • STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
  • Not applicable.
  • REFERENCE TO A MICROFICHE APPENDIX
  • Not applicable.
  • BACKGROUND
  • The United States Medicaid program was enacted in 1965 to provide a medical assistance program for individuals and families with low incomes. The Medicaid program comprises three main entities—the patients, the healthcare providers, and the agency administering the plan (i.e., the payer). The Medicaid program is financed through joint federal and state funding. The Medicaid program is administered by each state according to an approved state plan. The specifics of the Medicaid program differ from state to state. Differences may include covered healthcare procedures, allowable procedure costs, and patient eligibility criteria. The state administrators of the Medicaid program are required to have a Medicaid management information system (MMIS) that provides for mechanized and/or computerized Medicaid claims processing. Recently, the Medicaid information technology architecture (MITA) has been promulgated by the U.S. government to provide a blueprint and a set of standards that individual states are to follow in administering the Medicaid program and for developing the next generation MMIS.
  • Rules engines may be pluggable software components that execute rules that have been externalized from application code. The rules define business rules and/or business logic that may change frequently. Typically, rules may be defined by nonprogrammers and may be provided to the rules engine in the form of data or data files. Using a rules engine to provide business rules to an application may reduce time to market and reduce total cost of ownership, with reference to the alternative of encoding the business logic in high level programming language code.
  • Automation designs, for example business rules to be executed by a rules engine and/or applications encoded in a high level programming language, are typically tested extensively before deployment in customer facing products and/or systems. Testing may take the form of executing a number of test cases on the target computer systems, where each test case comprises a single set of inputs to the subject rules engine and/or application. In addition to defining the inputs, the test cases may define the expected output and/or results of executing the rules engine and/or the application on the inputs. Because testing every possible combination of inputs is typically infeasible, an attempt is often made to identify a representative selection of test cases that may provide sufficient confidence in the automation design under test. Testing automation designs may be an expensive process that consumes substantial business resources including personnel, equipment, and schedule time. Insufficient testing can result in deployment of a flawed automation design that may fail inopportunely, possibly damaging the enterprise brand and/or incurring liability.
  • SUMMARY
  • In an embodiment, a system is provided. The system comprises a computer system, a builder component, and a test execution component. When executed by the computer system, the builder component promotes defining a plurality of test cases and a plurality of test scenarios. When executed by the computer system, the text execution component simulates at least one service application, invokes execution of a plurality of rules on a rules engine with an input based on one of the test cases defined using the builder component, wherein the rules engine interacts with the simulated service application, and stores the result of the rule execution in a database.
  • In another embodiment, a method is provided. The method comprises providing a builder interface for building a plurality of test cases and a plurality of test scenarios, the interface comprising a list of rules and a plurality of input controls for defining a plurality of inputs for the test cases. The method also comprises storing the test cases and the test scenarios in a database, invoking the rules on a rules engine based on at least one of the test cases and the test scenarios, and simulating at least one service application, wherein the rules engine interacts with the service application when executing the rules. The method also comprises collecting the results of the invocation of the rules on the rules engine and storing the results in the database.
  • In another embodiment, an apparatus is provided. The apparatus comprises a test case editor and a rule test manager. When executed on a first computer system, the test case editor provides an environment for building a plurality of rule test cases by selecting an entry point from a list of available entry points, by defining a plurality of inputs for the rule, and by defining expected results for one of the test cases. When executed on the first computer system, the rule test manager executes the rule test cases by simulating at least one service program, invoking a plurality of rules on a rules engine based on the rule test cases built using the test case editor, receives results from the rules engine, and stores the results, wherein the rules engine interacts with the simulated service program when executing the rules.
  • These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.
  • FIG. 1 is a schematic diagram of a rules engine test harness according to an embodiment of the disclosure.
  • FIG. 2 is a flowchart of a method of testing rules according to an embodiment of the disclosure.
  • FIG. 3 is a schematic diagram of an exemplary general-purpose computer system suitable for implementing some aspects of the several embodiments of the disclosure.
  • DETAILED DESCRIPTION
  • It should be understood at the outset that although illustrative implementations of one or more embodiments are illustrated below, the disclosed systems and methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, but may be modified within the scope of the appended claims along with their full scope of equivalents.
  • A rules engine test harness for testing rules is disclosed. The rules engine test harness may promote testing rules without using a fully configured and/or duplicate execution environment. In an embodiment, a rules engine framework is used to create and optionally to compile rules that may be executed by a rules engine. Further details about a rules engine framework are provided in U.S. patent application Ser. No. 12/257,782 filed Oct. 24, 2008 entitled “Rules Engine Framework” by Uma Kandasamy et al., which is hereby incorporated by reference in its entirety. When deployed as a released product, the rules may be executed by a rules engine on a process server computer system in cooperation and coordination with a business process. In an embodiment, the business process and rules engine cooperatively provide at least a portion of a healthcare management information system. A user of the healthcare management information system may interact with the business process and rules engine by entering information and selecting actions from a user interface presented at a workstation communicatively coupled to the process server computer system. The business process may promote enrolling healthcare providers into the healthcare system, for example enrolling physicians, therapists, hospitals, minor emergency medical centers, and other healthcare providers. The business process may promote enrolling healthcare recipients and/or patients. The business process may promote receiving and processing claims from enrolled healthcare providers for services provided to enrolled healthcare recipients.
  • A function of the business process may be invoked by the user of the workstation, for example processing of a claim for healthcare services. The business process may process this claim by invoking a series of rules and/or a rule flow on the rules engine. The use of rules to provide automated processing may have the advantage of allowing for the rapid creation, modification, and deployment of rules with respect to processing based on specifically designed computer programs. The processing of the claim may involve the rules engine executing a sequence of rules that validate the enrollment of the healthcare provider, validate the enrollment of the healthcare recipient, determine a coverage status of the subject procedure, determines a maximum coverage amount, and other such processes. In an embodiment, the rules engine may return a rule result object to the business process that invoked the rules engine, wherein the rule result object may identify one or more exceptions if something went wrong, or no exceptions if the rule processed successfully.
  • Turning now to FIG. 1, a system 100 for rules testing is discussed. The system 100 comprises a computer system 102, a workstation 108, a network 110, and a database 112. The computer system 102 comprises a builder component 104 and a test execution component 106. The workstation 108 may execute a builder interface 126 and a test control interface 128. The database 112 may also be referred to as a test case repository, and may comprise a plurality of test cases 114, a plurality of test scenarios 116, a plurality of test results 122, and a plurality of success/fail reports 124. It is understood that when the system 100 is first placed into service, the database 112 may be substantially empty and the test cases 114, the test scenarios 116, the test results 122, and the success/fail reports 124 may accumulate in number over time. While the computer system 102 in FIG. 1 is shown to be in direct communication with the database 112, in some embodiments the computer system 102 may communicate with the database 112 via the network 110. The computer system 102, the workstation 108, and the database 112, along with their contained components, applications, and data, may be referred to in some contexts as a rules engine test harness. The network 110 may or may not be considered as a part of the rules engine test harness. The system 100 also comprises a server 118 and a rules engine 120. The test execution component 106 then invokes the server 118 to execute the test cases. The computer system 102, the workstation 108, and the server 118 may each be implemented as a general-purpose computer system, which is discussed in greater detail hereinafter.
  • The builder component 104 may comprise one or more components. The builder component 104 promotes defining a plurality of test cases and a plurality of test scenarios for testing rules, for example using the builder interface 126 on the workstation 108. The builder interface 126 may be a web interface that is provided by the builder component 104, for example as a hypertext mark-up language (HTML) page transmitted from the builder component 104 to the workstation 108. Likewise, the test cases and test scenarios may be stored in the database 112. As used herein, a test case 114 identifies a set of inputs, a rule, and a set of expected results from executing the rule based on the set of inputs. In an embodiment, a rule may comprise a condition portion and an action portion. When the condition evaluates TRUE when executed by the rules engine 120, the action is performed. The condition may comprise a left hand variable associated with a right hand value by a generic template. Worded differently, the condition may comprise a left hand operand associated with a right hand operand by an operator. As an example, a rule may comprise a condition that evaluates the enrollment of an individual in a healthcare management information system and an action that indicates the rule succeeded if the condition evaluates to TRUE and sets an exception code if the condition evaluated to FALSE. As used herein, a test scenario 116 comprises an ordered sequence of test cases 114. Each test scenario 116 may be used to determine the correct operability of one or more rules.
  • The set of inputs associated with a test case 114 may comprise a number of different input parameters and the values associated with those different input parameters. Similarly, the set of expected results may comprise an output parameter and the values associated with the output parameter. For exemplary purposes, a simple test case 114 may comprise pairs of parameter names and input values and a result as follows (member identity=1234567, age=67, procedure code=2007, provider identity=7654321, expected action=SUCCESS). An alternative simple test case 114 may comprise pairs of parameter names and input values in the form of XML and a result in XML as well (member identity=1234567, age=−2, procedure code=2007, provider identity=7654321, expected action=set exception code 1234). In some embodiments, as many as 6,000 or more rules may be deployed to be executed by a rules engine in an enterprise function, for example a portion of a healthcare management information system. A representative set of test cases 114 adequate to provide an acceptable level of confidence in the rules may exceed 100,000 test cases 114.
  • In an embodiment, the builder component 104 provides drop down menus for composing test cases 114, for example in a web interface provided to the workstation 108. The builder interface 126 may be a web interface that is provided by the builder component 104, for example as an HTML page transmitted from the builder component 104 to the workstation 108. The drop down menus may comprise drop down menus for selecting invocation points and/or entry points, for defining inputs, and for defining results, for example rule return object contents and/or values. In an embodiment, the builder component 104 may provide a way to begin defining a new test case 114 by first cloning or copying a previously defined test case 114. The new test case 114 is then changed in some manner, for example by specifying a different value for an input parameter and a different expected value of the rule return object. The builder component 104 also provides means for composing a test scenario 116 as an ordered sequence of test cases 114. The builder component 104 writes test cases 114 to the test cases 114 in the database 112 and writes test scenarios 116 to the test scenarios 116 in the database 112. In some embodiments, the builder component 104 writes test cases 114 and test scenarios 116 to the database 112 by using a test case data access object (DAO). In some contexts, the builder component 104 may be referred to as a test case editor.
  • The test execution component 106 simulates one or more service applications that the rules engine 120 interacts with when executing rules. For example, the test execution component 106 may simulate a portal server processor, a plurality of databases, a plurality of database management applications, a messaging application, and other applications and/or services. The test execution component 106 also invokes execution of rules on the rules engine 120 according to the test cases 114 and/or the test scenarios 116 stored in the database 112, for example under the control of a web interface on the workstation 108. The test control interface 128 may be a web interface that is provided by the builder component 104, for example as a HTML page transmitted from the builder component 104 to the workstation 108. The test execution component 106 may read the test cases 114 and/or test scenarios 116 from the database 112 and, based on the content of the test cases 114 and/or test scenarios 116, invoke the execution of specific rules with appropriate input parameter values specified by the test cases 114 and/or test scenarios 116.
  • Additionally, the test execution component 106 receives the results of the execution of the rules by the rules engine 120, for example receiving a rules return object containing SUCCESS and/or exception code values. The test execution component 106 saves the results of executing the test cases 114 and/or the test scenarios 116 in the test results 122 of the database 112. In an embodiment, the test execution component 106 compares the rule return object received from the rules engine 120 to the expected rule return object defined in the test cases 114 and generates a report based on this comparison. The test execution component 106 may compare the rule return object from the current execution of a test case 114 with the expected result that was optionally provided as input to the test case. If the expected result is not given as an input, the result will be marked as not available (N/A) since there is no data for the comparison.
  • The report may comprise a summary of numbers of test cases 114 that passed, numbers of test cases 114 that failed, numbers of test scenarios 116 that passed, numbers of test scenarios 116 that failed. The report may comprise trend analysis of pass/fail statistics over a plurality of testing sessions. The test execution component 106 stores the report in the success/fail report 124 in the database 112. In an embodiment, the test execution component 106 may communicate results information to the database 112 at least in part using a reporting data access object. In some contexts, the rule return object may be referred to as results and/or test results.
  • In an embodiment, the test execution component 106 may optionally perform timing of the execution of one or more of the test cases 114, for example test cases 114 that include an elapsed time of execution expected value. For example, a test case 114 may be directed to testing the timeliness of a response to an operator command input. The test execution component 106 may store the timing results along with other test results in the test results 122 of the database 112. In an embodiment, comparisons among timing results for executing the same or similar test cases 114 stored in the test results 122 of the database 112 may be performed to determine a trend of performance of the rules engine 120.
  • The network 110 may comprise any combination of private networks and public networks. The network 110 may comprise electrical or optical wired links as well as wireless links. In an embodiment, the network 110 may be confined to a single facility or campus, but in another embodiment, the network 110 may be distributed across a wide geographical area, including across the entire globe.
  • The rules engine 120 may include some portions of a commercial off the shelf (COTS) rules engine tool. The COTS rules engine tool may be customized and/or encapsulated by a wrapper or a rules engine framework to provide functionality and convenience otherwise not provided by the COTS rules engine tool. In an embodiment, a Fair Isaac BLAZE ADVISOR COTS rules engine package may be encapsulated in the rules engine 120. In an embodiment, the rules engine 120 may extend the COTS rules engine tool, for example, by providing automated input value validation between a left hand variable and a right hand input. The rules engine 120 also may feature other extensions of the COTS rules engine tool.
  • Turning now to FIG. 2, a method 200 is described. At block 204, a test builder interface is provided comprising a list of rules to be tested. The tested rules may be all of the rules or only some of the rules. The test builder interface 126 may be provided as a web interface that may execute on the workstation 108. The list of rules and/or entry points may be provided as one or more drop down menus in the test builder interface 126. At block 208, a plurality of test cases 114 are defined using the test builder interface. Defining test cases 114 may comprise identifying a specific rule, an entry point, one or more input parameter values associated with the rule, and one or more expected output parameter values associated with executing the rule based on the input parameter values. At block 212, the rule test cases 114 and test scenarios 116 defined at block 208 above are stored, e.g., in the database 112. In an embodiment, a plurality of test case designers may be concurrently defining test cases 114 and test scenarios 116 by executing the test builder interface on separate workstations 108, and may be concurrently storing the test cases 114 and the test scenarios 116 in the database 112.
  • At block 216, the rules are invoked on the rules engine 120 based on the test cases 114 and based on the test scenarios 116. A single invocation point may be invoked at one time, providing appropriate input parameter values based on the test case 114. Alternatively, a plurality of invocation points may be serially invoked by a test scenario 116, thereby providing appropriate input parameter values based on the test cases 114 composing the test scenario 116. The rules engine 120 may access the rule and/or rules out of a rule repository database (not shown), or the rule definition may be provided along with the input parameter values. In an embodiment, the rules are precompiled and/or preprocessed into a binary file format that is stored on the server 118, for example in Fair Isaac binary (ADB) file format.
  • At block 220, one or more service applications are simulated by the test execution component 106. The rules engine 120 may interact with the service applications during the course of executing the rule under test. The service applications may comprise a portal server processor, a database management system, a messaging service, and other services. At block 224, the elapsed time of execution of the rule processing by the rules engine 120 is optionally timed by the test execution component 106. At block 228, the results of executing the rule and/or rules by the rules engine 120, for example output parameter values and optionally elapsed time information is collected. At block 232, the results of executing the rule and/or rules are stored in the database 112. The results may also be displayed by an interface, for example a web interface that may execute on the workstation 108.
  • At block 236, a success/fail report is optionally generated based on the results of executing the rule and/or rules. The expected output parameter values defined in the test cases 114 are compared to the results to determine whether the subject test cases 114 succeeded or failed. Summary statistics, for example total numbers of successes and failures, may be aggregated and placed in the success/fail report 124. The success/fail report 124 may also include information comparing the current test session results with earlier test results to report a test success trend. The success/fail report 124 may be stored in the database 112 and may be displayed by an interface, for example a web interface that may execute on the workstation 108. The process 200 then exits.
  • Some aspects of the system described above, for example the computer system 102, the workstation 108, and the server 118, may be implemented on any general-purpose computer with sufficient processing power, memory resources, and network throughput capability to handle the necessary workload placed upon it. FIG. 3 illustrates a typical, general-purpose computer system suitable for implementing one or more embodiments disclosed herein. The computer system 380 includes a processor 382 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage 384, read only memory (ROM) 386, random access memory (RAM) 388, input/output (I/O) devices 390, and network connectivity devices 392. The processor 382 may be implemented as one or more CPU chips.
  • The secondary storage 384 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 388 is not large enough to hold all working data. Secondary storage 384 may be used to store programs that are loaded into RAM 388 when such programs are selected for execution. The ROM 386 is used to store instructions and perhaps data that are read during program execution. ROM 386 is a non-volatile memory device, which typically has a small memory capacity relative to the larger memory capacity of secondary storage 384. The RAM 388 is used to store volatile data and perhaps to store instructions. Access to both ROM 386 and RAM 388 is typically faster than to secondary storage 384.
  • I/O devices 390 may include printers, video monitors, liquid crystal displays (LCDs), touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
  • The network connectivity devices 392 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards such as code division multiple access (CDMA), global system for mobile communications (GSM), and/or worldwide interoperability for microwave access (WiMAX) radio transceiver cards, and other well-known network devices. These network connectivity devices 392 may enable the processor 382 to communicate with an Internet or one or more intranets. With such a network connection, it is contemplated that the processor 382 might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed using processor 382, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.
  • Such information, which may include data or instructions to be executed using processor 382 for example, may be received from and outputted to the network, for example, in the form of a computer data baseband signal or signal embodied in a carrier wave. The baseband signal or signal embodied in the carrier wave generated by the network connectivity devices 392 may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media, for example optical fiber, or in the air or free space. The information contained in the baseband signal or signal embedded in the carrier wave may be ordered according to different sequences, as may be desirable for either processing or generating the information or transmitting or receiving the information. The baseband signal or signal embedded in the carrier wave, or other types of signals currently used or hereafter developed, referred to herein as the transmission medium, may be generated according to several methods well known to one skilled in the art.
  • The processor 382 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 384), ROM 386, RAM 388, or the network connectivity devices 392. While only one processor 382 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor 382, the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors 382.
  • While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted or not implemented.
  • In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.

Claims (20)

1. A system, comprising:
a computer system;
a builder component that, when executed by the computer system, promotes defining a plurality of test cases and a plurality of test scenarios; and
a test execution component that, when executed by the computer system, simulates at least one service application, invokes execution of a plurality of rules on a rules engine with an input based on one of the test cases defined using the builder component, wherein the rules engine interacts with the simulated service application, and stores the result of the rule execution in a results file.
2. The system of claim 1, wherein the test execution component further times execution of rules, rule sets, and rule flows and stores the execution time results and the result of the rule execution in the results file.
3. The system of claim 1, wherein defining test cases and test scenarios further comprises defining expected results of the test cases and the test scenarios.
4. The system of claim 1, wherein test scenarios are defined using the builder component as an ordered sequence of at least two test cases.
5. The system of claim 1, wherein the service application comprises at least one of a data base management system, a message service, and a portal server.
6. The system of claim 1, further comprising a database, wherein the builder component stores test cases and test scenarios in the database and the test execution component stores the results file in the database.
7. A method, comprising:
providing a builder interface for building a plurality of test cases and a plurality of test scenarios, the interface comprising a list of rules and a plurality of input controls for defining a plurality of inputs for the test cases;
storing the test cases and the test scenarios in a database;
invoking the rules on a rules engine based on at least one of the test cases and the test scenarios;
simulating at least one service application, wherein the rules engine interacts with the service application when executing the rules;
collecting the results of the invocation of the rules on the rules engine; and
storing the results in the database.
8. The method of testing rules of claim 7, further comprising timing the completion of the rules, wherein collecting the results of the invocation of the rules comprises collecting the timing results.
9. The method of testing rules of claim 7, further comprising comparing the results of the invocation of the rules on the rules engine with results of a previous invocation of the rules on the rules engine stored in the database.
10. The method of testing rules of claim 7, wherein the builder interface generates extensible markup language files based on user inputs defining test cases and test scenarios and wherein storing the test cases and test scenarios in the database comprises storing the extensible markup language files in the database.
11. The method of testing rules of claim 7, wherein the builder interface further comprises input controls for defining expected results of test cases.
12. The method of testing rules of claim 7, further comprising generating a success and fail report and storing the success and fail report in the database, wherein revising is further based on the success and fail report.
13. The method of claim 12, further comprising timing the completion of the rules, wherein collecting the results of the invocation of the rules comprises collecting the timing results and revising is further based on the timing results.
14. An apparatus, comprising:
a test case editor, executable on a first computer system, that provides an environment for building a plurality of rule test cases by selecting an entry point from a list of available entry points, by defining a plurality of inputs for the rule, and by defining expected results for one of the test cases; and
a rule test manager, executable on the first computer system, that executes the rule test cases by simulating at least one service program, invoking a plurality of rules on a rules engine based on the rule test cases built using the test case editor, receives results from the rules engine, and stores the results, wherein the rules engine interacts with the simulated service program when executing the rules.
15. The apparatus of claim 14, wherein the rules engine executes on a second computer system and the rules test manager invokes the rules on the rules engine using JAVA remote method invocation.
16. The apparatus of claim 14, wherein the test case editor comprises a web interface.
17. The apparatus of claim 14, further comprising a database wherein the test case editor stores test cases in the database and the rule test manager stores the results in the database.
18. The apparatus of claim 17, wherein the test case editor communicates with the database using a test case data access object and the rule test manager communicates with the database using a reporting data access object.
19. The apparatus of claim 14, wherein the at least one service program is one of a database management system and a message system.
20. The apparatus of claim 14, wherein the rule test cases comprise a plurality of healthcare claims.
US12/323,707 2008-04-29 2008-11-26 Rules engine test harness Abandoned US20090271351A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/323,707 US20090271351A1 (en) 2008-04-29 2008-11-26 Rules engine test harness

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US4881008P 2008-04-29 2008-04-29
US12/323,707 US20090271351A1 (en) 2008-04-29 2008-11-26 Rules engine test harness

Publications (1)

Publication Number Publication Date
US20090271351A1 true US20090271351A1 (en) 2009-10-29

Family

ID=41215889

Family Applications (3)

Application Number Title Priority Date Filing Date
US12/257,782 Abandoned US20090271214A1 (en) 2008-04-29 2008-10-24 Rules engine framework
US12/323,707 Abandoned US20090271351A1 (en) 2008-04-29 2008-11-26 Rules engine test harness
US12/416,746 Active 2030-01-23 US8103683B2 (en) 2008-04-29 2009-04-01 Converting between software objects

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US12/257,782 Abandoned US20090271214A1 (en) 2008-04-29 2008-10-24 Rules engine framework

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12/416,746 Active 2030-01-23 US8103683B2 (en) 2008-04-29 2009-04-01 Converting between software objects

Country Status (1)

Country Link
US (3) US20090271214A1 (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090271214A1 (en) * 2008-04-29 2009-10-29 Affiliated Computer Services, Inc. Rules engine framework
US7836346B1 (en) * 2007-06-11 2010-11-16 Oracle America, Inc. Method and system for analyzing software test results
US20110072027A1 (en) * 2009-09-22 2011-03-24 Siemens Product Lifecycle Management Software Inc. System and method for customized file comparison
US20130042222A1 (en) * 2011-08-08 2013-02-14 Computer Associates Think, Inc. Automating functionality test cases
US8429391B2 (en) 2010-04-16 2013-04-23 Micron Technology, Inc. Boot partitions in memory devices and systems
US8660861B2 (en) 2012-01-24 2014-02-25 Fmr Llc Allocation of financial incentives for employee wellness programs
US20140236662A1 (en) * 2013-02-15 2014-08-21 Mckesson Financial Holdings Apparatuses, systems, and methods for providing a rules engine system
US20150249823A1 (en) * 2014-02-28 2015-09-03 Airbus Helicopters Method of testing an electronic system
US20150278059A1 (en) * 2008-06-04 2015-10-01 Oracle International Corporation System and method for supporting a sliding window for testing an event processing system
US9208045B2 (en) 2011-03-03 2015-12-08 Hewlett-Packard Development Company, L.P. Testing integrated business systems
US10102091B2 (en) 2008-06-04 2018-10-16 Oracle International Corporation System and method for supporting a testing framework for an event processing system using multiple input event streams
US10146678B2 (en) 2014-05-15 2018-12-04 Oracle International Corporation Test bundling and batching optimizations
CN111552645A (en) * 2020-04-29 2020-08-18 周赟 Open type safety compliance permeability testing system
CN112015636A (en) * 2020-07-14 2020-12-01 北京淇瑀信息科技有限公司 Decision engine testing method and device based on support vector machine and electronic equipment
US11080178B2 (en) 2018-12-28 2021-08-03 Paypal, Inc. Rules testing framework
US20220138089A1 (en) * 2020-10-30 2022-05-05 EMC IP Holding Company LLC Smart test executor
US11645195B1 (en) * 2020-02-18 2023-05-09 Wells Fargo Bank, N.A. Auto-decisioning test interface and test database for bypassing functionalities of decision engines and simulating return values
CN116340187A (en) * 2023-05-25 2023-06-27 建信金融科技有限责任公司 Rule engine migration test method and device, electronic equipment and storage medium

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055107A1 (en) * 2009-09-03 2011-03-03 Von Unwerth Catherine D Industry standards modeling systems and methods
DE102010021382A1 (en) * 2010-05-25 2011-12-01 Abb Ag Method and system for generating an integration model
US8494997B2 (en) 2010-07-20 2013-07-23 Samuel W. Bellamy, III System and method for validation of transaction data
US8505047B2 (en) 2010-11-20 2013-08-06 Motorola Solutions, Inc. Method and system for policy-based re-broadcast video on demand service
US8621074B2 (en) 2012-04-27 2013-12-31 Xerox Business Services, Llc Intelligent work load manager
US10621523B1 (en) * 2012-05-09 2020-04-14 Parvinder CHADHA Computer methods and computer systems for facilitating business rule generation and validation
US20140122518A1 (en) * 2012-10-29 2014-05-01 Hewlett-Packard Development Company, L.P. Codeless array validation
US20150032907A1 (en) * 2013-07-26 2015-01-29 Alcatel-Lucent Canada, Inc. Universal adapter with context-bound translation for application adaptation layer
US20150160938A1 (en) * 2013-12-10 2015-06-11 Fmr Llc Automation in rule engine in an integration development environment
US20150331875A1 (en) * 2014-05-16 2015-11-19 Syntel, Inc. System and method for validating integrated data recasting objects
CN110209575A (en) * 2019-05-20 2019-09-06 深圳壹账通智能科技有限公司 Test document generation method, device, electronic equipment and storage medium
CN110674047B (en) * 2019-09-26 2023-06-27 北京字节跳动网络技术有限公司 Software testing method and device and electronic equipment
CN113176995B (en) * 2021-04-28 2024-03-19 平安银行股份有限公司 Interface testing method, device, equipment and medium based on testing scene
US20230385544A1 (en) * 2022-04-19 2023-11-30 Tencent America LLC Tools for conformance of network-based media processing (nbmp) documents and entities

Citations (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5786816A (en) * 1995-10-20 1998-07-28 Araxsys, Inc. Method and apparatus for graphical user interface-based and variable result healthcare plan
US5881219A (en) * 1996-12-26 1999-03-09 International Business Machines Corporation Random reliability engine for testing distributed environments
US6039688A (en) * 1996-11-01 2000-03-21 Salus Media Inc. Therapeutic behavior modification program, compliance monitoring and feedback system
US6283761B1 (en) * 1992-09-08 2001-09-04 Raymond Anthony Joao Apparatus and method for processing and/or for providing healthcare information and/or healthcare-related information
US6343271B1 (en) * 1998-07-17 2002-01-29 P5 E.Health Services, Inc. Electronic creation, submission, adjudication, and payment of health insurance claims
US20020091968A1 (en) * 2001-01-08 2002-07-11 Donald Moreaux Object-oriented data driven software GUI automated test harness
US20030037314A1 (en) * 2001-08-01 2003-02-20 International Business Machines Corporation Method and apparatus for testing and evaluating a software component using an abstraction matrix
US20030069760A1 (en) * 2001-10-04 2003-04-10 Arthur Gelber System and method for processing and pre-adjudicating patient benefit claims
US20030233253A1 (en) * 2002-04-30 2003-12-18 Peth Thomas C. Point-of-care clinical documentation software system and associated methods
US20040078228A1 (en) * 2002-05-31 2004-04-22 Fitzgerald David System for monitoring healthcare patient encounter related information
US20050010436A1 (en) * 2003-07-08 2005-01-13 Richard Merkin Health care administration method
US20050144529A1 (en) * 2003-10-01 2005-06-30 Helmut Gotz Method for defined derivation of software tests from use cases
US6915265B1 (en) * 1997-10-29 2005-07-05 Janice Johnson Method and system for consolidating and distributing information
US20050166094A1 (en) * 2003-11-04 2005-07-28 Blackwell Barry M. Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems
US6934934B1 (en) * 1999-08-30 2005-08-23 Empirix Inc. Method and system for software object testing
US20050209893A1 (en) * 2004-03-18 2005-09-22 Nahra John S System and method for identifying and servicing medically uninsured persons
US6993748B2 (en) * 2001-10-26 2006-01-31 Capital One Financial Corporation Systems and methods for table driven automation testing of software programs
US20060052965A1 (en) * 2004-08-13 2006-03-09 International Business Machines Corporation Event driven testing method, system and program product
US7039628B2 (en) * 2004-04-21 2006-05-02 Logan Jr Carmen Portable health care history information system
US20060129992A1 (en) * 2004-11-10 2006-06-15 Oberholtzer Brian K Software test and performance monitoring system
US20060143050A1 (en) * 2004-12-27 2006-06-29 The Trizetto Group, Inc. Healthcare management system using patient profile data
US20060224719A1 (en) * 2005-03-30 2006-10-05 Integrated Informatics, Inc. Operator simulator and non-invasive interface engine
US20060235280A1 (en) * 2001-05-29 2006-10-19 Glenn Vonk Health care management system and method
US20070006037A1 (en) * 2005-06-29 2007-01-04 Microsoft Corporation Automated test case result analyzer
US20080097789A1 (en) * 2006-10-24 2008-04-24 Huffer Robert L Quality Management Of Patient Data For Health Care Providers
US20080275729A1 (en) * 2007-04-09 2008-11-06 Nina Mithi Taggart System and method for population health management
US20080312951A1 (en) * 2004-08-25 2008-12-18 Berd Herpichboehm Method for Optimizing Design Delivery and Implementation of Innovative Products in Healthcare
US20090271214A1 (en) * 2008-04-29 2009-10-29 Affiliated Computer Services, Inc. Rules engine framework
US7656802B2 (en) * 2006-11-14 2010-02-02 International Business Machines Corporation Simulating services on demand
US7917895B2 (en) * 2001-07-27 2011-03-29 Smartesoft, Inc. Automated software testing and validation system

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6047284A (en) * 1997-05-14 2000-04-04 Portal Software, Inc. Method and apparatus for object oriented storage and retrieval of data from a relational database
US6429882B1 (en) * 1999-03-15 2002-08-06 Sun Microsystems, Inc. User interface component
US7617240B2 (en) * 1999-05-04 2009-11-10 Accenture Llp Component based task handling during claim processing
US7013284B2 (en) * 1999-05-04 2006-03-14 Accenture Llp Component based interface to handle tasks during claim processing
AR030878A1 (en) * 2000-10-11 2003-09-03 Healthtrio Inc SYSTEM FOR NORMALIZING DATA AS A BASIS FOR COMMUNICATION BETWEEN PARTICIPANTS OF HEALTH CARE AND INSURERS ON A WORK NETWORK
US6931429B2 (en) * 2001-04-27 2005-08-16 Left Gate Holdings, Inc. Adaptable wireless proximity networking
US6865573B1 (en) * 2001-07-27 2005-03-08 Oracle International Corporation Data mining application programming interface
US6917944B1 (en) * 2001-08-30 2005-07-12 Cisco Technology, Inc. Method and apparatus for configuring access to a plurality of data repositories
JP2006523353A (en) * 2003-03-24 2006-10-12 シーベル システムズ,インコーポレイティド Common common object
US7480894B2 (en) * 2004-02-20 2009-01-20 International Business Machines Corporation Method and system for retaining formal data model descriptions between server-side and browser-side javascript objects
US7756882B2 (en) * 2004-10-01 2010-07-13 Microsoft Corporation Method and apparatus for elegant mapping between data models
WO2006050056A2 (en) * 2004-10-29 2006-05-11 Upstream Software, Inc. Transaction network
US7440955B2 (en) * 2005-01-14 2008-10-21 Im2, Inc Method and system to compare data objects
EP1708099A1 (en) * 2005-03-29 2006-10-04 BRITISH TELECOMMUNICATIONS public limited company Schema matching
US20070157155A1 (en) * 2005-12-30 2007-07-05 Peters Eric C System and method for software generation and execution
US7747569B2 (en) * 2006-09-22 2010-06-29 Raytheon Company Systems, methods, and language for selection and retrieval of information from databases
US7870412B2 (en) * 2006-09-28 2011-01-11 Oracle International Corporation Passing client or server instructions via synchronized data objects

Patent Citations (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6283761B1 (en) * 1992-09-08 2001-09-04 Raymond Anthony Joao Apparatus and method for processing and/or for providing healthcare information and/or healthcare-related information
US5786816A (en) * 1995-10-20 1998-07-28 Araxsys, Inc. Method and apparatus for graphical user interface-based and variable result healthcare plan
US6039688A (en) * 1996-11-01 2000-03-21 Salus Media Inc. Therapeutic behavior modification program, compliance monitoring and feedback system
US5881219A (en) * 1996-12-26 1999-03-09 International Business Machines Corporation Random reliability engine for testing distributed environments
US6915265B1 (en) * 1997-10-29 2005-07-05 Janice Johnson Method and system for consolidating and distributing information
US6343271B1 (en) * 1998-07-17 2002-01-29 P5 E.Health Services, Inc. Electronic creation, submission, adjudication, and payment of health insurance claims
US6934934B1 (en) * 1999-08-30 2005-08-23 Empirix Inc. Method and system for software object testing
US20020091968A1 (en) * 2001-01-08 2002-07-11 Donald Moreaux Object-oriented data driven software GUI automated test harness
US20060235280A1 (en) * 2001-05-29 2006-10-19 Glenn Vonk Health care management system and method
US7917895B2 (en) * 2001-07-27 2011-03-29 Smartesoft, Inc. Automated software testing and validation system
US20030037314A1 (en) * 2001-08-01 2003-02-20 International Business Machines Corporation Method and apparatus for testing and evaluating a software component using an abstraction matrix
US20030069760A1 (en) * 2001-10-04 2003-04-10 Arthur Gelber System and method for processing and pre-adjudicating patient benefit claims
US6993748B2 (en) * 2001-10-26 2006-01-31 Capital One Financial Corporation Systems and methods for table driven automation testing of software programs
US20030233253A1 (en) * 2002-04-30 2003-12-18 Peth Thomas C. Point-of-care clinical documentation software system and associated methods
US20040078228A1 (en) * 2002-05-31 2004-04-22 Fitzgerald David System for monitoring healthcare patient encounter related information
US20050010436A1 (en) * 2003-07-08 2005-01-13 Richard Merkin Health care administration method
US20050144529A1 (en) * 2003-10-01 2005-06-30 Helmut Gotz Method for defined derivation of software tests from use cases
US20050166094A1 (en) * 2003-11-04 2005-07-28 Blackwell Barry M. Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems
US20050209893A1 (en) * 2004-03-18 2005-09-22 Nahra John S System and method for identifying and servicing medically uninsured persons
US7039628B2 (en) * 2004-04-21 2006-05-02 Logan Jr Carmen Portable health care history information system
US20060052965A1 (en) * 2004-08-13 2006-03-09 International Business Machines Corporation Event driven testing method, system and program product
US20080312951A1 (en) * 2004-08-25 2008-12-18 Berd Herpichboehm Method for Optimizing Design Delivery and Implementation of Innovative Products in Healthcare
US20060129992A1 (en) * 2004-11-10 2006-06-15 Oberholtzer Brian K Software test and performance monitoring system
US20060143050A1 (en) * 2004-12-27 2006-06-29 The Trizetto Group, Inc. Healthcare management system using patient profile data
US20060224719A1 (en) * 2005-03-30 2006-10-05 Integrated Informatics, Inc. Operator simulator and non-invasive interface engine
US20070006037A1 (en) * 2005-06-29 2007-01-04 Microsoft Corporation Automated test case result analyzer
US20080097789A1 (en) * 2006-10-24 2008-04-24 Huffer Robert L Quality Management Of Patient Data For Health Care Providers
US7656802B2 (en) * 2006-11-14 2010-02-02 International Business Machines Corporation Simulating services on demand
US20080275729A1 (en) * 2007-04-09 2008-11-06 Nina Mithi Taggart System and method for population health management
US20090271214A1 (en) * 2008-04-29 2009-10-29 Affiliated Computer Services, Inc. Rules engine framework

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7836346B1 (en) * 2007-06-11 2010-11-16 Oracle America, Inc. Method and system for analyzing software test results
US20090271214A1 (en) * 2008-04-29 2009-10-29 Affiliated Computer Services, Inc. Rules engine framework
US20150278059A1 (en) * 2008-06-04 2015-10-01 Oracle International Corporation System and method for supporting a sliding window for testing an event processing system
US10140196B2 (en) 2008-06-04 2018-11-27 Oracle International Corporation System and method for configuring a sliding window for testing an event processing system based on a system time
US10102091B2 (en) 2008-06-04 2018-10-16 Oracle International Corporation System and method for supporting a testing framework for an event processing system using multiple input event streams
US9892009B2 (en) * 2008-06-04 2018-02-13 Oracle International Corporation System and method for supporting a sliding window for testing an event processing system
US9753825B2 (en) 2008-06-04 2017-09-05 Oracle International Corporation System and method for using an event window for testing an event processing system
US20110072027A1 (en) * 2009-09-22 2011-03-24 Siemens Product Lifecycle Management Software Inc. System and method for customized file comparison
US8458204B2 (en) * 2009-09-22 2013-06-04 Siemens Product Lifecycle Management Software Inc. System and method for customized file comparison
US8762703B2 (en) 2010-04-16 2014-06-24 Micron Technology, Inc. Boot partitions in memory devices and systems
US9342371B2 (en) 2010-04-16 2016-05-17 Micron Technology, Inc. Boot partitions in memory devices and systems
US8429391B2 (en) 2010-04-16 2013-04-23 Micron Technology, Inc. Boot partitions in memory devices and systems
US9208045B2 (en) 2011-03-03 2015-12-08 Hewlett-Packard Development Company, L.P. Testing integrated business systems
US9990274B2 (en) 2011-03-03 2018-06-05 Entit Software Llc Testing integrated business systems
US9477583B2 (en) 2011-08-08 2016-10-25 Ca, Inc. Automating functionality test cases
US8893087B2 (en) * 2011-08-08 2014-11-18 Ca, Inc. Automating functionality test cases
US20130042222A1 (en) * 2011-08-08 2013-02-14 Computer Associates Think, Inc. Automating functionality test cases
US8660861B2 (en) 2012-01-24 2014-02-25 Fmr Llc Allocation of financial incentives for employee wellness programs
US9990598B2 (en) * 2013-02-15 2018-06-05 Allscripts Software, Llc Apparatuses, systems, and methods for providing a rules engine system
US20140236662A1 (en) * 2013-02-15 2014-08-21 Mckesson Financial Holdings Apparatuses, systems, and methods for providing a rules engine system
US9288483B2 (en) * 2014-02-28 2016-03-15 Airbus Helicopters Method of testing an electronic system
US20150249823A1 (en) * 2014-02-28 2015-09-03 Airbus Helicopters Method of testing an electronic system
US10146678B2 (en) 2014-05-15 2018-12-04 Oracle International Corporation Test bundling and batching optimizations
US10802955B2 (en) 2014-05-15 2020-10-13 Oracle International Corporation Test bundling and batching optimizations
US11080178B2 (en) 2018-12-28 2021-08-03 Paypal, Inc. Rules testing framework
US11645195B1 (en) * 2020-02-18 2023-05-09 Wells Fargo Bank, N.A. Auto-decisioning test interface and test database for bypassing functionalities of decision engines and simulating return values
CN111552645A (en) * 2020-04-29 2020-08-18 周赟 Open type safety compliance permeability testing system
CN112015636A (en) * 2020-07-14 2020-12-01 北京淇瑀信息科技有限公司 Decision engine testing method and device based on support vector machine and electronic equipment
US20220138089A1 (en) * 2020-10-30 2022-05-05 EMC IP Holding Company LLC Smart test executor
CN116340187A (en) * 2023-05-25 2023-06-27 建信金融科技有限责任公司 Rule engine migration test method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
US20090271214A1 (en) 2009-10-29
US20090271426A1 (en) 2009-10-29
US8103683B2 (en) 2012-01-24

Similar Documents

Publication Publication Date Title
US20090271351A1 (en) Rules engine test harness
Kumeno Sofware engneering challenges for machine learning applications: A literature review
US11791021B2 (en) System and method for building intuitive clinical trial applications
US6609128B1 (en) Codes table framework design in an E-commerce architecture
US6523027B1 (en) Interfacing servers in a Java based e-commerce architecture
US6601233B1 (en) Business components framework
US6633878B1 (en) Initializing an ecommerce database framework
US6718535B1 (en) System, method and article of manufacture for an activity framework design in an e-commerce based environment
US7100195B1 (en) Managing user information on an e-commerce system
US6704873B1 (en) Secure gateway interconnection in an e-commerce based environment
CN109344170B (en) Stream data processing method, system, electronic device and readable storage medium
US10079858B2 (en) Managing access in one or more computing systems
US20220318004A1 (en) System and method for predicting the impact of source code modification based on historical source code modifications
CN108647141A (en) Automatic test approach, device, computer-readable medium and electronic equipment
US20210173762A1 (en) Dynamic integration testing
CN109977012B (en) Joint debugging test method, device, equipment and computer readable storage medium of system
US11544055B2 (en) System and method for identifying source code defect introduction during source code modification
CN107526676A (en) A kind of cross-system method of testing and device
US20210117313A1 (en) Language agnostic automation scripting tool
US20070225943A1 (en) Executable application operation monitoring system
Vargas et al. Enabling real-time feedback in software engineering
Li et al. Testing machine learning systems in industry: an empirical study
Lakshmanan et al. A business centric end-to-end monitoring approach for service composites
Fagerström et al. Verdict machinery: On the need to automatically make sense of test results
Kim et al. Machine learning frameworks for automated software testing tools: a study

Legal Events

Date Code Title Description
AS Assignment

Owner name: AFFILIATED COMPUTER SERVICES, INC., TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KANDASAMY, UMA MAHESWARI;DATLA, KRISHNAM RAJU B.;REEL/FRAME:021895/0484

Effective date: 20081118

AS Assignment

Owner name: AFFILIATED COMPUTER SERVICES, LLC, TEXAS

Free format text: CHANGE OF NAME;ASSIGNOR:AFFILIATED COMPUTER SERVICES, INC.;REEL/FRAME:028994/0470

Effective date: 20111216

AS Assignment

Owner name: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT, NEW YORK

Free format text: SECURITY AGREEMENT;ASSIGNORS:XEROX COMMERCIAL SOLUTIONS, LLC (F/K/A ACS COMMERCIAL SOLUTIONS, INC.);XEROX STATE & LOCAL SOLUTIONS, INC. (F/K/A ACS STATE AND LOCAL SOLUTIONS, INC.);RSA MEDICAL LLC;AND OTHERS;REEL/FRAME:040905/0458

Effective date: 20161207

Owner name: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT

Free format text: SECURITY AGREEMENT;ASSIGNORS:XEROX COMMERCIAL SOLUTIONS, LLC (F/K/A ACS COMMERCIAL SOLUTIONS, INC.);XEROX STATE & LOCAL SOLUTIONS, INC. (F/K/A ACS STATE AND LOCAL SOLUTIONS, INC.);RSA MEDICAL LLC;AND OTHERS;REEL/FRAME:040905/0458

Effective date: 20161207

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: CONDUENT HEALTH ASSESSMENTS, LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: CONDUENT CASUALTY CLAIMS SOLUTIONS, LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: CONDUENT BUSINESS SOLUTIONS, LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: CONDUENT COMMERCIAL SOLUTIONS, LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: ADVECTIS, INC., GEORGIA

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: CONDUENT TRANSPORT SOLUTIONS, INC., NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: CONDUENT STATE & LOCAL SOLUTIONS, INC., NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015

Owner name: CONDUENT BUSINESS SERVICES, LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180

Effective date: 20211015