Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/21—Design, administration or maintenance of databases
  • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F11/00—Error detection; Error correction; Monitoring
  • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/24—Querying
  • G06F16/245—Query processing
  • G06F16/2455—Query execution
  • G06F16/24564—Applying rules; Deductive queries
  • G06F16/24565—Triggers; Constraints
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q10/00—Administration; Management
  • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Definitions

• the invention of the present disclosure relates to the field of data diagnostics. More specifically, embodiments of the present disclosure relate to a system and method for the management of a data anomaly associated with the data in a database.
• Digital databases are used to record, store and organize large sets of data.
• a large number of databases are based on manual feeding of data into a database by one or more users within an organization.
• An object of the invention is to provide a system and method for the management of anomalies in one or more data sets associated with a database.
• Another object of the present invention is to provide a solution that enables centralised monitoring and analysis of databases spread across different verticals, branches and or geographical locations of an organization.
• Another object of the invention is to provide a resource sparing approach to standardisation of rules and instructions for the management of anomalies in a database.
• Yet another object of the present invention is to enhance the quality and accuracy of one or more data sets in a database.
• Yet another object of the present invention is to increase cost-efficiency of data analysis and management of anomalies in a database, while reducing the time-duration of operations.
• Yet another object of the present disclosure is to enable users of database to receive personalised alerts regarding one or more anomalies and requisite corrective measures in data associated with the users.
• Yet another object of the present disclosure is to provide a solution to process and translate data sets in real time for anomaly management in a database.
• the one or more anomalies comprise at least one of one or more gaps, errors, inconsistencies, irregularities, incongruities and deviations leading to a degradation in quality of data.
• the method further comprises dividing the one or more sets of data into a valid data set and an anomalous data set, based on the detection of one or more anomalies in the one or more sets of data.
• the predetermined set of rules comprise at least one of a minimum threshold for a set of data to qualify as a valid data and one or more parameters for detecting the one or more anomalies.
• the predetermined set of instructions comprise one or more instructions to convert an anomalous data into a valid data.
• the database may be connected to an extract, transform, and load (ETL) tool.
• ETL extract, transform, and load
• the method further comprises converting the predetermined set of rules to one or more data flows in an extract, transform, and load (ETL) tool.
• ETL extract, transform, and load
• the detection of one or more anomalies in the one or more sets of data and identification of at least one rectification action corresponding to the one or more anomalies are performed using one or more LLM models.
• the alert corresponding to the one or more anomalies comprises an individual alert for each of the users in the subset of users.
• the transmission of the alert to a subset of users is performed based on the performance of one or more actions by an admin user.
• the method further comprises displaying, via one or more user interfaces, at least one of the valid data set, the anomalous data set and the at least one rectification action.
• Another aspect of the present disclosure may relate to a system for anomaly management in a database.
• the said system comprises a processor configured to retrieve from a database, one or more sets of data generated by a set of users and detect one or more anomalies in the one or more sets of data, based on a predetermined set of rules.
• the processor may be further configured identify at least one rectification action corresponding to the one or more anomalies, the at least one rectification action being based on a predetermined set of instructions, and thereafter, generate an alert corresponding to the one or more anomalies.
• the processor may be further configured to determine a subset of users from the set of users, wherein the subset of users is based on at least one of the one or more anomalies and the at least one rectification action, and then transmit the alert to the subset of users.
• FIG. 1 illustrates an exemplary system block diagram for anomaly management in a database, in accordance with an exemplary implementation of the present disclosure.
• FIG. 2 illustrates an exemplary method flow diagram for anomaly management in a database, in accordance with an exemplary implementation of the present disclosure.
• FIG. 3 illustrates an exemplary representation of an interface depicting translation done via the one or more large language models (LLMs), in accordance with exemplary implementations of the present disclosure.
• LLMs large language models
• FIG. 4 illustrates an exemplary representation of an interface depicting of a gender identification via the LLMs, in accordance with exemplary implementations of the present disclosure.
• FIG. 5 illustrates an exemplary representation of an interface depicting identification of a standard industry code, in accordance with exemplary implementations of the present disclosure.
• exemplary and/or “demonstrative” is used herein to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples.
• any aspect or design described herein as “exemplary” and/or “demonstrative” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.
• a ‘processing unit” or a “processor” includes processing unit, wherein processor refers to any logic circuitry for processing instructions.
• the processor may be a general-purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASIC), Field Programmable Gate Array circuits (FPGA), any other type of integrated circuits, etc.
• the processor may perform signal coding, data processing, input/output processing, and/or any other functionality that enables the working of the system according to the present disclosure. More specifically, the processor is a hardware processor.
• Storage refers to a machine or computer-readable medium including any mechanism for storing information including but not limited to text, images, audio, and video files in a form readable by a computer or similar machine.
• a computer-readable medium includes read-only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices or other types of machine-accessible storage media.
• the storage unit stores at least the data that may be required by one or more units of the server/system/user device to perform their respective functions.
• the present disclosure aims to overcome the shortcomings discussed above and other existing problems in the field of data diagnostics, specifically, for the management of data anomalies associated with the data in a database.
• the present disclosure provides a technically advanced system and method for anomaly management in a database that enables analysis of large data sets in a database simultaneously in significantly shorter time span as compared to the hiring of one or more data analysts.
• the present disclosure provides a technically advanced solution to streamline and standardise rules for data analysis, anomaly detection and management across verticals and branches of an organisations while maintaining cost and time efficiency.
• the present disclosure provides a technically advanced solution that increase data quality and data accuracy for better data-driven decision making. In part, the present disclosure enables this by providing real-time translation of data sets for consistent and accurate analysis and anomaly management.
• the present disclosure provides an advanced solution for providing generating and providing personalised alerts for data anomalies and rectification actions in response to detected anomalies.
• system [100] may comprise any other configuration of one or more components, units and/or modules as may be required to implement the features of the present disclosure, and the present description is not intended to limit the scope of the present disclosure.
• system [100] may comprise only one or more of processor [102] that may be configured to implement the features of the present disclosure independently.
• the system [100] may be implemented in a wide variety of computer electronic devices that may be configured to host one or more databases, or may be connected and configured to access one or more databases, such as a hosting server, a laptop, a desktop, a tablet etc.
• the system [100] may be directly implemented on a server on which the database exists, and therefore, the system may be configured to natively perform anomaly management in a database on the server.
• the system [100] may be implemented on a computing device that may be remotely connected to a hosting server/device.
• the computing device on which the system [100] is implemented may be connected via wired connection, such as ethemet, USB, fiber optic and the like.
• the connection may be a wireless connection using any wireless communication technology as may be known by a person skilled in the art, e.g., wide area networks (WAN), such as internet, 4G or 5G networks etc, or a local area network (LAN), such as Wi-Fi, Li-Fi, Bluetooth, etc.
• WAN wide area networks
• LAN local area network
• the processor [102] of the system [100] may be configured to perform anomaly management in a database by retrieving one or more data sets from a database.
• the one or more data sets may comprise data generated by a set of users that are associated with the database.
• an organization may utilize a customer relationship management (CRM) database to maintain all client data records, and their interactions with the organization.
• CRM customer relationship management
• the client related information may be collected by a plurality of employees during their interactions with one or more clients at different stages of operations of the organization. Accordingly, the employees that record and input the data sets, i.e., the customer related information into the database, i.e., the CRM database, then such employees may be the set of users that generate the one or more data sets in the database.
• the one or more data sets may be related to different types and/or sources of data in a database, e.g., a database may comprise a data set relating to customer relationship management, as well as a data set related to sales operations.
• a database may comprise a data set relating to customer relationship management, as well as a data set related to sales operations.
• there may be multiple databases in an organization wherein each database may correspond to a different type of data set obtained from a different source.
• an organization may maintain a vast database spanning across different verticals and/or branches of the organization, wherein the database contains an independent data set for each vertical/branch of the organization.
• the organization may employ a separate database for any of its verticals and/or branches, each of which may further comprise one or more data sets.
• it may helpful that the one or more data sets may be retrieved using an ETL tool. This enables efficient collection of the data sets from multiple sources and allows organization of the data sets for anomaly management.
• the processor [102] may be further configured to translate the one or more data sets into the preferred language of operation using a Large Language Model (LLM).
• LLM Large Language Model
• the one or more sets of data may be retrieved from five different countries to be assessed at a centralised level, and therefore the data may exist in five different languages. Therefore, upon detecting that the one or more data sets are in a language other than the configured/preferred language of operation within the system [100], the processor [102] may translate the same using one or more LLMs.
• LLM Large Language Model
• the processor [102] may be further configured to detect one or more anomalies in the one or more sets of data, wherein the detection action may be performed on the basis of a predetermined set of rules [ 108] .
• the one or more anomalies may be anomalies relating to the data in the one or more data sets. Accordingly, the one or more anomalies may indicate an occurrence that may be attributed to a degradation or a loss in the quality of data present in the one or more data sets. As such, one or more occurrences of at least one of the following may constitute an anomaly, namely, a gap, an error, an inconsistency, an irregularity or an incongruity within the data.
• a set of rules may direct that in a data object wherein the designation of the client is missing, or wherein the appropriate country code is missing for the associated phone number, then such a data object within the data set may be categorized as an anomaly. Accordingly, for the data object to be classified as a valid data object, it is not sufficient that the data object contains only the correct name and phone number of a client, but that it must also contain the correct designation and country code associated with the phone number of the client.
• an organization may have multiple branches across the globe, and hence users of different nationalities and backgrounds may be generating the one or more data sets in the organization’s database.
• the oiganization is involved in financial management and the database comprises different numerical results associated with the testing of various financial models and their outcomes.
• the certain set of users may be inclined to enter the numerical values using the Indian place-value system for commas, whereas, a different set of users may be inclined to enter the numerical values using the international place-value system for commas.
• the data may still represent the same numeric value, however, inconsistent standardization of the data may pose a problem in the usage and interpretation of data, especially in databases comprising a laige number of complex data sets.
• the predetermined set of rules [108] may comprise parameters prescribing that for a data object to be considered valid, it must be entered using the Indian-place value system. Therefore, an entry that may be made using the international -place value system may technically represent the same value, however, it may be detected as an anomaly within the data set.
• This implementation aids in the standardization of data quality across different domains, verticals and branches, hence improving the overall data-driven decision making process.
• the processor [102] may be further configured to convert the predetermined set of rules [108] into one or more data flows using the ETL tool, wherein the converted data flows may then be used as the basis for the detection of one or more anomalies in the retrieved data sets within the ETL tool.
• the processor [102] may be configured to cause the rectification unit [104] to analyze the one or more anomalies detected by the processor [102], Thereafter, the rectification unit [104] may then identify at least one rectification action required to be performed by a user to fix the one or more anomalies.
• the at least one rectification action identified by the rectification unit [104] may be based on a predetermined set of instructions [110] that may be stored in the storage [106],
• the predetermined set of instructions [110] may comprise one or more instructions required to convert an anomalous data into a valid data, or to correct one or more anomalies present in one or more data sets.
• the predetermined set of instructions [110] may include a number of outcomes to be achieved in order to turn anomalous data objects within an anomalous data set, into valid data objects.
• a rectification action based on the predetermined set of instructions [110] may comprise at least one step required to be performed by a user in order to achieve an outcome for converting an anomalous data object, in an anomalous data set, into a valid data object.
• the processor [102] may detect the anomalies in the set of data and thereafter cause the rectification unit [104] to analyze the anomalies and identify at least one rectification action on the basis of the predetermined set of instructions.
• the predetermined set of instructions [110] may comprise a corresponding outcome to turn the anomalous data set into a valid data set, and as such the outcomes may prescribe, that the anomalous data objects be changed into numbers based on the Indian place-value system.
• the at least one rectification action may comprise steps to achieve the outcome, and hence may contain steps for a user to access and navigate to the specific anomalous data objects within a data set, the specific corrections to be performed with respect to each of the data objects, saving any changes, unflagging the anomaly or notifying the admin user and/or the system [100] of the rectification actions performed to facilitate review etc. Therefore, it may be understood that the at least one rectification action may comprise additional steps to be performed, which may be identified on the basis of the desired outcomes laid down in the predetermined set of instructions [HO].
• the predetermined set of instructions [110] may be defined by an admin user, and thereafter, stored in the storage [106].
• rectification unit [ 104] may be configured to generate one or more set of instructions to convert the one or more anomalies into valid data. Thereafter, the generated set of instructions may be stored in the storage [106], in addition to the predetermined set of instructions defined by the admin user.
• the predetermined set of instructions [110] may be wholly generated by the rectification unit [104],
• the processor [102] may perform the detection of the one or more anomalies, and cause the identification of the at least one rectification action, using at least one Large Language Models (LLMs).
• LLMs Large Language Models
• the processor [102] may be further configured to cause the rectification unit [104] to generate an alert corresponding to the one or more anomalies detected.
• an alert may include a notification or an indication of the one or more anomalies detected, and at least one rectification action to rectify the one or more anomalies.
• the processor [102] may determine a subset of users for the transmission of the generated alert, from within the set of users responsible for generating the one or more data sets.
• the processor [102] may determine the subset of users on the basis the one or more anomalies detected and/or the at least one rectification action.
• the subset of users may be the users responsible for causing the one or more data anomalies.
• the subset of users may also comprise a set of users other than those responsible for causing the one or more anomalies.
• the processor [ 102] may determine that based on the type and magnitude of anomalies within the one or more sets of data, another set of users may be better equipped to perform the at least one rectification action identified for the said one or more anomalies.
• the processor [102] may determine that the subset of users responsible for causing the one or more anomalies are occupied in the rectification of prior anomalies, or another task, the processor [102] may determine that another subset of users may have a better availability to perform the rectification action.
• the determined subset of users may comprise at least one user.
• the alert may be transmitted to the subset of users in response to the performance of an action by the admin user.
• the performance of an action may comprise the admin user manually selecting when to transmit the alert to each of the subset of users.
• the system [100] may also transmit the at least one or more alerts to the user device in the form of one or more time-bound email notifications, wherein the user receives personalised email highlighting various information metrics such as, but not limited to, count of data anomalies that the user is required to rectify, user login credentials, and comments related to data anomalies.
• the scope of the present disclosure also encompasses that that the system [100] may provide one or more graphical representations of the outcomes of the anomaly management in a database to a admin user, user or any other person, via one or more user interfaces.
• the one or more graphical representations may take the form of a landing page, summary page, comparative page, recognition page, data preparation tool-based flowchart page, or email notification page.
• the one or more graphical representations may be controlled by the administrator (such as per the hierarchy in an organization, the user may be enabled to view one or more gaps on multiple levels. A junior level user may be enabled to view only his/her data gaps. Whereas a team lead user may be able to view the one or more gaps of their whole team and themselves) .
• the landing page and summary page may have one or more key performance indicators which enable the user to quickly know the count of gaps (i.e. anomalies) in each data object (i.e. data).
• the one or more graphical representations may have one or more time stamps, one or more user name (such as data owner name), one or more record type (such as account, contact, lease, meeting), one or more records of anomalies, one or more list of fields such as phone number), one or more values, one or more comments (such as brief description about what exactly is wrong in a data point), one or more links and one or more support contacts (such as email address of the administrator).
• the comparative page may showcase one or more current data accuracy level which may be compared with one or more past data accuracy levels.
• the recognition page may showcase a list of users who have corrected the maximum number of data gaps (i.e. anomalies) in a past one week.
• the email notification page may have one or more feature such as personalization, data champions and link.
• Every user may receive a personalized email which mentioned one or more counts of gaps (i.e. anomalies) and a username that the user may require to input to log into the system.
• a list of weekly data champions may be shared with all the users to recognize one or more top performers.
• a link may be shared to the user and the user may directly open the link from the email.
• system [100] may further provide one or more graphical representation of rankings of data creators based on one or more metrics such as, but not limited to, number of data anomalies corrected within a fixed timespan.
• the alert corresponding to the one or more anomalies may direct the subset of users to one or more user interfaces, wherein, each of the user interfaces may provide a graphical representation of the contents of the alert, including information relating to the valid data set, the anomalous data set and the at least one rectification action.
• FIG. 2 it illustrates an exemplary flow diagram for a method [200] for anomaly management in a database. As illustrated, the method [200] begins at step [202],
• step [204] one or more sets of data that are generated by a set of users are retrieved from a database.
• the method [200] may further comprise retrieving the one or more data sets using an extract, transform and load (ETL) tool.
• the method [200] may further comprise retrieving the one or more data sets from across a plurality of databases, via the ETL tool.
• one or more anomalies may be detected in the one or more data sets on the basis of a predetermined set of rules [108],
• the one or more anomalies may be anomalies relating to the data in the one or more data sets. Accordingly, the one or more anomalies may indicate an occurrence that may be attributed to a degradation or a loss in the quality of data present in the one or more data sets. As such, one or more occurrences of at least one of the following may constitute an anomaly, namely, a gap, an error, an inconsistency, an irregularity or an incongruity within the data.
• the predetermined set of rules [108] may comprise one or more parameters required to detect the one or more anomalies and the minimum threshold/standards required by a data object to qualify as a valid data.
• the parameters may include the necessary elements required in a data object for it to be valid, and in absence of which it may be categorized as an invalid data.
• the parameters may include prescription on appropriate qualitative and quantitative ranges for the data objects in a data set, and/or one or more standardized formats for the entry of data objects into a set of data.
• the parameter values may include any other metrics that may be known to a person ordinarily skilled in the art for the management of anomalies in one or more data sets and setting thresholds for the validity of data.
• the predetermined set of rules [108] may be converted into one or more data flows using the ETL tool, wherein the converted data flows may then be used as the basis for the detection of one or more anomalies in the retrieved data sets within the ETL tool.
• the method [200] may comprise identifying at least one rectification action corresponding to the one or more anomalies.
• the at least one rectification action may be an action based on a predetermined set of instructions [110],
• the predetermined set of instructions [HO] may comprise one or more instructions required to convert an anomalous data into a valid data, or to correct one or more anomalies present in one or more data sets.
• the predetermined set of instructions [110] may include a number of outcomes to be achieved in order to turn anomalous data objects within an anomalous data set, into valid data objects.
• a rectification action, based on the predetermined set of instructions [110] may comprise at least one step required to be performed by a user in order to achieve an outcome for converting an anomalous data object, in an anomalous data set, into a valid data object.
• the method [200] may comprise detecting the one or more anomalies, and identifying the at least one rectification action, using at least one Large Language Models (LLMs).
• LLMs Large Language Models
• an alert corresponding to the detected one or more anomalies may generated.
• the method [200] may comprise determining a subset of users from the set of users responsible for generating the one or more sets of data.
• the determination of the subset of users maybe based upon the one or more anomalies detected and the at least one rectification action identified.
• the alert may be transmitted to the determined subset of users.
• an individual alert corresponding to the one or more anomalies may be transmitted to each of the subset of users, wherein, each of the alerts may be personalized for the recipient, and therefore each alert may only comprise the corresponding one or more anomalies and the associated rectification action(s) for the specific user from the subset of users.
• FIG. 3 an exemplary representation of an interface [300] depicting translation done via the one or more large language models (LLMs), in accordance with exemplary implementations of the present disclosure is depicted.
• the processing unit [102] may translate in real time, the set of data stored in the database in different languages via utilizing one or more large language models (LLMs). Thereafter, a translated version of the one or more sets of data may be retrieved for anomaly management.
• LLMs large language models
• FIG. 5 an exemplary representation of an interface [500] depicting identification of a standard industry code, in accordance with exemplary implementations of the present disclosure is depicted.
• the system [100] may, via the processing unit [102], use at least one or more large language models (LLMs) to identify one or more standard industry codes for one or more companies present in the database, wherein, a user may use the one or more standard industry code to identify companies within the relevant industry that are present in the database.
• LLMs large language models

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Databases & Information Systems (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Quality & Reliability (AREA)
• Data Mining & Analysis (AREA)
• Business, Economics & Management (AREA)
• Economics (AREA)
• Human Resources & Organizations (AREA)
• Strategic Management (AREA)
• Entrepreneurship & Innovation (AREA)
• Marketing (AREA)
• Game Theory and Decision Science (AREA)
• Educational Administration (AREA)
• Development Economics (AREA)
• Operations Research (AREA)
• Computational Linguistics (AREA)
• Tourism & Hospitality (AREA)
• General Business, Economics & Management (AREA)
• Computing Systems (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=97869683

Family Applications (1)

Country Status (2)

Families Citing this family (1)

Citations (8)

• 2024
  • 2024-05-31 IN IN202411042565A patent/IN202411042565A/en unknown
• 2025
  • 2025-05-30 WO PCT/IB2025/055613 patent/WO2025248503A1/en active Pending

Patent Citations (8)

Also Published As

Similar Documents

Legal Events