CA2829985C - System and method for validation and conversion of electronic data - Google Patents
System and method for validation and conversion of electronic data Download PDFInfo
- Publication number
- CA2829985C CA2829985C CA2829985A CA2829985A CA2829985C CA 2829985 C CA2829985 C CA 2829985C CA 2829985 A CA2829985 A CA 2829985A CA 2829985 A CA2829985 A CA 2829985A CA 2829985 C CA2829985 C CA 2829985C
- Authority
- CA
- Canada
- Prior art keywords
- electronic data
- content
- format
- validation
- valid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a system and method for validating electronic data and content associated with said data. A receiving module receives input electronic data and validation engine validates the electronic data type and its content against predefined regular expressions. The invalid electronic data and its content found during validation is transmitted back to the input electronic data as error messages in order to modify input electronic data and its content. The validated input electronic file is further processed in order to convert the electronic data and its content into user defined object in target file format using predefined mapping process.
Description
SYSTEM AND METHOD FOR VALIDATION AND CONVERSION OF
ELECTRONIC DATA
FIELD OF THE INVENTION
[001] The present invention relates to the field of an electronic data conversion. More particularly, the present invention relates to validation of the electronic data format and content associated with the electronic data and converting the electronic data and its content into a user defined object.
BACKGROUND OF THE INVENTION
ELECTRONIC DATA
FIELD OF THE INVENTION
[001] The present invention relates to the field of an electronic data conversion. More particularly, the present invention relates to validation of the electronic data format and content associated with the electronic data and converting the electronic data and its content into a user defined object.
BACKGROUND OF THE INVENTION
[002] Conventional enterprise applications gather information as an input from multiple sources in multiple data format types. Validating the gathered information for the electronic data type and its content for processing information requires various validation logics depending on the type of the electronic data and its format. Developing tools for validating multiple electronic data formats and its contents involves complexity and operational problems in designing such tools.
10031 Enterprise applications such as retail applications require input electronic data from various merchants in order to obtain relevant information which is then processed for multiple purposes like billing etc. The merchants may need to send day to day transaction details of the store to a central database or other repositories. Likewise, sponsors may send electronic data containing offer related information, or information pertaining to creation of new store locations in the form of excel, csv or flat files. The electronic data need to be validated before they are processed to save the content of the electronic data.
The input electronic data sent by different vendors or sponsors may comprise electronic data in multiple type of formats. The logic implemented in processing the input electronic data in order to validate the input electronic data may vary from one project to another.
[004] Currently, Extract Transform Load (ETL) tools such as Pentaho are being used to validate input electronic data prior to information processing. These ETL
process involves complexity and operational problem as most of these ETL systems are improperly designed. Further, these ETL systems require external applications with their own GUI to validate electronic data at run-time. These external applications require their own validation logic for validation of various electronic data and its content. When the number of electronic data in different formats increases, additional application code needs to be written in order to validate the electronic data of large mass.
[005] Available ETL tools require licenses in order to be implemented in an enterprise.
Small enterprises may require validating electronic data and its content in order to report errors in the electronic data. Small enterprises tend to develop their own application code that may provide validation logic with respect to the specific electronic data type formats.
Moreover, present ETL tools may become overburdened for small applications which require validation of few electronic data.
[006] Further, ETL systems require external applications in order to validate electronic data. Therefore, these ETL tools may not be plugged into existing application code. ETL
systems require change in script logic in cases of change in validation.
Further, ETL
systems require separate ETL mapping for different types of electronic data.
[007] Conventional ETL tools do not provide the functionality of converting the validated electronic data and its content into a user defined objects in the target file format.
Currently available application program interfaces (API) only facilitates in reading the electronic data but do not help in validating them and converting them into a user-defined objects.
[008] Therefore, there is a need to provide a system and method capable of validating the electronic data and its content. The system and method should also be capable of converting the validated electronic data and its content into a user defined object and should also facilitate the electronic data validation without changing the application code.
SUMMARY OF THE INVENTION
[009] The present invention provides a system for validating an electronic data and content associated with said electronic data. The system comprises of receiving module in a memory storage unit configured to receive and identify format type of the electronic data along with its content. The system further comprises of a validation engine configured to = 81774756 validate the electronic data and its content against predefined regular expressions. The validation engine further comprises of a processing engine configured to convert the format of validated electronic data and its content into target file format and to identify a set of invalid electronic data and its content to further report one or more error messages obtained during validation and connecting the error messages with the invalid electronic data in order to modify the input electronic data and its content. The system for validating electronic data further comprises of a data conversion engine configured to convert the validated electronic data and its content into a user defined object by using a predefined mapping process.
[0010] The present invention also provides a method for validating an electronic data and content associated with said electronic data. The method comprising steps of identifying a format type of the electronic data along with its content received as input.
The method further comprises steps of validating the input electronic data and its content against predefined regular expressions. The method of validating the input electronic data comprises of processing a set of validated electronic data and its content in order to convert the format of the validated electronic data and its content into a target file format and identifying a set of invalid electronic data and its content to further report one or more error messages obtained during the validation and connecting the error messages with the invalid electronic data in order to modify the input electronic data and its content. The method for validating said electronic data and content associated with said electronic data further comprises of converting the validated electronic data into a user defined object by using a predefined mapping process.
[0010a] According to one aspect of the present invention, there is provided a system for validating an electronic data and content associated with the electronic data, the system comprising; a processor; a memory coupled to the processor, wherein the processor is capable of executing a plurality of modules stored in the memory, and wherein the plurality of modules comprising: a receiving module to: receive the electronic data and content associated with the electronic data; identify a format type of the electronic data along with its content; a validation engine coupled with a processing engine, wherein the
10031 Enterprise applications such as retail applications require input electronic data from various merchants in order to obtain relevant information which is then processed for multiple purposes like billing etc. The merchants may need to send day to day transaction details of the store to a central database or other repositories. Likewise, sponsors may send electronic data containing offer related information, or information pertaining to creation of new store locations in the form of excel, csv or flat files. The electronic data need to be validated before they are processed to save the content of the electronic data.
The input electronic data sent by different vendors or sponsors may comprise electronic data in multiple type of formats. The logic implemented in processing the input electronic data in order to validate the input electronic data may vary from one project to another.
[004] Currently, Extract Transform Load (ETL) tools such as Pentaho are being used to validate input electronic data prior to information processing. These ETL
process involves complexity and operational problem as most of these ETL systems are improperly designed. Further, these ETL systems require external applications with their own GUI to validate electronic data at run-time. These external applications require their own validation logic for validation of various electronic data and its content. When the number of electronic data in different formats increases, additional application code needs to be written in order to validate the electronic data of large mass.
[005] Available ETL tools require licenses in order to be implemented in an enterprise.
Small enterprises may require validating electronic data and its content in order to report errors in the electronic data. Small enterprises tend to develop their own application code that may provide validation logic with respect to the specific electronic data type formats.
Moreover, present ETL tools may become overburdened for small applications which require validation of few electronic data.
[006] Further, ETL systems require external applications in order to validate electronic data. Therefore, these ETL tools may not be plugged into existing application code. ETL
systems require change in script logic in cases of change in validation.
Further, ETL
systems require separate ETL mapping for different types of electronic data.
[007] Conventional ETL tools do not provide the functionality of converting the validated electronic data and its content into a user defined objects in the target file format.
Currently available application program interfaces (API) only facilitates in reading the electronic data but do not help in validating them and converting them into a user-defined objects.
[008] Therefore, there is a need to provide a system and method capable of validating the electronic data and its content. The system and method should also be capable of converting the validated electronic data and its content into a user defined object and should also facilitate the electronic data validation without changing the application code.
SUMMARY OF THE INVENTION
[009] The present invention provides a system for validating an electronic data and content associated with said electronic data. The system comprises of receiving module in a memory storage unit configured to receive and identify format type of the electronic data along with its content. The system further comprises of a validation engine configured to = 81774756 validate the electronic data and its content against predefined regular expressions. The validation engine further comprises of a processing engine configured to convert the format of validated electronic data and its content into target file format and to identify a set of invalid electronic data and its content to further report one or more error messages obtained during validation and connecting the error messages with the invalid electronic data in order to modify the input electronic data and its content. The system for validating electronic data further comprises of a data conversion engine configured to convert the validated electronic data and its content into a user defined object by using a predefined mapping process.
[0010] The present invention also provides a method for validating an electronic data and content associated with said electronic data. The method comprising steps of identifying a format type of the electronic data along with its content received as input.
The method further comprises steps of validating the input electronic data and its content against predefined regular expressions. The method of validating the input electronic data comprises of processing a set of validated electronic data and its content in order to convert the format of the validated electronic data and its content into a target file format and identifying a set of invalid electronic data and its content to further report one or more error messages obtained during the validation and connecting the error messages with the invalid electronic data in order to modify the input electronic data and its content. The method for validating said electronic data and content associated with said electronic data further comprises of converting the validated electronic data into a user defined object by using a predefined mapping process.
[0010a] According to one aspect of the present invention, there is provided a system for validating an electronic data and content associated with the electronic data, the system comprising; a processor; a memory coupled to the processor, wherein the processor is capable of executing a plurality of modules stored in the memory, and wherein the plurality of modules comprising: a receiving module to: receive the electronic data and content associated with the electronic data; identify a format type of the electronic data along with its content; a validation engine coupled with a processing engine, wherein the
3 validation engine and the processing engine collectively validate the electronic data and the content by fetching a configuration of a validation logic from a database, wherein the validation comprises: identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type; verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with predefined regular expressions; reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database; modifying the invalid content based upon the one or more error messages to generate a modified content; a data converting engine to convert the valid electronic data comprising the valid content or the modified content into a target file format.
[0010b] According to another aspect of the present invention, there is provided a method for validating an electronic data and content associated with the electronic data, the method comprising; identifying a format type of the electronic data along with its content received as an input; validating the electronic data and its content by fetching a configuration of a validation logic from a database, wherein the validating further comprises: identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type; verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with a predefined regular expressions; reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change 3a determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database; modifying the invalid content based upon the one or more error messages to generate a modified content; and converting the valid electronic data comprising the valid content or modified content into a target file format.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Further objects, embodiments, features and advantages of the present invention will become more apparent and may be better understood when read together with the detailed description and the accompanied drawings. The components of the figures are not necessarily to scales, emphasis instead being placed on better illustration of the underlying principle of the subject matter. Different numeral references on figures 3b designate corresponding elements throughout different views. However, the manner in which the above depicted features, aspects, and advantages of the present subject matter are accomplished, does not limit the scope of the subject matter, for the subject matter may admit to other equally effective embodiments.
[0012] Figure 1 illustrates the system architecture for validating electronic data and content associated with said data.
[0013] Figure 2 illustrates the process flow for validating electronic data and content associated with said data in accordance with an alternate embodiment of the invention.
[0014] Figure 3 illustrates validation and conversion of electronic data and its content in accordance with an exemplary embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
[0015] Some embodiments of this invention, illustrating its features, will now be discussed:
[0016] The words "comprising". "having", "containing", and "including", and other forms thereof, are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items.
[0017] It must also be noted that as used herein and in the appended claims, the singular forms "a", "an", and "the" include plural references unless the context clearly dictates otherwise. Although any systems, methods, apparatuses, and devices similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present invention, the preferred, systems and parts are now described.
In the following description for the purpose of explanation and understanding reference has been made to numerous embodiments for which the intent is not to limit the scope of the invention.
[0018] One or more components of the invention are described as module for the understanding of the specification. For example, a module may include self-contained component in a hardware circuit comprising of logical gate, semiconductor device, integrated circuits or any other discrete component. The module may also be a part of any
[0010b] According to another aspect of the present invention, there is provided a method for validating an electronic data and content associated with the electronic data, the method comprising; identifying a format type of the electronic data along with its content received as an input; validating the electronic data and its content by fetching a configuration of a validation logic from a database, wherein the validating further comprises: identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type; verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with a predefined regular expressions; reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change 3a determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database; modifying the invalid content based upon the one or more error messages to generate a modified content; and converting the valid electronic data comprising the valid content or modified content into a target file format.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Further objects, embodiments, features and advantages of the present invention will become more apparent and may be better understood when read together with the detailed description and the accompanied drawings. The components of the figures are not necessarily to scales, emphasis instead being placed on better illustration of the underlying principle of the subject matter. Different numeral references on figures 3b designate corresponding elements throughout different views. However, the manner in which the above depicted features, aspects, and advantages of the present subject matter are accomplished, does not limit the scope of the subject matter, for the subject matter may admit to other equally effective embodiments.
[0012] Figure 1 illustrates the system architecture for validating electronic data and content associated with said data.
[0013] Figure 2 illustrates the process flow for validating electronic data and content associated with said data in accordance with an alternate embodiment of the invention.
[0014] Figure 3 illustrates validation and conversion of electronic data and its content in accordance with an exemplary embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
[0015] Some embodiments of this invention, illustrating its features, will now be discussed:
[0016] The words "comprising". "having", "containing", and "including", and other forms thereof, are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items.
[0017] It must also be noted that as used herein and in the appended claims, the singular forms "a", "an", and "the" include plural references unless the context clearly dictates otherwise. Although any systems, methods, apparatuses, and devices similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present invention, the preferred, systems and parts are now described.
In the following description for the purpose of explanation and understanding reference has been made to numerous embodiments for which the intent is not to limit the scope of the invention.
[0018] One or more components of the invention are described as module for the understanding of the specification. For example, a module may include self-contained component in a hardware circuit comprising of logical gate, semiconductor device, integrated circuits or any other discrete component. The module may also be a part of any
4 software programme executed by any hardware entity for example processor. The implementation of module as a software programme may include a set of logical instructions to be executed by a processor or any other hardware entity.
Further a module may be incorporated with the set of instructions or a programme by means of an interface.
[0019] The disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms.
[0020] The present invention provides a system and method for validating electronic data and content associated with said electronic data. The present invention identifies and receives electronic data as an input from multiple sources in multiple formats. The received input electronic data is validated for the electronic data type and its content against predefined regular expressions. The invalid electronic data and its content is transmitted back to the input electronic data as error messages in order to modify input electronic data and its content. The validated input electronic data is further processed in order to convert the electronic data and its content into a user defmed object in a target file format using predefined mapping process.
[0021] In accordance with an embodiment of the invention, Figure 1 illustrates a system 100 for validating an electronic data and content associated with said electronic data. The system 100 comprises of a receiving module 104 stored in a memory 102 configured to receive and identify a format type of the electronic data along with its content. The system 100 further comprises of a validation engine 106 coupled to the memory configured to validate the electronic data and its content against a predefined regular expressions. The validation engine 106 further comprises of a processing engine 108 configured to convert the format of validated electronic data and its content into a target file format (not shown) and to identify a set of invalid electronic data and its content to further report one or more error messages obtained during validation and connecting said error messages with the invalid electronic data in order to modify said electronic data and its content received as the input. The system 100 further comprises of a data converting engine 110 configured to convert the validated electronic data and its content into a user defined object by using a predefined mapping process.
[0022] In accordance with another embodiment of the invention, the system 100 comprises the electronic data, the electronic data further comprises of a flat file, an xis file, an xlsx format (excel format), a txt format, and a comma separated value (csv) format.
The system 100 receives input electronic data using the receiving module 104 stored in the memory 102. The receiving module 104 is configured to receive and identify a format type of the input electronic data along with its content (as shown in step 202 of figure 2).
The receiving module 104 may include but not limited to a key board, a touch pad or some similar hardware entity though which the electronic data could be provided as the input.
[0023] The validation engine 106 is configured to validate the input electronic data (as shown in step 204 of figure 2) by using an apache POI in order to read and write excel electronic data formats. The apache POI facilitates in creating and maintaining APIs for controlling various electronic data formats built on the Office Open XML
standards (00XML) and Microsoft's OLE 2 Compound Document format (OLE2). Further, the apache POI facilitates in reading and writing Microsoft (TM) excel electronic data formats using Java for validation. The validation engine 106 may configure an application which runs within a Java Virtual Machine (JVM). Further, the validation engine 106 may have an access to all of constructs and classes which are apparent to Java Virtual Machine (JVM). The Java Virtual Machine and further details associated are well understood by those persons skilled in the art.
[0024] The validation engine 106 verifies the input electronic data format such that the input electronic data is a supported data format which includes xis, xlsx, csv, txt, etc. Further, the validation engine 106 is configured to validate the content of the input electronic data.
The validation engine 106 comprises of a generic field level validations for said various electronic data. The validation engine 106 validates the electronic data and its content against the predefined regular expressions. The electronic data columns are mapped in a database table which can be configured at run time.
[0025] In accordance with another embodiment of the invention, the validation engine 106 comprises of the processing engine 108 configured to identify set of invalid electronic data format against predefined said electronic data formats. In another embodiment, the processing engine 108 identifies a set of electronic data formats and invalid content in the input electronic data by validating the input electronic data and its content against predefined regular expressions or user defined pattern. The processing engine 108 is further configured to report one or more error messages obtained during validation and connecting said error messages with the invalid electronic data in order to modify said input electronic data and its content. The processing engine 108 retrieves a configuration for validation from the database and validates the electronic data and its content against the configuration. The error messages found during the validation are transmitted back to the input electronic data in a new and separate error column in the input electronic data.
Further, the processing engine 108 processes the validated electronic data and its content for converting the electronic data and its content (as shown in step 206 of figure 2) into user defined objects in the target file using predefined mapping process.
[0026] The at least one processing engine 108 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the at least one processing engine 108 is configured to fetch and execute computer-readable instructions stored in the memory 102.
10027] In an exemplary embodiment of the invention, the error messages reported during validation are in internationalization and localization (ii 8n) format such that the application may be adapted in different languages and locations without any changes in the electronic data and its content of the input electronic data. The error messages are reported in multiple languages such that various formats of the electronic data may be represented as date, time, number and currency formats.
100281 In accordance with another embodiment of the invention, the system 100 comprises of the data converting engine 110 configured to convert the validated electronic data (as shown in step 208 of figure 2) and its content into the user defined object by using the predefined mapping process. Further, the conversion of the validated electronic data and its content into the target file format comprises converting the electronic data and its content into a list of user defined object with each row or column corresponding to one object in the list. The data converting engine 110 reads the mapping from the XML files in order to convert the validated electronic data type and its content into user defined objects in the target file format. The data converting engine 110 may convert the validated input electronic data format such as XLS file format and its content to XLS or CS V format as defined by the user during converting process. The working of data converting engine 110 adapted to convert the validated input electronic data from one type to same type or to another data type in the target file is well understood by those persons skilled in the art.
[0029] In another exemplary embodiment of the invention, the data converting engine 110 comprises an XML parser to convert the validated electronic data and its content into the user defined object. The XML parser reads the XML files and converts the XML
files into an XML Document Object Model (DOM) object which can be accessed with Java.
Further, the XML parser may comprise Simple API for XML (SAX), pull parsing and data binding to convert the validated electronic data and its content into the user defined object.
[0030] The validation and conversion of the electronic data and its content comprises of the memory 102, the receiving module 104, the validation engine 106, the processing engine 108 and the data converting engine 110 that are capable of being executed in collaboration or worked with each other.
[0031] In accordance with another embodiment of the invention, the target file format further comprises of the input electronic data type format i.e. the electronic data received as the input. Further, the target file format comprises of the XML file format. The converted target file is stored in a compressed java archive (JAR) file format. When compared with existing tools such as flat file checker which facilities in conversion of similar electronic data upon validated comprises file size of 630 KB when compared to 20 KB
provided in JAR file with respect to embodiment of the invention. Further, similar validation using Pentaho comprises 300 MB file size.
[0032] In accordance with another embodiment of the invention, the converted target file is stored in the compressed java archive (JAR) file format which may comprise applications.
[0033] Existing enterprise applications such as flat file checker and Penhato requires third party applications in order to change the existing code when there is change in configuration of error messages. In an embodiment of the invention, the system defines the error messages in the database. Accordingly, any change in the error messages during validation of the input electronic data and its content requires only change in configuration of the validation logic in the database. Therefore, any change in the error messages do not require additional application code to be written during the run time of conversion in electronic data formats.
BEST MODE/EXAMPLE FOR WORKING OF THE INVENTION
[0034] The system and method illustrated for validating electronic data and content associated with said electronic data may be illustrated by working example stated in the following paragraph; the process is not restricted to the example only:
[0035] Figure 3 illustrates validation and conversion 300 of input electronic data 302 such as xis, xlsx, csv, txt, etc. The validation engine 106 receives input electronic data and identifies the input electronic data for validation 304. The file validation 304 comprises validation of mandatory fields 306 and contents 308 in the input electronic data 302. In the case of input electronic data 302 is not supported by the validation engine 106, the error input electronic data is 310 transmitted back for validating the electronic data.
Further, in the case of input electronic data 302 comprises invalid content, then the validation engine 106 comprising the processing engine 108 reports the error messages against invalid content in the input electronic data 302 in a new and separate error column in the input electronic data 302.
[0036] For example, considering an input.xls as an input electronic data 302 comprising three columns and three records as represented below.
A
1 Name Age Sex 2 Subasish 30 3 Siben 28 4 Richa 24 Table 1 100371 The validation engine 106 fetches configuration data for validating the data from the database. The input electronic data 302 and its contents are validated against predefined expressions. If the input electronic data 302 formats and content is valid, the input electronic data 302 is passed for validation and the input electronic data 302 is processed further for electronic data conversion into user defined object in the target file format.
[0038] Considering another input.xls as an input electronic data 302 comprising three columns and three records as represented below.
A
1 Name Age Sex 2 Subasish 30 3 Si ben @#% Fer444 4 Richa @#% @#%
Table 2 [0039] The input electronic data 302 is validated against predefined expressions. The three columns in the input electronic data 302 with three records comprise of content in second and third column of second row consisting errors. The validation engine 106 comprising the processing engine 108 retrieves configuration for validation from the database and validates the input electronic data 302 content against the configuration. The input electronic data 302 and its content is transmitted back by adding new column in the input electronic data 302 with header values as ERROR and two error messages which are comma separated for second and third column appended to the same input electronic data 302 in the second row as user defined. The input electronic data 302 transmitted back is presented below:
A
1 Name Age Sex ERROR
2 Subasish 30 3 Siben @#% Fer444 INVALID-AGE, INVALID SEX
4 Richa @#% @#% INVALID-AGE, INVALID SEX
Table 3 [0040] The input electronic data 302 transmitted back with invalid content may comprise same file name or renamed file such as output.xls. For the purpose of illustration the error messages is presented in il 8n which may include English. Further, the error messages may include configuration in any other language as apparent to those skilled in the art.
[0041] Considering the case of Table 1, the input electronic data 302 included the valid content and hence passed the validation with three columns (name, age, and sex) and three rows. The validated input electronic data 302 is converted 312 into user defined object in the target file format using XML based mapping of electronic data columns. An exemplary means of configuring for converting validated electronic data and its content into user defined objects is presented as below:
1 <entity>
2 <subentity name="offer" class="com.tcs.pattent.person">
3 <attribute cellIndex="0"
methodName="setName"dataType="java.lang.string"/>
4 <attribute cellIndex="1" methodName="setAge"dataType="java.lang.integer"/>
<attribute cellIndex="2" methodName="setSex"dataType="javalang.string"/>
6 < subentity>
7 <entity>
=
[0042] Wherein cellindex is electronic data cell index. Further, <subentity name="offer"
class="com.tcs.pattent.person"> is data transfer object definition. Further, methodName and dataType are data transfer object attributes for mapping with the electronic data cell index.
[0043] The data converting engine 110 converts the validated electronic data and its content into said user defined object 314 by using said predefined mapping process.
[0044] In another example, considering an input.txt as input electronic data 302 comprising content in txt format as represented below:
Subasish 30M
Siben 28M
Richa 24F
Table 4 [0045] In the case of validating txt format electronic data, columns values are fetched using their position in the electronic data 302 i.e. name position begins from 0 to 20, age position begins from 21 to 22 and sex position begins from 22 to 23. The column positions are presented as a master data which is present in database and the column positions are provided as a part of database insert or update scripts.
[0046] Considering the input.txt comprising the electronic data content as presented below.
Subasish 30M
Siben @%gh Richa @%asdf Table 5 [0047] The input electronic data 302 along with its content is validated against predefined expressions. The input.txt electronic data 302 as shown in table 5 comprises errors, the validation engine 106 comprising the processing engine 108 retrieves configuration for validation from the database. The input.txt electronic data 302 is transmitted back by adding errors in separate column in the input electronic data 302, the transmitted back input.txt electronic data 302 is presented as below.
Subasish 30M
Siben @%gh INVALID-AGE, INVALID SEX
Richa @%asdf INVALID-AGE, INVALID SEX
Table 6 [0048] The transmitted back input electronic data 302 with invalid content may comprise file name such as input.txt or output.txt.
[0049] The error column value is appended to the input electronic data format 302 at one position right upon the maximum position of the last column value in the input electronic data 302. The error position is determined at runtime based on the file row whichever length is maximum.
[0050] Considering the case of Table 4, the input electronic data 302 comprises the valid electronic data and content, and hence the input electronic data 302 passes the validation.
The validated input electronic data 302 is then converted 312 into user defined object in the target file format using XML based mapping of the electronic data columns.
Further a module may be incorporated with the set of instructions or a programme by means of an interface.
[0019] The disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms.
[0020] The present invention provides a system and method for validating electronic data and content associated with said electronic data. The present invention identifies and receives electronic data as an input from multiple sources in multiple formats. The received input electronic data is validated for the electronic data type and its content against predefined regular expressions. The invalid electronic data and its content is transmitted back to the input electronic data as error messages in order to modify input electronic data and its content. The validated input electronic data is further processed in order to convert the electronic data and its content into a user defmed object in a target file format using predefined mapping process.
[0021] In accordance with an embodiment of the invention, Figure 1 illustrates a system 100 for validating an electronic data and content associated with said electronic data. The system 100 comprises of a receiving module 104 stored in a memory 102 configured to receive and identify a format type of the electronic data along with its content. The system 100 further comprises of a validation engine 106 coupled to the memory configured to validate the electronic data and its content against a predefined regular expressions. The validation engine 106 further comprises of a processing engine 108 configured to convert the format of validated electronic data and its content into a target file format (not shown) and to identify a set of invalid electronic data and its content to further report one or more error messages obtained during validation and connecting said error messages with the invalid electronic data in order to modify said electronic data and its content received as the input. The system 100 further comprises of a data converting engine 110 configured to convert the validated electronic data and its content into a user defined object by using a predefined mapping process.
[0022] In accordance with another embodiment of the invention, the system 100 comprises the electronic data, the electronic data further comprises of a flat file, an xis file, an xlsx format (excel format), a txt format, and a comma separated value (csv) format.
The system 100 receives input electronic data using the receiving module 104 stored in the memory 102. The receiving module 104 is configured to receive and identify a format type of the input electronic data along with its content (as shown in step 202 of figure 2).
The receiving module 104 may include but not limited to a key board, a touch pad or some similar hardware entity though which the electronic data could be provided as the input.
[0023] The validation engine 106 is configured to validate the input electronic data (as shown in step 204 of figure 2) by using an apache POI in order to read and write excel electronic data formats. The apache POI facilitates in creating and maintaining APIs for controlling various electronic data formats built on the Office Open XML
standards (00XML) and Microsoft's OLE 2 Compound Document format (OLE2). Further, the apache POI facilitates in reading and writing Microsoft (TM) excel electronic data formats using Java for validation. The validation engine 106 may configure an application which runs within a Java Virtual Machine (JVM). Further, the validation engine 106 may have an access to all of constructs and classes which are apparent to Java Virtual Machine (JVM). The Java Virtual Machine and further details associated are well understood by those persons skilled in the art.
[0024] The validation engine 106 verifies the input electronic data format such that the input electronic data is a supported data format which includes xis, xlsx, csv, txt, etc. Further, the validation engine 106 is configured to validate the content of the input electronic data.
The validation engine 106 comprises of a generic field level validations for said various electronic data. The validation engine 106 validates the electronic data and its content against the predefined regular expressions. The electronic data columns are mapped in a database table which can be configured at run time.
[0025] In accordance with another embodiment of the invention, the validation engine 106 comprises of the processing engine 108 configured to identify set of invalid electronic data format against predefined said electronic data formats. In another embodiment, the processing engine 108 identifies a set of electronic data formats and invalid content in the input electronic data by validating the input electronic data and its content against predefined regular expressions or user defined pattern. The processing engine 108 is further configured to report one or more error messages obtained during validation and connecting said error messages with the invalid electronic data in order to modify said input electronic data and its content. The processing engine 108 retrieves a configuration for validation from the database and validates the electronic data and its content against the configuration. The error messages found during the validation are transmitted back to the input electronic data in a new and separate error column in the input electronic data.
Further, the processing engine 108 processes the validated electronic data and its content for converting the electronic data and its content (as shown in step 206 of figure 2) into user defined objects in the target file using predefined mapping process.
[0026] The at least one processing engine 108 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the at least one processing engine 108 is configured to fetch and execute computer-readable instructions stored in the memory 102.
10027] In an exemplary embodiment of the invention, the error messages reported during validation are in internationalization and localization (ii 8n) format such that the application may be adapted in different languages and locations without any changes in the electronic data and its content of the input electronic data. The error messages are reported in multiple languages such that various formats of the electronic data may be represented as date, time, number and currency formats.
100281 In accordance with another embodiment of the invention, the system 100 comprises of the data converting engine 110 configured to convert the validated electronic data (as shown in step 208 of figure 2) and its content into the user defined object by using the predefined mapping process. Further, the conversion of the validated electronic data and its content into the target file format comprises converting the electronic data and its content into a list of user defined object with each row or column corresponding to one object in the list. The data converting engine 110 reads the mapping from the XML files in order to convert the validated electronic data type and its content into user defined objects in the target file format. The data converting engine 110 may convert the validated input electronic data format such as XLS file format and its content to XLS or CS V format as defined by the user during converting process. The working of data converting engine 110 adapted to convert the validated input electronic data from one type to same type or to another data type in the target file is well understood by those persons skilled in the art.
[0029] In another exemplary embodiment of the invention, the data converting engine 110 comprises an XML parser to convert the validated electronic data and its content into the user defined object. The XML parser reads the XML files and converts the XML
files into an XML Document Object Model (DOM) object which can be accessed with Java.
Further, the XML parser may comprise Simple API for XML (SAX), pull parsing and data binding to convert the validated electronic data and its content into the user defined object.
[0030] The validation and conversion of the electronic data and its content comprises of the memory 102, the receiving module 104, the validation engine 106, the processing engine 108 and the data converting engine 110 that are capable of being executed in collaboration or worked with each other.
[0031] In accordance with another embodiment of the invention, the target file format further comprises of the input electronic data type format i.e. the electronic data received as the input. Further, the target file format comprises of the XML file format. The converted target file is stored in a compressed java archive (JAR) file format. When compared with existing tools such as flat file checker which facilities in conversion of similar electronic data upon validated comprises file size of 630 KB when compared to 20 KB
provided in JAR file with respect to embodiment of the invention. Further, similar validation using Pentaho comprises 300 MB file size.
[0032] In accordance with another embodiment of the invention, the converted target file is stored in the compressed java archive (JAR) file format which may comprise applications.
[0033] Existing enterprise applications such as flat file checker and Penhato requires third party applications in order to change the existing code when there is change in configuration of error messages. In an embodiment of the invention, the system defines the error messages in the database. Accordingly, any change in the error messages during validation of the input electronic data and its content requires only change in configuration of the validation logic in the database. Therefore, any change in the error messages do not require additional application code to be written during the run time of conversion in electronic data formats.
BEST MODE/EXAMPLE FOR WORKING OF THE INVENTION
[0034] The system and method illustrated for validating electronic data and content associated with said electronic data may be illustrated by working example stated in the following paragraph; the process is not restricted to the example only:
[0035] Figure 3 illustrates validation and conversion 300 of input electronic data 302 such as xis, xlsx, csv, txt, etc. The validation engine 106 receives input electronic data and identifies the input electronic data for validation 304. The file validation 304 comprises validation of mandatory fields 306 and contents 308 in the input electronic data 302. In the case of input electronic data 302 is not supported by the validation engine 106, the error input electronic data is 310 transmitted back for validating the electronic data.
Further, in the case of input electronic data 302 comprises invalid content, then the validation engine 106 comprising the processing engine 108 reports the error messages against invalid content in the input electronic data 302 in a new and separate error column in the input electronic data 302.
[0036] For example, considering an input.xls as an input electronic data 302 comprising three columns and three records as represented below.
A
1 Name Age Sex 2 Subasish 30 3 Siben 28 4 Richa 24 Table 1 100371 The validation engine 106 fetches configuration data for validating the data from the database. The input electronic data 302 and its contents are validated against predefined expressions. If the input electronic data 302 formats and content is valid, the input electronic data 302 is passed for validation and the input electronic data 302 is processed further for electronic data conversion into user defined object in the target file format.
[0038] Considering another input.xls as an input electronic data 302 comprising three columns and three records as represented below.
A
1 Name Age Sex 2 Subasish 30 3 Si ben @#% Fer444 4 Richa @#% @#%
Table 2 [0039] The input electronic data 302 is validated against predefined expressions. The three columns in the input electronic data 302 with three records comprise of content in second and third column of second row consisting errors. The validation engine 106 comprising the processing engine 108 retrieves configuration for validation from the database and validates the input electronic data 302 content against the configuration. The input electronic data 302 and its content is transmitted back by adding new column in the input electronic data 302 with header values as ERROR and two error messages which are comma separated for second and third column appended to the same input electronic data 302 in the second row as user defined. The input electronic data 302 transmitted back is presented below:
A
1 Name Age Sex ERROR
2 Subasish 30 3 Siben @#% Fer444 INVALID-AGE, INVALID SEX
4 Richa @#% @#% INVALID-AGE, INVALID SEX
Table 3 [0040] The input electronic data 302 transmitted back with invalid content may comprise same file name or renamed file such as output.xls. For the purpose of illustration the error messages is presented in il 8n which may include English. Further, the error messages may include configuration in any other language as apparent to those skilled in the art.
[0041] Considering the case of Table 1, the input electronic data 302 included the valid content and hence passed the validation with three columns (name, age, and sex) and three rows. The validated input electronic data 302 is converted 312 into user defined object in the target file format using XML based mapping of electronic data columns. An exemplary means of configuring for converting validated electronic data and its content into user defined objects is presented as below:
1 <entity>
2 <subentity name="offer" class="com.tcs.pattent.person">
3 <attribute cellIndex="0"
methodName="setName"dataType="java.lang.string"/>
4 <attribute cellIndex="1" methodName="setAge"dataType="java.lang.integer"/>
<attribute cellIndex="2" methodName="setSex"dataType="javalang.string"/>
6 < subentity>
7 <entity>
=
[0042] Wherein cellindex is electronic data cell index. Further, <subentity name="offer"
class="com.tcs.pattent.person"> is data transfer object definition. Further, methodName and dataType are data transfer object attributes for mapping with the electronic data cell index.
[0043] The data converting engine 110 converts the validated electronic data and its content into said user defined object 314 by using said predefined mapping process.
[0044] In another example, considering an input.txt as input electronic data 302 comprising content in txt format as represented below:
Subasish 30M
Siben 28M
Richa 24F
Table 4 [0045] In the case of validating txt format electronic data, columns values are fetched using their position in the electronic data 302 i.e. name position begins from 0 to 20, age position begins from 21 to 22 and sex position begins from 22 to 23. The column positions are presented as a master data which is present in database and the column positions are provided as a part of database insert or update scripts.
[0046] Considering the input.txt comprising the electronic data content as presented below.
Subasish 30M
Siben @%gh Richa @%asdf Table 5 [0047] The input electronic data 302 along with its content is validated against predefined expressions. The input.txt electronic data 302 as shown in table 5 comprises errors, the validation engine 106 comprising the processing engine 108 retrieves configuration for validation from the database. The input.txt electronic data 302 is transmitted back by adding errors in separate column in the input electronic data 302, the transmitted back input.txt electronic data 302 is presented as below.
Subasish 30M
Siben @%gh INVALID-AGE, INVALID SEX
Richa @%asdf INVALID-AGE, INVALID SEX
Table 6 [0048] The transmitted back input electronic data 302 with invalid content may comprise file name such as input.txt or output.txt.
[0049] The error column value is appended to the input electronic data format 302 at one position right upon the maximum position of the last column value in the input electronic data 302. The error position is determined at runtime based on the file row whichever length is maximum.
[0050] Considering the case of Table 4, the input electronic data 302 comprises the valid electronic data and content, and hence the input electronic data 302 passes the validation.
The validated input electronic data 302 is then converted 312 into user defined object in the target file format using XML based mapping of the electronic data columns.
Claims (14)
1. A system for validating an electronic data and content associated with the electronic data, the system comprising;
a processor;
a memory coupled to the processor, wherein the processor is capable of executing a plurality of modules stored in the memory, and wherein the plurality of modules comprising:
a receiving module to:
receive the electronic data and content associated with the electronic data;
identify a format type of the electronic data along with its content;
a validation engine coupled with a processing engine, wherein the validation engine and the processing engine collectively validate the electronic data and the content by fetching a configuration of a validation logic from a database, wherein the validation comprises:
identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type;
verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with predefined regular expressions;
reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database;
modifying the invalid content based upon the one or more error messages to generate a modified content;
a data converting engine to convert the valid electronic data comprising the valid content or the modified content into a target file format.
a processor;
a memory coupled to the processor, wherein the processor is capable of executing a plurality of modules stored in the memory, and wherein the plurality of modules comprising:
a receiving module to:
receive the electronic data and content associated with the electronic data;
identify a format type of the electronic data along with its content;
a validation engine coupled with a processing engine, wherein the validation engine and the processing engine collectively validate the electronic data and the content by fetching a configuration of a validation logic from a database, wherein the validation comprises:
identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type;
verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with predefined regular expressions;
reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database;
modifying the invalid content based upon the one or more error messages to generate a modified content;
a data converting engine to convert the valid electronic data comprising the valid content or the modified content into a target file format.
2. The system of claim 1, wherein the format type of the electronic data is at least one of:
a flat file, an xls file, an xlsx format, txt format and comma separated value format.
a flat file, an xls file, an xlsx format, txt format and comma separated value format.
3. The system of claim 1, wherein the validation engine validates the electronic data by using an Apache Poor Obfuscation Implementation in order to read and write excel formats.
4. The system of claim 1, wherein the data converting engine converts the valid electronic data comprising the valid content into a list of user defined object with each row or column corresponding to one object in the list using a predefined mapping process.
5. The system of claim 1, wherein the target file format further comprises of the electronic data type format received as an input.
6. A method for validating an electronic data and content associated with the electronic data, the rnethod comprising;
identifying a format type of the electronic data along with its content received as an input;
validating the electronic data and its content by fetching a configuration of a validation logic from a database, wherein the validating further comprises:
identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type;
verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with a predefined regular expressions;
reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database;
modifying the invalid content based upon the one or more error messages to generate a modified content; and converting the valid electronic data comprising the valid content or modified content into a target file format.
identifying a format type of the electronic data along with its content received as an input;
validating the electronic data and its content by fetching a configuration of a validation logic from a database, wherein the validating further comprises:
identifying whether the electronic data is a valid electronic data or an invalid electronic data based upon the format type;
verifying, when the electronic data is identified as the valid electronic data, whether the content of the electronic file is a valid content or an invalid content, wherein the content is verified by comparing the content with a predefined regular expressions;
reporting one or more error messages for the invalid content, wherein the reporting comprises adding the one or more error messages in a new and separate column corresponding to the invalid content in the electronic data upon determining an error position of the invalid content in the electronic data at runtime, wherein the one or more error messages are defined in the database and any change determined in the one or more error messages during the validation results in change of the configuration of the validation logic in the database;
modifying the invalid content based upon the one or more error messages to generate a modified content; and converting the valid electronic data comprising the valid content or modified content into a target file format.
7. The method of claim 6, wherein the format type of the electronic data is at least one of a flat file (.txt, ..tau.), an xls file, an xlsx format and a comma separated value format.
8. The method of claim 6, wherein validating the electronic data to read and write excel electronic data formats is performed by using an Apache Poor Obfuscation Implementation.
9. The method of claim 6, wherein the one or more error messages reported during validation are in i18n format.
10. The method of claim 6, wherein converting valid electronic data further comprises of converting the electronic data into a list of user defined object using a predefined mapping process.
11. The method of claim 6, wherein the target file format further comprises of the electronic data type format received as an input.
12. The method of claim 10, wherein the predefined mapping process further includes mapping using XML files.
13. The method of claim 10, wherein converting the electronic data further comprises mapping each column of valid content to a user defined object in the list.
14. The system of claim 4, wherein converting the electronic data further comprises mapping each column of the valid content to a user defined object in the list.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| IN256/MUM/2013 | 2013-01-29 | ||
| IN256MU2013 | 2013-01-29 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2829985A1 CA2829985A1 (en) | 2014-07-29 |
| CA2829985C true CA2829985C (en) | 2018-06-26 |
Family
ID=51257898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2829985A Expired - Fee Related CA2829985C (en) | 2013-01-29 | 2013-10-16 | System and method for validation and conversion of electronic data |
Country Status (2)
| Country | Link |
|---|---|
| CA (1) | CA2829985C (en) |
| MX (1) | MX345262B (en) |
-
2013
- 2013-10-16 CA CA2829985A patent/CA2829985C/en not_active Expired - Fee Related
- 2013-10-24 MX MX2013012464A patent/MX345262B/en active IP Right Grant
Also Published As
| Publication number | Publication date |
|---|---|
| CA2829985A1 (en) | 2014-07-29 |
| MX2013012464A (en) | 2015-05-13 |
| MX345262B (en) | 2017-01-23 |
| NZ616680A (en) | 2015-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10169034B2 (en) | Verification of backward compatibility of software components | |
| US9075833B2 (en) | Generating XML schema from JSON data | |
| US9280442B1 (en) | System and method for generating coverage reports for software unit tests | |
| US20160357531A1 (en) | Cross-library framework architecture feature sets | |
| US20080235661A1 (en) | System and method for using annotations to automatically generate a framework for a custom javaserver faces (jsf) component | |
| US11231979B2 (en) | Method and system for generating an application programming interface based on declarative schemas for individual services | |
| AU2015315203A1 (en) | Conditional validation rules | |
| US11379209B2 (en) | Upgrade of a software product | |
| US9779014B2 (en) | Resilient mock object creation for unit testing | |
| US10140473B2 (en) | Storage and privacy service | |
| US20160070542A1 (en) | Facilitating use of techniques implemented by external library functions in process definitions of analytical applications | |
| US10503497B2 (en) | Techniques for utilizing an expression language in service configuration files | |
| US20160342808A1 (en) | Storage and privacy service | |
| CN112256252B (en) | Interface generation method and device, storage medium and electronic equipment | |
| US20110246967A1 (en) | Methods and systems for automation framework extensibility | |
| CN115328569A (en) | Method, system, electronic device, and computer-readable storage medium for handling data conflict | |
| CA2829985C (en) | System and method for validation and conversion of electronic data | |
| CN117370160A (en) | Code auditing method, apparatus, computer device, storage medium and program product | |
| CN117234517A (en) | Interface parameter verification method, device, equipment and storage medium | |
| NZ616680B (en) | System and method for validation and conversion of electronic data | |
| US20240143485A1 (en) | Presubmit Test Run Minimization Analysis Using Runtime Isolation Guarantees | |
| US20250307037A1 (en) | Declarative query schema-based application programming interface | |
| US20140200950A1 (en) | Methods and Systems for Automatic Operation Abstraction and Custom Service Instantiation | |
| US8839275B1 (en) | Method for intercepting input/output requests and responses | |
| CN117540124A (en) | Buried data generation method, system, computing device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKLA | Lapsed |
Effective date: 20201016 |