CN115544134A

CN115544134A - Data access method, device, computer equipment and storage medium

Info

Publication number: CN115544134A
Application number: CN202211156847.XA
Authority: CN
Inventors: 余辉
Original assignee: Shenzhen Coocaa Network Technology Co Ltd
Current assignee: Shenzhen Coocaa Network Technology Co Ltd
Priority date: 2022-09-22
Filing date: 2022-09-22
Publication date: 2022-12-30

Abstract

The invention relates to the technical field of software engineering, in particular to a data access method, a device, computer equipment and a storage medium, which are used for acquiring configuration information of data to be accessed by a user, performing consumption processing on the data to be accessed according to the configuration information to obtain a theme of the data to be accessed, determining a data format of the theme, selecting a corresponding calculation program from the configuration information according to the data format to perform analysis calculation on the data to be accessed to obtain analyzed data, establishing a storage table in the storage medium corresponding to the configuration information, and storing the analyzed data in the storage table. In the invention, consumption information is configured through the configuration interface, the information is analyzed and stored, a data access program is started for data access, any data to be accessed is directly accessed in a configuration mode according to the configuration information, multiple sets of codes do not need to be set according to different data sources, and the data access efficiency is improved.

Description

Data access method, device, computer equipment and storage medium

Technical Field

The present invention relates to the field of software engineering technologies, and in particular, to a data access method and apparatus, a computer device, and a storage medium.

Background

With the rapid development of internet technology, in the application of the current network technology, the big data analysis is performed on various data by accessing data of different products, so as to realize the operation of various indexes and data, which is an important content applied by the internet. In order to perform big data analysis, basic data related to each product is accessed from each product, which is an important content, and the efficiency of accessing the basic data directly affects the efficiency of subsequent analysis processing and other processes.

The current method of accessing the basic data of each product generally accesses the data source of each product, that is, corresponding program codes are written for each data source, and each data source performs various encoding analysis processes such as file verification, data reading, data verification, data writing, data warning, and the like by using the corresponding program codes. Or the data formats of all the data sources are directly unified, so that the whole process can be processed in batch by only one set of codes. However, when the data sources of the products are accessed respectively, each data source needs to have a corresponding program code, which not only needs to increase many repetitive tasks, resulting in high costs for development and testing, but also increases the code amount with the increase of the accessed data sources or data amount, and increases the number of modules that need to be added, which causes the efficiency of data access to be lower and lower, and therefore, how to improve the efficiency of data access becomes an urgent problem to be solved.

Disclosure of Invention

Accordingly, it is necessary to provide a data access method, an apparatus, a computer device and a storage medium to solve the problem of low data access efficiency.

A first aspect of an embodiment of the present application provides a data access method, where the data access method includes:

acquiring configuration information of data to be accessed of a user, performing consumption processing on the data to be accessed according to the configuration information to obtain a theme of the data to be accessed, and determining a data format of the theme;

selecting a corresponding calculation program from the configuration information according to the data format to perform analysis calculation on the data to be accessed to obtain analyzed data;

and establishing a storage table in a storage medium corresponding to the configuration information, and storing the analyzed data in the storage table.

A second aspect of an embodiment of the present application provides a data access apparatus, where the apparatus includes:

the acquisition module is used for acquiring configuration information of data to be accessed by a user, performing consumption processing on the data to be accessed according to the configuration information to obtain a theme of the data to be accessed, and determining a data format of the theme;

the analysis module is used for selecting a corresponding calculation program from the configuration information according to the data format to analyze and calculate the data to be accessed to obtain analyzed data;

and the storage module is used for establishing a storage table in a storage medium corresponding to the configuration information and storing the analyzed data to the storage table.

In a third aspect, an embodiment of the present invention provides a computer device, where the computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the data access method according to the first aspect when executing the computer program.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and when executed by a processor, the computer program implements the data access method according to the first aspect.

Compared with the prior art, the invention has the following beneficial effects:

the method comprises the steps of obtaining configuration information of data to be accessed of a user, conducting consumption processing on the data to be accessed according to the configuration information to obtain a theme of the data to be accessed, determining a data format of the theme, selecting a corresponding calculation program from the configuration information according to the data format to conduct analysis calculation on the data to be accessed to obtain analyzed data, establishing a storage table in a storage medium corresponding to the configuration information, and storing the analyzed data to the storage table. According to the invention, consumption information is configured through the configuration interface, information and storage information are analyzed, a data access program is started to carry out data access, any data to be accessed are directly subjected to configuration access according to the configuration information, a plurality of sets of codes do not need to be set according to different data sources, and the data access efficiency is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

Fig. 1 is a schematic diagram of an application environment of a data access method according to an embodiment of the present invention;

fig. 2 is a flowchart illustrating a data access method according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a data access apparatus according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Furthermore, in the description of the present invention and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

It should be understood that, the sequence numbers of the steps in the following embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

In order to explain the technical means of the present invention, the following description will be given by way of specific examples.

An embodiment of the present invention provides a data access method, which can be applied to an application environment as shown in fig. 1, where a client communicates with a server. The client includes, but is not limited to, a palm top computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and other computer devices. The server can be implemented by an independent server or a server cluster generated by a plurality of servers.

Referring to fig. 2, which is a flowchart illustrating a data access method according to an embodiment of the present invention, where the data access may be applied to a server in fig. 1, and the server is connected to a corresponding client, as shown in fig. 2, the data access method may include the following steps.

S201: the method comprises the steps of obtaining configuration information of data to be accessed of a user, conducting consumption processing on the data to be accessed according to the configuration information, obtaining a theme of the data to be accessed, and determining a data format of the theme.

In step S201, the data to be accessed is log data generated by user behaviors, for example, web browsing, searching, clicking, other user behaviors, and the like, the configuration information is information acquired by configuring the page, the data to be accessed is consumed according to the configuration information, a theme (topic) of the data to be accessed is obtained, a data format of the theme is determined, and different themes may be different data formats.

In this embodiment, corresponding configuration information is acquired through configuration operations on a configuration page, where the configuration page includes consumption configuration, analysis configuration, and storage configuration, a theme corresponding to data to be accessed is acquired through the consumption configuration, an analysis method for the data to be accessed is acquired through the analysis configuration, and a corresponding storage medium is acquired through the storage configuration.

In this embodiment, the data to be accessed is log data generated by web browsing, and the generated log data is output to an afka cluster, where Kafka is a high-throughput distributed publish-subscribe message system that can process all action flow data in a consumer-scale website, such as web browsing, searching, clicking, and other user behaviors, and in this embodiment, the collected log information can be directly output to a Kafka message queue through a log output component; wherein the logging output component comprises a log4j2 component, the kafka message queue is a queue formed by topic messages, one topic message may comprise a plurality of different storage directories, and the kafka message queue stores the log information collected by the logging output component through the storage directories.

Specifically, the acquired log information is output to a kafka message queue, namely the acquired log information is output to a storage directory of the kafka message queue; the storage directory performs log storage in such a manner that a plurality of different storage directories are equally allocated to a plurality of log data with the same size, that is, the log data are equal in size, but the number of the log data is not necessarily equal, and the number of the log data may be the same or different. The kafka message queue can be used as intermediate storage of the log information in the transmission process, can ensure the high efficiency and the persistence of the security of the log information, and meets the requirement of large-capacity intermediate storage. The log data is sent to different topics through a kafka message queue. And monitoring the theme by configuring and acquiring the corresponding kafka critical defect address and the corresponding theme, and acquiring the data format corresponding to the theme.

Optionally, the obtaining configuration information of the data to be accessed by the user includes:

acquiring an information configuration interface of data to be accessed;

receiving data to be configured corresponding to the configuration items, which are input by a user through an information configuration interface;

and acquiring the configuration information of the data to be accessed of the user according to the data to be configured.

In this embodiment, when a configuration interface is obtained, a configuration file and a description file associated with the configuration interface to be loaded are obtained according to a configuration tool, the configuration file is analyzed to obtain a configuration item, the description file is read to obtain a description item associated with the configuration item, and the configuration interface is loaded and displayed based on the configuration item and the description item. The configuration tool firstly obtains a configuration file and a description file which are associated with a configuration interface to be loaded in a specified directory. The configuration file is a set of settings and files of an environment required by loading of a configuration tool of the software system, and comprises all configuration settings, such as program items, screen colors, network connections, sizes and positions of windows and the like; by modifying the configuration file, the configuration tool of the software system can change some resources used in the program under the condition of not recompiling the program, such as an external file path, a function enabling switch and a constant used in the program running. Specifically, the configuration File includes, but is not limited to, an Initialization File (INI) or an eXtensible Markup Language (XML) File. The INI file is generally used in a microsoft Windows operating (Windows) system to manage various configurations of the Windows system; XML documents provide a format for describing structured data, simplifying the exchange and representation of data in a network, allowing code, data and representation to be separated, usually as a standard format for data exchange. The process of parsing the INI file is as follows: all sections (sections) in the INI file are read, and then key (key) value pairs contained in each section are read, wherein the key is a configuration item name, the value is a configuration item value, and the configuration item consists of the key value pair, namely the configuration item name and the configuration item value. After obtaining the configuration items in the configuration file, the configuration interface management apparatus may continue to read the description items associated with the configuration items from the description file, where the description items are used to describe the attributes of the associated configuration items, and load and display the interface of the configuration tool according to the configuration items and the description items, so as to generate a corresponding information interface.

It should be noted that the information plane includes a consumption configuration interface, an analysis configuration interface, and a storage configuration interface, selects corresponding configuration data according to configuration items in the configuration interface, and acquires corresponding configuration information based on the configuration data.

Optionally, the consuming processing is performed on the data to be accessed according to the configuration information to obtain a theme of the data to be accessed, and determining a data format of the theme includes:

determining a consumption starting position for reading data to be accessed according to the configuration information;

obtaining a theme of the data to be accessed corresponding to the consumption initial position according to the consumption initial position;

and monitoring the theme of the data to be accessed and determining the data format in the theme.

In this embodiment, the data to be accessed is consumed by configuring the stream computation execution environment of the flink data stream API, which supports the conversion (such as filter, aggregation, and window functions) on bounded or unbounded data streams and can be used in Java and Scala. The configuration of the stream computing execution environment declares and assigns variables, parameters and functions called by the flink data stream API. Determining a consumption starting position for reading the data to be accessed according to the configuration information, calling a flash data stream API according to the consumption starting position, pointing to a theme in a Kafka-based message queue, monitoring the theme of the data to be accessed, and determining a data format in the theme.

It should be noted that the data format in the theme is a JSON (JavaScript Object notification) data format, and the JSON data format includes three data formats, the first is a data format in a value form, the second is a data format in an Object form, and the third is a data format in an array form. And different themes correspond to different data formats, the data format in the theme is determined according to the theme of the data to be accessed, and the object form is a set of key and value pairs. An object starts with "{" and ends with "}", multiple key value pairs with "," separate ", and keys with": separate values. The array form is a collection of values. An array begins with "[" and ends with "]", with values separated by "intervals.

S202: and selecting a corresponding calculation program from the configuration information according to the data format to perform analysis calculation on the data to be accessed, so as to obtain the analyzed data.

In step S202, according to the data format and the parsing rule in the configuration information, a computing program corresponding to the parsing function is called to parse the data to be accessed, so as to obtain parsed data.

In this embodiment, the data to be accessed is stored in the topic topicA of the distributed message system kafka, and when the data to be accessed stored in the kafka system is read, the data to be accessed is read from the topic topicA of the kafka system, and at this time, the source of the data to be accessed is the topic topicA in the kafka system. The analysis rules can be understood as the function of analyzing the data to be accessed, each analysis rule corresponds to an analysis function according to different requirements of analyzing the data to be accessed, and the analysis rules and the analysis functions realize one-to-one correspondence through the identification of the analysis rules. Where the parsing rule identifies a set of numbers that may be randomly assigned.

It should be noted that each parsing rule may be displayed in a list form for a programmer to select, and when it is detected that a certain parsing rule is selected, the corresponding parsing function may be searched according to the parsing rule identifier corresponding to the parsing rule. Specifically, the analytic function corresponding to each analytic rule is pre-stored in the function library, and the analytic function corresponding to the selected analytic rule may be searched in the function library according to the identifier corresponding to the selected analytic rule.

For example, if a parsing rule "timestamp converted to time text" is selected, a parsing function corresponding to the conversion of the timestamp to time text is looked up in the function library based on the parsing rule identification of the selected parsing rule.

Optionally, selecting a corresponding calculation program from the configuration information according to the data format to perform analysis calculation on the data to be accessed, so as to obtain analyzed data, where the method includes:

identifying the data format and the field name of the data to be accessed based on the calculation program;

carrying out separation processing on data to be accessed to obtain a separator list;

and analyzing the data to be accessed according to the separator list and the field name to obtain analyzed data.

In this embodiment, the format of the data to be accessed is a JSON data format, the separator nodes and the field name nodes in the data to be accessed are processed to obtain an analyzed separator list, and data analysis is implemented through the separator list.

It should be noted that, before analyzing the data to be accessed, the data to be accessed may also be preprocessed to obtain effective data to be accessed. For example, the first field in the log data is typically the date the log was generated; or a field with the same attribute although changed, for example, a fixed position in nova data may display the request _ ID. The parsing of the log by these items not only does not help, but also increases the parsing cost. The extraneous item can be removed through the corresponding interface pair.

It should be noted that different expressions of the same data can be processed during preprocessing, although the log formats generated by different systems are different, and some fields in the log follow the "international convention", such as IP addresses, timestamp formats, file paths, and so on. The expression of the fields in the respective logs is diversified, and a certain load is added to the log analysis process without preprocessing.

Optionally, analyzing the data to be accessed according to the separator list and the field name to obtain analyzed data, including:

associating the field name with the separator list to obtain a separator association result;

and analyzing the data to be accessed according to the separator correlation result to obtain analyzed data.

In this embodiment, the field names are multiple, the parsing process of the data to be accessed needs to associate the multiple field names with the separator list to obtain multiple separator association results, which is a process of repeating association and separation processing, and the data to be accessed is completely parsed by loop processing, and finally the parsed data.

It should be noted that after the analyzed data is obtained, the analyzed data needs to be verified, and the analyzed data is verified by using a regular expression, for example, by using/[ 0-9] {1,20} $/this regular expression to verify whether the analysis result is composed of numbers.

S203: and establishing a storage table in a storage medium corresponding to the configuration information, and storing the analyzed data in the storage table.

In step S203, the analyzed data is placed in the corresponding storage table in the storage configuration interface according to the analyzed data.

In this embodiment, a storage table corresponding to the storage configuration interface is configured, the analyzed data is placed in the storage table, and the data in the storage table is stored in a corresponding storage medium, where the storage medium may be a data lake HUDI, a big data storage medium Hive in a distributed file system, and the like.

It should be noted that Hudi is an open source solution for Data Lakes, and Hudi is a shorthand for Hadoop Updates and indexes, which is a Data Lakes solution developed and open source by Uber.

In this embodiment, the data to be accessed, which is consumed by the Flink framework from a theme, is stored in the data table corresponding to the theme in the storage medium. That is, one theme corresponds to one data table. For example, the data consumed in the Topic a may be stored in a hudi table a, the data consumed from the Topic b may be stored in a hudi table b, and the low-level intervention data may be stored in a corresponding storage medium in a form of a data table, so as to query and process the data to be accessed.

Optionally, establishing a storage table in a storage medium corresponding to the configuration information, and storing the analyzed data in the storage table, including:

generating an SQL statement corresponding to the analyzed data according to the configuration information;

and storing the analyzed data to a storage table based on the SQL sentences corresponding to the analyzed data.

In this embodiment, according to the configuration information, SQL statements corresponding to the analyzed data are generated, in the storage configuration interface, the analyzed data are displayed in the corresponding storage configuration interface, and the analyzed data are dragged to the storage table displayed in the storage interface, where it should be noted that the storage configuration interface includes multiple storage tables, a corresponding storage table may be selected according to the attribute information of the analyzed data, and the analyzed data are put into the corresponding storage table, where the storage table is a data table established in the current storage medium, and the storage table may include a database name, a table name, and each field under the table, such as id (primary key id), and the like; wherein id is field name, and primary key id is field label.

It should be noted that a draggable component library is integrated in the VUE framework of the storage configuration interface, so that the database table field in the current page has a draggable attribute. The dragging instruction is an instruction for dragging the analyzed data to the storage table, wherein the analyzed data and the storage table are dragged to another area from one area in different areas of the storage configuration interface, under the condition that the analyzed data are multiple, multiple fields can be dragged simultaneously through one dragging operation, one field can also be dragged at one time, and a user can select the fields according to actual requirements.

And according to the SQL code segment of the storage table, generating a format according to the SQL statement, splicing with the SQL code segment generated after the data is analyzed, obtaining a corresponding SQL statement, and storing the SQL statement.

Optionally, generating an SQL statement corresponding to the parsed data according to the configuration information includes:

acquiring field information corresponding to the analyzed data configured by the item to be configured according to the configuration information;

and filling the SQL sentences corresponding to the storage table according to the field information to obtain the SQL sentences corresponding to the analyzed data.

In this embodiment, when the analyzed data is selected for configuration in the storage configuration interface, field information in the configuration item is obtained, where the field information includes a field name, and the SQL statement in the storage table is extracted, and when the corresponding data is selected for configuration in the analyzed data, the selected field information corresponding to the corresponding data is filled in the SQL statement in the storage table, so as to obtain the SQL statement corresponding to the analyzed data.

For example, the SQL statement corresponding to the storage table is SQL > DESC emp, the data table corresponding to emp is queried, after the field information corresponding to the configuration item is obtained, for example, data in all columns in the emp data table is configured, after the SQL statement of the storage table is filled, the SQL statement SQL > SELECT FROM emp corresponding to the field information in all columns is obtained, when a specified column in the storage table is queried, the corresponding query statement is SQL > SELECT DISTINCT mgr FROM emp, wherein DISTINCT is the field information in the specified column.

Referring to fig. 3, fig. 3 is a schematic structural diagram of a data access device according to an embodiment of the present invention. The units included in the terminal in this embodiment are used to execute the steps in the embodiment corresponding to fig. 2. Please refer to fig. 2 and the related description of the embodiment corresponding to fig. 2. For convenience of explanation, only the portions related to the present embodiment are shown.

Referring to fig. 3, the data access device 30 includes:

the obtaining module 31 is configured to obtain configuration information of data to be accessed by a user, perform consumption processing on the data to be accessed according to the configuration information, obtain a theme of the data to be accessed, and determine a data format of the theme;

the analysis module 32 is configured to select a corresponding calculation program from the configuration information according to the data format to perform analysis calculation on the data to be accessed, so as to obtain analyzed data;

and the storage module 33 is configured to establish a storage table in the storage medium corresponding to the configuration information, and store the analyzed data in the storage table.

Optionally, the obtaining module 31 includes:

the information configuration interface acquisition unit is used for acquiring an information configuration interface of data to be accessed;

the configuration unit is used for receiving data to be configured, which is input by a user through the information configuration interface and corresponds to the configuration items;

and the configuration information acquisition unit is used for acquiring the configuration information of the data to be accessed by the user according to the data to be configured.

Optionally, the obtaining module 31 includes:

the consumption starting position determining unit is used for determining the consumption starting position for reading the data to be accessed according to the configuration information;

the theme determining unit is used for obtaining a theme of the data to be accessed corresponding to the consumption starting position according to the consumption starting position;

and the data format acquisition unit is used for monitoring the theme of the data to be accessed and determining the data format in the theme.

Optionally, the parsing module 32 includes:

the field name identification unit of the access data is used for identifying the field name of the data to be accessed based on a calculation program;

the separation unit is used for separating the data to be accessed to obtain a separator list;

and the analyzed data determining unit is used for analyzing the data to be accessed according to the separator list and the field name to obtain the analyzed data.

Optionally, the analyzed data determining unit includes:

the association subunit is used for associating the field name with the separator list to obtain a separator association result;

and the sub-unit Fu Jiexi is divided and used for analyzing the data to be accessed according to the separator association result to obtain analyzed data.

Optionally, the storage module 33 includes:

the SQL sentence generating unit is used for generating an SQL sentence corresponding to the analyzed data according to the configuration information;

and the storage table storage unit is used for storing the analyzed data into a storage table based on the SQL sentences corresponding to the analyzed data.

Optionally, the SQL statement generating unit includes:

the field information determining subunit is used for acquiring field information corresponding to the analyzed data configured by the item to be configured according to the configuration information;

and the filling subunit is used for filling the SQL statement corresponding to the storage table according to the field information to obtain the SQL statement corresponding to the analyzed data.

It should be noted that, because the contents of information interaction, execution process, and the like between the above units are based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof may be specifically referred to a part of the method embodiment, and details thereof are not described herein.

Fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention. As shown in fig. 4, the computer apparatus of this embodiment includes: at least one processor (only one shown in fig. 4), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor when executing the computer program implementing the steps in any of the various data access method embodiments described above.

The computer device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that fig. 4 is merely an example of a computer device and is not intended to be limiting, and that a computer device may include more or fewer components than those shown, or some components may be combined, or different components may be included, such as a network interface, a display screen, and input devices, etc.

The Processor may be a CPU, or other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory includes readable storage media, internal memory, etc., wherein the internal memory may be the internal memory of the computer device, and the internal memory provides an environment for the operating system and the execution of the computer-readable instructions in the readable storage media. The readable storage medium may be a hard disk of the computer device, and in other embodiments may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device. Further, the memory may also include both internal and external storage units of the computer device. The memory is used for storing an operating system, application programs, a BootLoader (BootLoader), data, and other programs, such as program codes of a computer program, and the like. The memory may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the present invention. The specific working processes of the units and modules in the above-mentioned apparatus may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method of the above embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium and used by a processor to implement the steps of the above method embodiments. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In some jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and proprietary practices.

The present invention may also be implemented by a computer program product, which when executed on a computer device, enables the computer device to implement all or part of the processes in the method according to the above embodiments.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/computer device and method may be implemented in other ways. For example, the above-described apparatus/computer device embodiments are merely illustrative, and for example, a module or a unit may be divided into only one logical function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein.

Claims

1. A data access method, characterized in that the data access method comprises:

2. The data access method of claim 1, wherein the obtaining configuration information of the user about data to be accessed comprises:

acquiring an information configuration interface of the data to be accessed;

and acquiring the configuration information of the user on the data to be accessed according to the data to be configured.

3. The data access method of claim 1, wherein the consuming the data to be accessed according to the configuration information to obtain a theme of the data to be accessed, and determining a data format of the theme comprises:

determining a consumption starting position for reading the data to be accessed according to the configuration information;

obtaining a theme of the data to be accessed corresponding to the consumption starting position according to the consumption starting position;

4. The data access method of claim 1, wherein the selecting a corresponding calculation program from the configuration information according to the data format to perform parsing calculation on the data to be accessed to obtain parsed data includes:

identifying a field name of the data to be accessed based on the computing program;

separating the data to be accessed to obtain a separator list;

5. The data access method of claim 4, wherein the parsing the data to be accessed according to the delimiter list and the field name to obtain parsed data comprises:

and analyzing the data to be accessed according to the separator association result to obtain analyzed data.

6. The data access method of claim 1, wherein the creating a storage table in a storage medium corresponding to the configuration information and storing the parsed data in the storage table comprises:

and storing the analyzed data to the storage table based on the SQL sentence corresponding to the analyzed data.

7. The data access method of claim 6, wherein the generating the SQL statement corresponding to the parsed data according to the configuration information comprises:

8. A data access apparatus, the apparatus comprising:

9. A computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the data access method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the data access method according to any one of claims 1 to 7.