CN113297203A

CN113297203A - Data query and write-in method and device, computer storage medium and electronic equipment

Info

Publication number: CN113297203A
Application number: CN202010678385.2A
Authority: CN
Inventors: 李辉; 潘岳; 胡进; 肖孝云
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-07-15
Filing date: 2020-07-15
Publication date: 2021-08-24
Anticipated expiration: 2040-07-15
Also published as: CN113297203B

Abstract

The application discloses a data query method and device, a query data acquisition method and device, a query data identification information sending method and device, a data writing request sending method and device, a written data reading method and device, an analytical database system, a computer storage medium and an electronic device; the query method comprises the following steps: acquiring a data query request; determining whether the identification information of the target data to be queried is matched with the stored identification information of the current written data or not according to the data query request; if not, acquiring new written data which is adjacent to the identification information of the current written data and has a writing sequence after the current written data and the identification information of the new written data; determining whether the identification information of the newly written data is matched with the identification information of the target data to be inquired; if yes, determining the newly written data as target data to be queried; inaccurate or deviation of query data caused by delay and the like is avoided.

Description

Data query and write-in method and device, computer storage medium and electronic equipment

Technical Field

The application relates to the field of computer application, in particular to a data query method and device, a query data acquisition method and device, a query data identification information sending method and device, a data writing request sending method and device, a written data reading method and device, an analytic database system, a computer storage medium and electronic equipment.

Background

The traditional database is mainly used for storing data and performing related data processing operations such as reading and writing aiming at the data. With the development of computer information technology, a ubiquitous great change is the generation of big data, and because it is very important to extract useful information from collected data so as to improve the development positioning of various industries and promote economic development, a large amount of data is needed to provide a service basis for related operations such as the extraction of subsequent data information.

With the continuous development of big data in recent years, various industries have higher requirements on the exploration of data values. In a traditional data storage system for big data, operations such as offline read-write calculation and the like cannot meet the work of exploring information data values such as data online statistics, data online analysis, ad hoc query and the like. As a result, the conventional database facing the processing of big data (BigData) is gradually developing into a database capable of providing higher requirements for timeliness and processing speed.

Disclosure of Invention

The application provides a data query method to solve the problem that queried data in the prior art is poor in real-time visibility.

The application provides a data query method, which comprises the following steps:

acquiring a data query request, wherein the data query request carries identification information of target data to be queried, the identification information is used for identifying information of a write-in data write-in sequence, and the identification information of the target data to be queried is the identification information of the write-in data written last determined according to a write-in sequence range;

determining whether the identification information of the target data to be inquired is matched with the stored identification information of the current written data or not according to the data inquiry request;

if not, acquiring new written data which is adjacent to the identification information of the current written data and has a writing sequence after the current written data and the identification information of the new written data;

determining whether the identification information of the newly written data is matched with the identification information of the target data to be inquired;

and if so, determining the new write-in data as the target data to be queried.

In some embodiments, further comprising:

reading the written data information;

determining the identification information in the read write data information as the identification information of the current write data;

storing the written data corresponding to the identification information of the current written data and the identification information of the current written data locally;

the determining whether the identification information of the target data to be queried matches with the stored identification information of the current written data according to the data query request includes:

and determining whether the identification information of the target data to be inquired is matched with the identification information of the current write data stored locally.

In some embodiments, the reading the write data information comprises:

and reading the written data information according to a reading period in an asynchronous mode.

In some embodiments, further comprising:

and when the identification information of the newly written data is matched with the identification information of the target data to be inquired, updating the currently written data into the acquired newly written data, and updating the identification information of the currently written data into the identification information of the newly written data.

In some embodiments, the determining whether the identification information of the target data matches the identification information of the stored current write data according to the data query request includes:

comparing the identification code of the identification information of the target data to be inquired with the identification code of the identification information of the currently written data;

and if the identification code of the identification information of the target data to be queried is larger than the identification code of the identification information of the currently written data, determining that the identification code of the identification information of the target data to be queried is not matched with the identification code of the identification information of the currently written data.

In some embodiments, further comprising:

if the identification code of the identification information of the target data to be queried is less than or equal to the identification code of the identification information of the current written data, determining that the identification code of the identification information of the target data to be queried is matched with the identification code of the identification information of the current written data;

and determining the current write data as the target data to be queried.

In some embodiments, the determining whether the identification information of the new written data matches the identification information of the target data to be queried includes:

comparing the identification code of the identification information of the newly written data with the identification code of the identification information of the target data to be inquired;

and if the identification code of the identification information of the newly written data is greater than or equal to the identification code of the identification information of the target data to be inquired, determining that the identification information of the newly written data is matched with the identification information of the target data.

In some embodiments, the comparing the identification code of the identification information of the newly written data with the identification code of the identification information of the target data to be queried further includes:

if the identification code of the identification information of the newly written data is smaller than the identification code of the identification information of the target data to be inquired, determining that the identification information of the newly written data is not matched with the identification information of the target data;

and returning to the step of acquiring the new write data which is adjacent to the identification information of the current write data and has the write sequence after the current write sequence and the identification information of the new write data.

In some embodiments, further comprising:

and sending the determined target data to be inquired.

The present application also provides a data query apparatus, including:

the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a data query request, and the data query request carries identification information of target data to be queried, the identification information is used for identifying information of a write-in data write-in sequence, and the identification information of the target data to be queried is identification information of the write-in data which is written last and is determined according to a write-in sequence range;

a first determining unit, configured to determine, according to the data query request, whether the identification information of the target data to be queried matches with the identification information of the stored current write data;

a second obtaining unit, configured to obtain, when the matching result of the first determining unit is not matched, new write data and identification information of the new write data that are adjacent to the identification information of the current write data and have a write order that is subsequent to the current write order;

a second determining unit, configured to determine whether the identification information of the newly written data matches the identification information of the target data to be queried;

and the third determining unit is used for determining the new written data as the target data to be inquired when the matching result in the second determining unit is matching.

The application also provides a query data acquisition method, which comprises the following steps:

receiving a data query request;

acquiring identification information of newly written data according to the data query request, wherein the identification information of the newly written data is the identification information of the last written data determined according to the writing sequence range;

packaging the identification information into the data query request and sending the data query request;

and acquiring target data returned in response to the data query request encapsulated with the identification information.

The present application further provides an apparatus for acquiring query data, including:

a receiving unit, configured to receive a data query request;

a first obtaining unit, configured to obtain, according to the data query request, identification information of newly written data, where the newly written data is last written data determined according to a writing sequence range;

the packaging sending unit is used for packaging the identification information into the data query request and sending the data query request;

and the second acquisition unit is used for acquiring target data returned in response to the data query request encapsulated with the identification information.

The application also provides a sending method for inquiring the data identification information, which comprises the following steps:

receiving an acquisition request aiming at identification information of newly written data, wherein the identification information of the newly written data is the identification information of the last written data determined according to a writing sequence range;

and according to the acquisition request, sending the identification information of the last written data determined according to the writing sequence range as the identification information of the new written data.

The present application further provides a sending apparatus for querying data identification information, including:

a receiving unit, configured to receive an acquisition request for identification information of newly written data, where the identification information of the newly written data is identification information of last written data determined according to a writing sequence range;

and the sending unit is used for sending the identification information of the last written data determined according to the writing sequence range as the identification information of the new written data according to the acquisition request in the receiving unit and the acquisition request.

The present application further provides a data writing method, including:

receiving a data writing request, wherein the data writing request carries writing data needing to be written;

configuring identification information for the written data according to the data writing request; wherein the identification information is used for identifying the information of the writing sequence of the written data;

and performing writing operation according to the written data and the identification information of the written data.

In some embodiments, further comprising:

receiving state information for describing a write state of the write data;

and sending the state information.

In some embodiments, the receiving a data write request includes:

and receiving data write requests of different write data aiming at the same write partition according to the set received data write request range.

In some embodiments, the receiving data write requests for different write data of the same write partition according to the set received data write request range includes:

receiving data writing requests of different writing data aiming at the same writing subarea according to the set receiving time range for receiving the data writing requests; or,

and receiving data writing requests of different writing data aiming at the same writing subarea according to the set receiving capacity range for receiving the data writing requests.

In some embodiments, said configuring identification information for said write data according to said data write request includes:

determining the different write data in the data write request as a write data set;

configuring the identification information for the write data set.

In some embodiments, the performing a write operation according to the write data and the identification information of the write data includes:

and performing writing operation according to the written data set and the identification information corresponding to the written data set.

The present application also provides a data writing apparatus, including:

a receiving unit configured to receive a data write request; the data writing request carries writing data needing to be written;

the configuration unit is used for configuring identification information for the written data according to the data writing request; wherein the identification information is used for identifying the information of the writing sequence of the written data;

and the writing unit is used for performing writing operation according to the written data and the identification information of the written data.

The application also provides a method for sending a data write request, which comprises the following steps:

sending a data writing request, wherein the data writing request carries writing data needing to be written;

receiving write status information for the write data returned in response to the data write request.

In some embodiments, said sending said data write request comprises:

determining a cache node corresponding to the written data according to attribute information which is carried in the data writing request and used for describing the written data;

and sending the data write request to the corresponding cache node.

In some embodiments, the determining, according to the attribute information for describing the write data carried in the data write request, a cache node corresponding to the write data includes:

establishing write-in path information aiming at the write-in data according to the identifier of the write-in database and the identifier of the write-in data table which describe the write-in data in the attribute information;

acquiring partition information of the write data table according to the write path information;

and determining the cache node corresponding to the written data according to the partition information.

The present application also provides a device for sending a data write request, including:

the data writing device comprises a sending unit, a writing unit and a writing unit, wherein the sending unit is used for sending a data writing request, and the data writing request carries writing data needing to be written;

a receiving unit, configured to receive write status information for the write data returned in response to the data write request.

The present application further provides a method for reading write data, including:

reading the written data and the identification information of the written data according to the sent reading request for reading the written data information;

determining the written data as current written data, and determining the identification information of the written data as the identification information of the current written data;

and storing the current written data and the identification information of the current written data locally.

In some embodiments, the reading the write data and the identification information of the write data according to the sent read request for reading the write data information includes:

sending the reading request according to a reading period in an asynchronous mode;

and reading the written data and the identification information of the written data according to the reading request.

The present application also provides a reading apparatus for writing data, including:

the reading unit is used for reading the written data and the identification information of the written data according to the sent reading request for reading the written data information;

a determining unit, configured to determine the write data as current write data, and determine identification information of the write data as identification information of the current write data;

and the storage unit is used for locally storing the current written data and the identification information of the current written data.

The present application further provides an analytic database system, comprising: the system comprises a front-end node, a cache node and a computing node;

the front-end node is used for sending a data operation request to the cache node, and returning operation state or target data or identification information of newly written data aiming at the data operation request; when the data operation request received by the front-end node is a data write request, sending the received data write request to the cache node, and receiving write state information, which is returned by the cache node in response to the data write request and aims at the data write request, of write data; when the data operation request received by the front-end node is a data query request, acquiring identification information of newly written data from the cache node according to the received data query request, wherein the identification information of the newly written data is the identification information of the last written data determined according to a writing sequence range, packaging the identification information of the newly written data into the data query request, and sending the data query request to the computing node to acquire target data determined by the computing node for the data query request;

the cache node is used for receiving the data operation request from the front-end node and sending information responding to the data operation request to the front-end node; when the data operation request received by the cache node is the data writing request, configuring identification information for written data carried in the data writing request according to the data writing request; the identification information is used for identifying the information of the writing sequence of the written data, and writing operation is carried out according to the written data and the identification information of the written data; when the data operation request received by the cache node is the data query request, the cache node receives an acquisition request for identification information of new write data from the front-end node, and according to the acquisition request, the identification information of the last write data determined according to the write sequence range is sent to the front-end node as the identification information of the new write data;

the computing node is configured to obtain a data query request from the front-end node, where the data query request carries identification information of target data to be queried, where the identification information is used to identify information of a write sequence of write data, and the identification information of the target data to be queried is identification information of the write data written last determined according to a write sequence range; determining whether the identification information of the target data to be inquired is matched with the stored identification information of the current written data or not according to the data inquiry request; if not, acquiring new write data which is adjacent to the identification information of the current write data and has a write sequence behind the current write sequence and the identification information of the new write data; determining whether the identification information of the newly written data is matched with the identification information of the target data to be inquired; if yes, determining the newly written data as the target data to be queried; or, reading the written data and the identification information of the written data according to a read request for reading the written data information sent to the cache node; determining the written data as current written data, and determining the identification information of the written data as the identification information of the current written data; and storing the current written data and the identification information of the current written data locally.

In some embodiments, further comprising:

the service coordination node is used for providing the cache node corresponding to the written data according to the attribute information of the written data in the data writing request in the front-end node;

and the front-end node sends the data write request to the cache node corresponding to the position information according to the cache node provided by the service coordination node.

In some embodiments, the service coordination node provides partition information of the write data table according to write path information for the write data, which is established by the front-end node based on an identifier of a write database and an identifier of a write data table used for describing the write data in the attribute information;

the front-end node can determine the cache node corresponding to the write-in data according to the address and the port of the cache node in the partition information.

In some embodiments, further comprising: and the storage node is used for storing the write-in data which is received by the cache node and needs to be written in, and returning write-in state information aiming at the write-in data.

The application also provides a computer storage medium for storing the data generated by the network platform and a program for processing the data generated by the network platform;

when the program is read and executed, the following steps are executed:

if not, acquiring new write data which is adjacent to the identification information of the current write data and has a write sequence behind the current write sequence and the identification information of the new write data;

if yes, determining the newly written data as the target data to be queried;

or, the following steps are executed:

receiving a data query request;

acquiring target data returned in response to the data query request encapsulated with the identification information;

or, the following steps are executed:

according to the acquisition request, sending the identification information of the last written data determined according to the writing sequence range as the identification information of the new written data;

or, the following steps are executed:

performing writing operation according to the written data and the identification information of the written data;

or, the following steps are executed:

receiving write status information for the write data returned in response to the data write request;

or, the following steps are executed:

The present application further provides an electronic device, comprising:

a processor;

a memory for storing a program for processing network platform generated data, said program when read and executed by said processor performing the steps of:

if yes, determining the newly written data as the target data to be queried;

or, the following steps are executed:

receiving a data query request;

or, the following steps are executed:

Compared with the prior art, the method has the following advantages:

the application provides a data query method, which comprises the following steps: acquiring a data query request, wherein the data query request carries identification information of target data to be queried, the identification information is used for identifying information of a write-in data write-in sequence, and the identification information of the target data to be queried is the identification information of the write-in data written last determined according to a write-in sequence range; determining whether the identification information of the target data to be inquired is matched with the stored identification information of the current written data or not according to the data inquiry request; if not, acquiring new written data which is adjacent to the identification information of the current written data and has a writing sequence after the current written data and the identification information of the new written data; determining whether the identification information of the newly written data is matched with the identification information of the target data to be inquired; and if so, determining the new write-in data as the target data to be queried. Whether the current written data is the latest data written into the analytical database system can be determined by matching the identification information in the data query request, so that the latest real-time data can be queried, and inaccurate or biased query data caused by factors such as delay can be avoided.

The application provides a data writing method, which comprises the following steps: receiving a data writing request, wherein the data writing request carries writing data needing to be written; configuring identification information for the written data according to the data writing request; wherein the identification information is used for identifying the information of the writing sequence of the written data; and performing writing operation according to the written data and the identification information of the written data. Therefore, information identification information for identifying the writing sequence of the written data can be configured for the written data when the data are written, and the real-time performance of data query is ensured.

Drawings

FIG. 1 is a flow chart of an embodiment of a data query method provided by the present application;

FIG. 2 is a schematic diagram illustrating a query principle of an embodiment of a data query method provided in the present application;

FIG. 3 is a schematic structural diagram of an embodiment of a data query apparatus provided in the present application;

FIG. 4 is a flowchart of an embodiment of a method for obtaining query data according to the present application;

FIG. 5 is a schematic structural diagram of an embodiment of an apparatus for acquiring query data according to the present application;

fig. 6 is a flowchart of an embodiment of a method for sending query data identification information according to the present application;

fig. 7 is a schematic structural diagram of an embodiment of a sending apparatus for querying data identification information provided in the present application;

FIG. 8 is a flow chart of an embodiment of a data writing method provided herein;

FIG. 9 is a schematic diagram illustrating a writing principle of an embodiment of a data writing method provided in the present application;

FIG. 10 is a schematic structural diagram of an embodiment of a data writing apparatus provided in the present application;

FIG. 11 is a flowchart of an embodiment of a method for sending a data write request according to the present application;

FIG. 12 is a schematic structural diagram of an embodiment of a device for sending a data write request according to the present application;

FIG. 13 is a flowchart of an embodiment of a method for reading data written in accordance with the present application;

FIG. 14 is a schematic structural diagram of an embodiment of a reading apparatus for writing data provided in the present application;

fig. 15 is a schematic structural diagram of an embodiment of an analytic database system provided in the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The description used in this application and in the appended claims is for example: the terms "a," "an," "first," and "second," etc., are not intended to be limiting in number or order, but rather are used to distinguish one type of information from another.

In combination with the description in the background art, with the development of big data, the analysis of the big data faces higher real-time visibility requirements, so as to improve the accuracy of data analysis. In the scene of mass data analysis, efficient writing of mass data and real-time query of the mass data need to be guaranteed, so that a user can be endowed with the capability of exploring the real-time data value, the increasingly strong demand of fast data (FastData) is met, and the data value is improved. However, the big data analysis technology provided in the prior art can only achieve near real time, and even near real time is difficult to achieve. By near real-time is meant the scale that can be felt in human-computer interaction (in seconds), rather than the real-time scale of machine automation (ns/us scale).

Currently, for big data analysis, the prior art provides two big data analysis techniques:

one is that Hive (data warehouse analysis system) is constructed based on Hadoop (distributed system infrastructure), and a rich SQL query mode is provided to realize the query analysis capability of big data. Hive maps the structured data File stored in HDFS (Hadoop Distributed File System) into one database table, and converts SQL submitted by a user into MapReduce (programming model for parallel operation of large-scale data sets (larger than 1 TB)) task operation, so as to realize large data analysis in SQL mode. The Hadoop depended on the Hive bottom layer can solve most problems of big data analysis, however, as a MapReduce task is executed on the Hadoop, specifically, a user puts a batch of data into the Hadoop to execute, and obtains the analyzed result, the execution time can also have significant difference according to the difference of data quantity, usually the difference of the execution time is from hours to days, and the newly written data can not participate in related analysis and calculation in the execution process, the Hive cannot guarantee timeliness when the Hadoop is adopted to analyze big data, and therefore the Hive does not have the capacity of real-time data visibility.

The other is PALO (data warehouse analysis system), which is a parallel database system for analyzing big data at hundred TB-PB level in real time and has low-delay point-and-search and high-throughput Ad-hoc (Ad-hoc query) query capabilities. At present, PALO supports two data import modes, namely object storage file import and local file import, and the near-real-time data can be visible by increasing the data import frequency.

Therefore, no matter which scheme is adopted, the real-time visualization of data analysis cannot be realized, and the accuracy of the data analysis cannot be correspondingly guaranteed.

Based on the analysis, the data query method provided by the application can ensure real-time visibility of data query so as to improve the accuracy of data analysis. Referring to fig. 1 and fig. 2, fig. 1 is a flowchart of an embodiment of a data query method provided by the present application, and fig. 2 is a schematic diagram of a query principle of the embodiment of the data query method provided by the present application. The query method of this embodiment is mainly described in terms of a computing node (ComputeNode) in an analytic database system, and specifically includes:

step S101: acquiring a data query request, wherein the data query request carries identification information of target data to be queried, and the identification information is used for identifying information of a write-in data writing sequence.

In the present embodiment, the data analysis can be mainly understood as a query operation on data and related data processing performed for the query operation.

The identification information of the target data to be queried, which is carried in the data query request, may be information for identifying a writing sequence of the written data, and the identification information may be represented in the form of a version number, and identifies the written data of different batches in the form of the version number. For the reasons that the present invention mainly relates to the operation when writing data, and therefore not described herein too much, please refer to the description of the following data writing portion, it should be explicitly stated that the writing data can be understood as including: insert, modify, delete, etc., operations involving data updates.

In this embodiment, the data query request may be a data query request received from a front end node (FrontNode), that is, the data query request sent by the front end node may carry a version number of target data to be queried, where identification information of the target data to be queried is a version number of the last written data determined according to a writing sequence range, that is, the data query request carries a version number of the last written data. If the version number takes a numerical value as an identification type and is set according to an ascending order, in order to ensure the real-time visibility of the data to be inquired, the data inquiry request carries the version number which is the largest in the range of the version number of the write-in data, and similarly, if the version number is set according to a descending order, the data inquiry request carries the version number which is the smallest in the range of the version number of the write-in data. Usually, the version number is determined to be the latest written data at the maximum, and the same is true in the present embodiment.

Step S102: and determining whether the identification information of the target data to be inquired is matched with the stored identification information of the current written data or not according to the data inquiry request.

The identification information of the current write data in step S102 may be write data read from a buffer node (buffer node) and stored locally. Of course, in this embodiment, for convenience of subsequent operations, the current write data is stored locally, but it does not mean that the current write data can only be stored locally, and the purpose of implementing the subsequent related operations can be achieved, and the storage location of the current write data is not limited at all.

Based on the above, it can be understood that after the data query request is obtained, the identification information of the target data to be queried in the data query request needs to be matched with the identification information of the currently written data, so that the method needs to obtain the identification information of the currently written data, and therefore the method further includes:

step S100+ 11: reading the written data information;

step S100+ 12: determining the identification information in the read write data information as the identification information of the current write data;

step S100+ 13: storing the written data corresponding to the identification information of the current written data and the identification information of the current written data locally;

step S100+ 14: and determining whether the identification information of the target data to be inquired is matched with the identification information of the current write data stored locally.

In the step S100+11, the reading and writing of the data information is mainly to read and write the data information to the cache node. The specific implementation procedure may be to read the written data information in an asynchronous manner according to a reading period, where the reading period may be, for example, 1 ms. 1ms sends a read request for reading write data once to a cache node, the cache node sends corresponding read write data information to the computing node according to the read request, and the write data information comprises: identification information of the write data and the write data.

The reading of the current write data can be performed according to a set reading period, or a reading operation of reading the current write data can be performed according to the obtained data query request.

In this embodiment, the computing node may read the write data in the cache node through an asynchronous data reading module (asyncincdataReaderservice), and specifically, may read the write data from a data service module (DataServer) in the cache node through an asynchronous data reading module (asyncincdataReaderservice); of course, the write data may also be read from the data service module (DataServer) in the cache node by a synchronization data reading module (syncincdatareadervice) in the compute node, and it should be noted that the write data mainly refers to incremental data in this embodiment.

How to match the identification information of the target data to be queried with the identification information of the stored current written data in step S102 may include the following steps:

step S102-1: comparing the identification code of the identification information of the target data to be inquired with the identification code of the identification information of the currently written data;

according to the description in step S101, if the identification information is version information, the identification code is a version number, and in this embodiment, the version number is represented by a numerical type. The specific implementation of step S102-1 may be to compare the value of the target data version number to be queried with the value of the stored current written data version number, where the same value is matching, and the different values are not matching.

Step S102-2: and if the identification code of the identification information of the target data to be queried is larger than the identification code of the identification information of the currently written data, determining that the identification code of the identification information of the target data to be queried is not matched with the identification code of the identification information of the currently written data.

The identification code of the identification information of the target data to be queried in the step S102-2 is greater than the identification code of the identification information of the currently written data, which indicates that the identification information of the currently written data stored in the computing node and the currently written data are not the latest written data, because, in this embodiment, in the above-mentioned ranking of the version numbers, the larger the version number is, the closer the time of writing the written data is to the request time of the data query request is, that is, the currently written data stored in the computing node is not the latest written data in the analytic database system.

In this embodiment, when the identification code of the identification information of the target data to be queried is less than or equal to the identification code of the identification information of the currently written data, it is determined that the identification code of the identification information of the target data to be queried matches the identification code of the identification information of the currently written data. That is, the value of the version number of the target data to be queried is less than or equal to the value of the version number of the current written data, which indicates that the version number of the target data to be queried is matched with the version number of the current written data. Namely: when the value of the version number of the target data to be queried is smaller than the value of the version number of the currently written data, it indicates that the target data to be queried can be near-real-time data, and when the value of the version number of the target data to be queried is equal to the value of the version number of the currently written data, it indicates that the target data to be queried is real-time data, and the currently written data needs to be determined as the target data to be queried.

And if the current write data are matched with the target data to be inquired, determining the current write data as the target data to be inquired.

If the value of the version number of the target data to be queried is greater than the value of the version number of the currently written data, that is, if the values are not matched, the following step S103 is executed.

Step S103: if not, acquiring new write data which is adjacent to the identification information of the current write data and has a write sequence behind the current write sequence and the identification information of the new write data;

generally, the writing sequence represents the sequence of data writing into the analytical database system, i.e. represents old and new, so-called old and new, i.e. the data written first is old and the data written later is new. Since the write-after data is incremental data for the write-before data, the total amount of the write-after data and the existing data amount before the write-after data is usually larger than the sum of the write-before data and the existing data amount. In the present embodiment, the data written into the analytical database system at each time is a data increment.

The step S103 is based on that, in the step S102, when the identification information of the target data to be queried does not match the identification information of the stored currently written data, new written data needs to be read from the cache node, and when the identification information of the target data to be queried does not match the identification information of the stored currently written data, as described in the step S101, the new written data is read from a data service module (datacerver) in the cache node through an asynchronous data reading module (asyncincdatareadservice) in the compute node, where the new written data is data read adjacent to the identification information of the currently written data currently stored in the compute node and written sequentially after the currently written data and corresponding identification information, that is, the version number value of the read new written data is adjacent to the currently written data value, the reason why the number is greater than the number of the version number of the current write-in data is that each time data is written in, the incremental data are not included, and therefore, in order to obtain complete data, reading needs to be performed sequentially according to a write-in sequence.

When new write data is read, step S104 needs to be executed.

Step S104: and determining whether the identification information of the newly written data is matched with the identification information of the target data to be inquired.

The matching determination process in step S104 is the same as that described above, and may specifically include:

step S104-1: comparing the identification code of the identification information of the newly written data with the identification code of the identification information of the target data to be inquired;

the step S104-1 is to compare the version number of the newly written data with the version number of the target data to be queried.

Step S104-2: and if the identification code of the identification information of the newly written data is greater than or equal to the identification code of the identification information of the target data to be inquired, determining that the identification information of the newly written data is matched with the identification information of the target data.

Here, the successful matching includes two cases, one is that the version number of the new written data is greater than the version number of the target data to be queried, and the other is that the version number of the new written data is equal to the version number of the target data to be queried.

If the number of the data to be written is greater than the number of the data to be written, the new written data read from the cache node is the latest written data, and the version number of the target data to be queried in the computing node indicates that the version number corresponds to the target data to be queried, and the writing operation is executed before the new written data, so that the new written data is the latest written data for the range of the written data in the cache node.

And for the situation equal to the first value, the version number of the newly written data read from the cache node is exactly the same as the version number of the target data to be queried in the computing node, which indicates that the version number of the latest written data in the current analytical database system is the version number of the newly written data, and the version number in the computing node is also corresponding to the version number of the newly written data, so that the computing node is ensured to determine the newly written data according to the version number of the newly written data and perform corresponding computing processing.

It can be understood that, in this embodiment, in the case that it is determined that the identification information of the newly written data matches the identification information of the target data, the method may further include:

and updating the current write data into the acquired new write data, and updating the identification information of the current write data into the identification information of the new write data, so that the current write data can be updated into the latest write data determined according to the matching result and the corresponding latest identification information before the next reading period of the computing node comes. Of course, since the computing node will read the written data according to the set reading period, the update of the read data in the computing node can still be guaranteed without performing the update.

It should be noted that the "latest" referred to in this embodiment is only a staged description, for example: the description at the time of matching does not mean that the write data successfully matched is the latest write data beyond the matching operation, because there is still another write of data while the matching is in progress, this time, the other data may be the latest write data with respect to the entire analytic database system.

As described above for the matching case, as a result of the comparison in step S104-1, it is also a case that the identification code of the identification information of the new written data is smaller than the identification code of the identification information of the target data to be queried, at this time, it indicates that the identification information of the new written data is not matched with the identification information of the target data, and it is necessary to return to step S103 again to continue reading the written data until the version number of the new written data read from the cache node is greater than or equal to the version number of the target data to be queried, so as to start to perform corresponding calculation, where the calculation refers to processing such as selecting the target data to be queried.

Based on the above, when the version number of the newly written data matches the version number of the target data to be queried, the process proceeds according to step S105.

Step S105: and if so, determining the new write-in data as the target data to be queried.

In the step S105, based on the matching result in the step S104, whether the matching result is greater than or equal to the version number of the new write data may be determined as the target data to be queried.

Therefore, the data query is completed, and it can be seen that in the whole query process, the latest written data is determined according to the version number of the target data to be queried carried in the data query request, so that the target data to be queried can be guaranteed to be the written data of the latest written system, and further the computing node can perform corresponding processing on the latest data, and guarantee the timeliness of the queried data.

The foregoing is a detailed description of an embodiment of a data query method provided in the present application, which corresponds to the foregoing embodiment of a data query method, and this application also discloses an embodiment of a data query apparatus, please refer to fig. 3, since the apparatus embodiment is substantially similar to the method embodiment, the description is relatively simple, and related points can be found in the partial description of the method embodiment. The device embodiments described below are merely illustrative.

As shown in fig. 3, fig. 3 is a schematic structural diagram of an embodiment of a data query apparatus provided in the present application, where the embodiment of the apparatus includes:

a first obtaining unit 301, configured to obtain a data query request, where the data query request carries identification information of target data to be queried, where the identification information is used to identify information of a write sequence of write data, and the identification information of the target data to be queried is identification information of the last write data that is determined according to a write sequence range;

a first determining unit 302, configured to determine, according to the data query request, whether the identification information of the target data to be queried matches with the identification information of the stored current write data;

the first determination unit 302 includes: the sub-unit is compared with the first match determination sub-unit.

The comparison subunit is configured to compare the identifier of the target data to be queried with the identifier of the currently written data;

the first matching determination subunit is configured to determine that the identification code of the identification information of the target data to be queried is not matched with the identification code of the identification information of the currently written data if the identification code of the identification information of the target data to be queried is greater than the identification code of the identification information of the currently written data.

Further comprising: a second match determination subunit and a data determination subunit;

the second match determination subunit is configured to determine that the identification code of the identification information of the target data to be queried matches the identification code of the identification information of the currently written data if the identification code of the identification information of the target data to be queried is less than or equal to the identification code of the identification information of the currently written data;

and the data determining subunit is configured to determine the current write data as the target data to be queried.

Further comprising:

a reading unit for writing data information;

a current identification determining unit, configured to determine identification information in the read write data information as identification information of the current write data;

the storage unit is used for locally storing the written data corresponding to the identification information of the current written data and the identification information of the current written data;

the first determining unit 302 is specifically configured to determine whether the identification information of the target data to be queried matches with the identification information of the current write data stored locally.

The reading unit may specifically read the written data information in an asynchronous manner according to a reading cycle. For example, as described in step S102, the asyncincdatareadservice module in the compute node reads the write data in the cache node.

A second obtaining unit 303, configured to obtain, when the matching result obtained by the first determining unit 302 is not matched, new write data and identification information of the new write data that are adjacent to the identification information of the current write data and have a write order that is subsequent to the current write order.

A second determining unit 304, configured to determine whether the identification information of the newly written data matches the identification information of the target data to be queried;

the second determination unit 304 includes: the sub-unit is compared with the first match determination sub-unit.

The comparison subunit is configured to compare the identification code of the identification information of the newly written data with the identification code of the identification information of the target data to be queried;

the first matching determination subunit is configured to determine that the identification information of the newly written data matches the identification information of the target data if the identification code of the identification information of the newly written data is greater than or equal to the identification code of the identification information of the target data to be queried.

The second determination unit 304 further includes: a second match determining subunit and a return subunit;

and the second match determination subunit is configured to determine that the identification information of the newly written data is not matched with the identification information of the target data if the identification code of the identification information of the newly written data is smaller than the identification code of the identification information of the target data to be queried.

The return subunit is configured to return to the second obtaining unit 303 for execution.

A third determining unit 305, configured to determine the new write data as the target data to be queried based on that the matching result in the second determining unit 302 is a match.

In this embodiment, the method may further include: an updating unit, configured to update the current write data to the new write data acquired in the second acquiring unit 303 and update the identification information of the current write data to the identification information of the new write data when the determination result of the second determining subunit is a match.

In this embodiment, the method may further include: in this embodiment, the determined target data to be queried may be sent to a front-end node, that is, an end side that sends a data query request to the computing node.

For the above description of the embodiment of the data query device provided for the embodiment of the data query method provided in the present application, specific contents may refer to the above embodiment of the data query method, and will not be described in detail.

Based on the above, the application also provides a query data acquisition method, which can understand how the query data is obtained, so that the real-time visibility of the query data can be ensured. In the present embodiment, the acquisition method is mainly described in terms of the front-end node. As shown in fig. 4, fig. 4 is a flowchart of an embodiment of a method for acquiring query data provided in the present application, where the embodiment of the method includes:

step S401: receiving a data query request;

the step S401 is specifically that the front-end node receives a data query request from a user.

Step S402: acquiring identification information of newly written data according to the data query request, wherein the identification information of the newly written data is the identification information of the last written data determined according to the writing sequence range;

the specific implementation process of step S402 may be that, after the current end node receives the data query request, in order to ensure that the latest data can be queried, identification information of newly written data needs to be acquired from the cache node, where the identification information may be a version number and is represented in a numerical form as described above in this embodiment. The identification of the newly written data herein may refer to the identification of the data last written to the analytic data system, i.e., the new version number, determined by the writing order range.

Step S403: packaging the identification information into the data query request and sending the data query request;

the step S403 may specifically include:

and packaging the identification information acquired in the step S402 into the data query request, and then sending the data query request packaged with the identification information to the computing node. The computing node performs the query of the corresponding data, and the query process is as described in the above steps S101 to S104.

Step S404: and acquiring target data returned in response to the data query request encapsulated with the identification information.

The front-end node receives the target data returned by the computing node relative to the data query request so as to provide the target data to the user.

Therefore, the query data acquisition process is completed, and the data query request carries the identification information of the newly written data, so that the data acquired during query can be guaranteed to be real-time data.

The above is a detailed description of an embodiment of a method for acquiring query data provided by the present application, and corresponds to the aforementioned embodiment of a method for acquiring query data, and the present application also discloses an embodiment of an apparatus for acquiring query data, please refer to fig. 5, since the apparatus embodiment is basically similar to the method embodiment, the description is relatively simple, and related points can be referred to partial description of the method embodiment. The device embodiments described below are merely illustrative.

As shown in fig. 5, fig. 5 is a schematic structural diagram of an embodiment of an apparatus for acquiring query data provided by the present application, where the embodiment of the apparatus includes:

a receiving unit 501, configured to receive a data query request;

a first obtaining unit 502, configured to obtain, according to the data query request, identification information of new write data, where the new write data is write data that is last written and is determined according to a write sequence range;

a packaging sending unit 503, configured to package the identification information into the data query request and send the data query request;

a second obtaining unit 504, configured to obtain target data returned in response to the data query request encapsulated with the identification information.

For the above description of the embodiment of the acquiring apparatus, reference may be made to steps S401 to S404 in the above embodiment of the acquiring method, and details are not repeated here.

Based on the above, the present application further provides an embodiment of a method for sending query data identifier information, where the embodiment of the sending method is to describe query data from the perspective of a cache node, and as shown in fig. 6, the embodiment of the sending method includes:

step S601: receiving an acquisition request aiming at identification information of newly written data, wherein the identification information of the newly written data is the identification information of the last written data determined according to a writing sequence range;

the obtaining request in step S601 may be a obtaining request for obtaining new write data from the cache node based on a query data request sent from the user to the front-end node.

Step S602: and according to the acquisition request, sending the identification information of the last written data determined according to the writing sequence range as the identification information of the new written data.

The step S602 may specifically be that the cache node correspondingly provides the identification information of the new write data to the front-end node based on the acquisition request received in the step S601; therefore, the identification information of the newly written data written into the analytical database system when the request is sent is carried in the query data request, and the subsequent computing nodes can perform corresponding computing processing on the newly written data so as to ensure the real-time performance of the data.

So far, how to transmit the query data identification information in the data query process is explained, the contents of this part can be combined with the descriptions of the above steps S101 to S104, steps S401 to S404.

Based on the above, the present application further provides an embodiment of a device for sending query data identification information corresponding to the embodiment of the sending method for query data identification information provided above, please refer to fig. 7, since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and related points can be referred to the partial description of the method embodiment. The device embodiments described below are merely illustrative.

As shown in fig. 7, an embodiment of a sending apparatus for query data identification information provided by the present application includes:

a receiving unit 701, configured to receive an acquisition request for identification information of newly written data, where the identification information of the newly written data is identification information of last written write data determined according to a write sequence range;

a sending unit 702, configured to send, according to the obtaining request in the receiving unit and according to the obtaining request, the identification information of the last written data determined according to the writing sequence range as the identification information of the new written data.

In order to ensure real-time visibility of data, fig. 8 and 9 are referred to, where fig. 8 is a flowchart of an embodiment of a data writing method provided in the present application, and fig. 9 is a schematic diagram of a writing principle of the embodiment of the data writing method provided in the present application. The data writing method embodiment is mainly described in terms of cache nodes, and the embodiment may include:

step S801: receiving a data writing request, wherein the data writing request carries writing data needing to be written;

the specific implementation process of step S801 may be receiving a data write request sent from the front-end node, where the data write request carries write data to be written.

In this embodiment, the cache node may receive a data write request through a data service module (DataServer). The receiving may be receiving data write requests for different write data of the same write partition according to the set received data write request range, that is: the data writing request can be accumulated for writing data in the same subarea, when the accumulated data writing request reaches the set data writing request receiving range, the accumulated data writing request is submitted, then the data writing request is continued to be accumulated, and the data writing request is submitted every time the data writing request receiving range is reached. By adopting the batch submitting mode, on one hand, the throughput of data writing can be improved, and on the other hand, the damage to the storage nodes caused when the data is respectively and independently written to the storage nodes aiming at each data request can also be avoided.

In this embodiment, the range of receiving the data write request may include at least two ranges, one is a reception time range, and the other is a reception capacity range. By a reception time range is understood the length of the set reception time, for example: for 1 minute. The receiving capacity range may be understood as the size of a received data write request, for example: 1MB, namely when the received data write request reaches 1MB, submitting the data write requests in the receiving range as a batch of requests.

Step S802: configuring identification information for the written data according to the data writing request; wherein the identification information is used for identifying the information of the writing sequence of the written data;

the purpose of step S802 is to configure the identification information for the data write request, in this embodiment, the identification information may be in the form of a self-increasing version number, and then the identification information may be understood as an identification used for writing an order of writing data, where the writing order may be a front-back order of writing, for example: the order of the writing time, etc. can identify the newly written data and the historical written data, and in this embodiment, the version number may be configured in an ascending manner, that is, the larger the version number is, the written data corresponding to the version number is the last written data, otherwise, it is also possible according to a descending order, but the smaller the version number is, the corresponding written data is the last written data, that is, the latest written data.

Of course, the configuration of the identification information may also be identified in other ways, for example: the identification is performed directly according to the writing time, as long as the sequence of the written data can be distinguished by the identification information, and the method is not limited to the representation mode such as the version number.

When the scraped write data request reaches the upper limit of the range of the received data write requests based on the description in the above step S801, the identification information is configured for the write data in the scraped batch of write data requests. Then, the specific implementation procedure of step S802 may include:

configuring the identification information for the write data set.

The processing mode can improve the throughput of the write data and can avoid the damage of the write data to the storage node. Of course, instead of using a scraping method, the identification information configuration may be performed on the write data in each received data write request, and then the write operation is performed on each write data once.

Based on the above, it is clear that the purpose of step S802 is to: identification information needs to be configured for writing data in a received data writing request to provide a basis for subsequently determining new writing data.

Step S803: and performing writing operation according to the written data and the identification information of the written data.

Based on the above description of step S802, the specific implementation process of step S803 may include:

In this embodiment, the write data or the write data set may be written into the underlying storage node together with the corresponding identification information, so as to complete the write operation of the write data.

After the write-based operation is completed, the method may further include:

step S80+ 11: receiving state information for describing a write state of the write data;

the step S80+11 is to receive status information for the write status of the write data after writing the write data into the underlying storage node, where the status information may include: the status information of write success or the status information of write failure.

Step S80+ 12: and sending the state information.

In the present embodiment, in the step S80+12, the state information is sent to the front-end node, so that the front-end node performs corresponding processing for the write state, for example: if the writing fails, the data writing request is sent again, and it can be understood that if the writing of the writing data succeeds, the writing state may not be returned, and the writing state may also be returned.

The above is a detailed description of an embodiment of a data writing method provided in the present application, and corresponds to the foregoing embodiment of a data writing method, and the present application also discloses an embodiment of a data writing apparatus, please refer to fig. 10, since the apparatus embodiment is basically similar to the method embodiment, the description is relatively simple, and related points can be referred to partial description of the method embodiment. The device embodiments described below are merely illustrative.

As shown in fig. 10, the data writing apparatus embodiment includes:

a receiving unit 1001 configured to receive a data write request; the data writing request carries writing data needing to be written;

the receiving unit 1001 may include: the device comprises a range setting subunit and a receiving subunit, wherein the range setting subunit is used for setting a received data writing request range. And the receiving subunit is used for receiving data writing requests of different writing data of the same writing partition according to the received data writing request range set in the range setting subunit.

The range setting subunit may include: a time range setting subunit and a capacity range setting subunit, wherein the time range setting subunit is configured to set a reception time range for receiving the data write request; the capacity range setting subunit is configured to set a reception capacity range in which the data write request is received.

The receiving subunit may be specifically configured to receive, according to a receiving time range set in the time range setting subunit and used for receiving the data write request, data write requests for different write data of the same write partition; or, the data writing unit is configured to receive data writing requests for different writing data of the same writing partition according to the receiving capacity range set in the capacity range setting subunit for receiving the data writing requests.

A configuration unit 1002, configured to configure identification information for the written data according to the data writing request; wherein the identification information is used for identifying the information of the writing sequence of the written data;

the configuration unit 1002 includes: determining a subunit and configuring the subunit;

the determining subunit is configured to determine the different write data in the data write request as a write data set;

the configuration subunit is configured to configure the identification information for the write data set.

A writing unit 1003, configured to perform a writing operation according to the written data and the identification information of the written data.

The writing unit 1003 is specifically configured to perform a writing operation according to the write data set determined in the determining subunit and the identification information of the corresponding write data set.

Further comprising:

a status information receiving unit and a transmitting unit;

the state information receiving unit is used for receiving state information for describing the writing state of the written data;

the sending unit is configured to send the status information received in the status information receiving unit.

The above is a schematic description of an embodiment of a data writing device provided in the present application, and for specific contents of the embodiment of the data writing device, reference may be made to the description of the embodiment of the data writing method, which is not described herein in detail.

For the above description of the embodiments of the data writing method and apparatus, mainly taking a cache node as an angle, the data writing process of the analytic database system further includes a front-end node, and in the embodiment, the core content mainly involved by the front-end node is the sending of the data writing request, so the present application further provides a sending method of the data writing request, please refer to fig. 11, where fig. 11 is a flowchart of an embodiment of the sending method of the data writing request provided by the present application. The sending method of the data write request may include:

step S1101: sending a data writing request, wherein the data writing request carries writing data needing to be written;

for the analytic database system, the data write request is a write operation through a cache node, and therefore, the front-end node in step S1101 sends the data write request sent by the user to the cache node, and the cache node configures identification information for the write data in the data write request, and certainly, as described in steps S801 to S803, configures identification information for a batch of write data for scraping.

For a distributed analysis-type database system, there may be a large number of cache nodes, and it is a problem to be considered which cache node the write data in the data write request should be written to, in this embodiment, a specific implementation process of the step S1101 may include:

step S1101-11: determining a cache node corresponding to the written data according to attribute information which is carried in the data writing request and used for describing the written data;

the attribute information in step S1101-11 may be an identification of a write database of the write data and an identification of a write data table, for example: a database name (database name) and a data table name (table name).

In the step S1101-11, it is determined that the cache node corresponding to the written data may be obtained by splicing a database name (database name) and a table name (table name) into written path information, and then searching a service coordination node (Zookeeper) in the analytic database system according to the written path information to obtain partition information corresponding to the written data, where address (ip) information and port (port) information of the cache node are recorded in the partition information in the service coordination node (Zookeeper), and the cache node corresponding to the written data may be determined according to the address information and the port information.

It should be noted that, in the analytic database, the cache node may include a plurality of cache nodes, where there are a Master cache node (Master) and a plurality of Slave cache nodes (Slave), a process of establishing a data table or a cache node is restarted, the Master cache node may determine, according to table structure information of the data table, distribution of each partition of the table on each cache node (including a Master cache node and a Slave cache node), and issue a command to a table partition corresponding to an upper line of the corresponding cache node partition, after the upper line of the table partition of the cache node succeeds, partition information of the table is registered on a service coordination module (Zookeeper), when a write module (Writer) writes data, address information and port information of the cache node where the partition is located may be obtained from the service coordination module (Zookeeper), and a data write request is sent to the cache node according to the address information and the port information of the cache node, so that the cache node completes the data write. In this embodiment, the data write request may be completed by an RPC (Remote Procedure Call) Call.

In this embodiment, performing the above determination of the cache node may be accomplished by a write module (Writer) in the front-end node.

Accordingly, the step S1101-11 may include:

step S1101-11-1: establishing write path information for the write data according to the identifier of the write database and the identifier of the write data table, which describe the write data, in the attribute information, for example: /dbName/tabename/;

step S1101-11-2: acquiring partition information of the write data table according to the write path information;

step S1101-11-3: and determining the cache node corresponding to the written data according to the partition information.

At this point, the cache node corresponding to the write data is determined, and the process proceeds to step S1101-12.

Step S1101-12: and sending the data write request to the corresponding cache node.

And sending the data writing request to a corresponding cache node for subsequent writing operation.

Step S1102: receiving write status information for the write data returned in response to the data write request.

The front-end node can receive write state information for the write data in the write data request, which is returned by the cache node, when sending the write data request, or after sending the write data request, or before sending the write data request, wherein the write state information is used for describing information of success or failure in writing the write data.

The foregoing is a detailed description of an embodiment of a method for sending a data write request provided in the present application, and, corresponding to the foregoing embodiment of a method for sending a data write request, an embodiment of a device for sending a data write request is also disclosed in the present application, please refer to fig. 12, since the device embodiment is substantially similar to the method embodiment, the description is relatively simple, and related points can be found in part of the description of the method embodiment. The device embodiments described below are merely illustrative.

As shown in fig. 12, fig. 12 is a schematic structural diagram of an embodiment of a sending apparatus for a data write request provided in the present application, where the sending apparatus embodiment includes:

a sending unit 1201, configured to send a data write request, where the data write request carries write data to be written;

the transmitting unit 1201 includes: determining a subunit and a sending subunit;

the determining subunit is configured to determine, according to attribute information, which is carried in the data write request and used to describe the write data, a cache node corresponding to the write data;

the sending subunit is configured to send the data write request to the corresponding cache node determined in the determining subunit.

The determining subunit includes: establishing a subunit and an acquiring subunit;

the establishing subunit is configured to establish, according to an identifier of a write database and an identifier of a write data table, which describe the write data in the attribute information, write path information for the write data;

the obtaining subunit is configured to obtain partition information of the write data table according to the write path information established in the establishing subunit;

the determining subunit is specifically configured to determine, according to the partition information acquired in the acquiring subunit, the cache node corresponding to the write data.

A receiving unit 1202, configured to receive write status information for the write data returned in response to the data write request.

Based on the above, the present application further provides a method for reading written data, where the reading method is described with a computing node as an angle. As shown in fig. 13, fig. 13 is a flowchart of an embodiment of a reading method for writing data provided in the present application, where the embodiment of the reading method includes:

step S1301: reading the written data and the identification information of the written data according to the sent reading request for reading the written data information;

step S1301 is directed to: whether during a write data operation or a query data operation, it is desirable that the compute node be able to read the write data into the cache node. Therefore, the compute node needs to send a read request to the cache node in order for the compute node to read the write data from the cache node.

As described above, in this embodiment, the specific implementation process of the step 1301 may include:

step S1301-1: sending the reading request according to a reading period in an asynchronous mode;

step S1301-2: and reading the written data and the identification information of the written data according to the reading request.

For the description that can refer to the above-mentioned data query method regarding step S100+11, repeated description is omitted here.

Step S1302: determining the written data as current written data, and determining the identification information of the written data as the identification information of the current written data;

step S1303: and storing the current written data and the identification information of the current written data locally.

For step S1302 and step S1303, the descriptions of step S102 to step S103 in the data query method may also be referred to, and are not repeated here.

The above is a detailed description of an embodiment of a method for reading written data provided in the present application, and corresponds to the foregoing embodiment of a method for reading written data, and the present application also discloses an embodiment of a device for reading written data, please refer to fig. 14, since the device embodiment is substantially similar to the method embodiment, the description is relatively simple, and related points can be referred to partial description of the method embodiment. The device embodiments described below are merely illustrative.

As shown in fig. 14, an embodiment of a reading apparatus for writing data provided by the present application includes:

a reading unit 1401, configured to read, according to a sent read request for reading written data information, written data and identification information of the written data;

the reading unit 1401 includes: a sending subunit and a reading subunit;

the sending subunit is configured to send the read request according to a read cycle in an asynchronous manner;

and the reading subunit is configured to read the written data and the identification information of the written data according to the reading request.

A determining unit 1402, configured to determine the write data as current write data, and determine identification information of the write data as identification information of the current write data;

a storage unit 1403, configured to store the current write data and the identification information of the current write data locally.

For the above summary description of the embodiments of the reading apparatus for writing data provided in the present application, please refer to the above description related to the embodiments of the data querying method, the data writing method, and the like.

Based on the above, the present application further provides an analytic database system, as shown in fig. 15, where fig. 15 is a schematic structural diagram of the analytic database system, and the analytic database system includes:

the system comprises a front end node (FrontNode), a cache node (BufferNode) and a computing node (ComputeNode), wherein the front end node is used for sending a data operation request to the cache node and the identification information of an operation state or target data or newly written data returned aiming at the data operation request; when the data operation request received by the front-end node is a data write request, sending the received data write request to the cache node, and receiving write state information, which is returned by the cache node in response to the data write request and aims at the data write request, of write data; when the data operation request received by the front-end node is a data query request, acquiring identification information of newly written data from the cache node according to the received data query request, wherein the identification information of the newly written data is the identification information of the last written data determined according to a writing sequence range, packaging the identification information of the newly written data into the data query request, and sending the data query request to the computing node to acquire target data determined by the computing node for the data query request;

The front-end node, the cache node, and the computing node may combine the embodiments of the data query method, the embodiments of the query data obtaining method, the embodiments of the query data identification information sending method, and the descriptions of the embodiments of the data write method, the data write request sending method, and the data write read method, which are not repeated herein.

It should be noted that, in this embodiment, the number of nodes in any one of the front-end node, the cache node, and the compute node may be expanded and reduced according to different usage scenarios, so as to meet different write traffic requirements or query traffic requirements.

In this embodiment, the method may further include: the service coordination node is used for providing the cache node corresponding to the written data according to the attribute information of the written data in the data writing request in the front-end node; and the front-end node sends the data write request to the cache node corresponding to the position information according to the cache node provided by the service coordination node. Reference may also be made to the description of step S1101-11 above with respect to the service coordination node.

The cache node needs to perform a write operation on write data, and therefore, the method may further include:

and the storage node is used for storing the write-in data which is received by the cache node and needs to be written in, and returning write-in state information aiming at the write-in data.

Based on the above, the present application further provides a computer storage medium for storing data generated by a network platform and a program for processing the data generated by the network platform;

when the program is read and executed, the following steps are executed:

if yes, determining the newly written data as the target data to be queried;

or, the following steps are executed:

receiving a data query request;

or, the following steps are executed:

Based on the above, the present application further provides an electronic device, including:

a processor;

if yes, determining the newly written data as the target data to be queried;

or, the following steps are executed:

receiving a data query request;

or, the following steps are executed:

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

1. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

2. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.

Claims

1. A method for querying data, comprising:

and if so, determining the new write-in data as the target data to be queried.

2. The data query method of claim 1, further comprising:

reading the written data information;

3. The data query method of claim 2, wherein the reading the written data information comprises:

4. The data query method of claim 1, further comprising:

5. The data query method of claim 1, wherein the determining whether the identification information of the target data matches the identification information of the stored current write data according to the data query request comprises:

6. The data query method of claim 5, further comprising:

and determining the current write data as the target data to be queried.

7. The data query method according to claim 1, wherein the determining whether the identification information of the newly written data matches the identification information of the target data to be queried comprises:

8. The data query method according to claim 7, wherein the comparing the identification code of the identification information of the newly written data with the identification code of the identification information of the target data to be queried further comprises:

9. The data query method of claim 1, further comprising:

and sending the determined target data to be inquired.

10. A data query apparatus, comprising:

11. A query data acquisition method is characterized by comprising the following steps:

receiving a data query request;

12. An apparatus for acquiring query data, comprising:

a receiving unit, configured to receive a data query request;

13. A method for sending query data identification information is characterized by comprising the following steps:

14. A transmission apparatus for inquiring data identification information, comprising:

15. A method of writing data, comprising:

16. The data writing method according to claim 15, further comprising:

receiving state information for describing a write state of the write data;

and sending the state information.

17. The data writing method according to claim 15, wherein the receiving a data writing request comprises:

18. The data writing method according to claim 17, wherein the receiving of the data writing request for different writing data of the same writing partition according to the set received data writing request range includes:

19. The data writing method according to claim 17, wherein the configuring identification information for the written data according to the data writing request includes:

configuring the identification information for the write data set.

20. The data writing method according to claim 19, wherein the performing the writing operation according to the writing data and the identification information of the writing data includes:

21. A data writing apparatus, comprising:

22. A method for sending a data write request, comprising:

23. The method according to claim 22, wherein the sending the data write request comprises:

and sending the data write request to the corresponding cache node.

24. The method according to claim 23, wherein the determining, according to attribute information that is carried in the data write request and used for describing the write data, a cache node corresponding to the write data includes:

25. A data write request transmitting apparatus, comprising:

26. A method of reading write data, comprising:

27. The method for reading data to be written according to claim 26, wherein the reading the write data and the identification information of the write data according to the transmitted read request for reading the write data information comprises:

28. A reading apparatus for writing data, comprising:

29. An analytical database system, comprising: the system comprises a front-end node, a cache node and a computing node;

30. The analytical database system of claim 29, further comprising:

31. The analytical database system of claim 30, wherein the service coordinator node provides partition information for a write data table according to write path information for the write data established by the front-end node based on an identification of a write database and an identification of the write data table in the attribute information for describing the write data;

32. The analytical database system of claim 30, further comprising: and the storage node is used for storing the write-in data which is received by the cache node and needs to be written in, and returning write-in state information aiming at the write-in data.

33. A computer storage medium for storing network platform generated data and a program for processing the network platform generated data;

when the program is read and executed, the following steps are executed:

if yes, determining the newly written data as the target data to be queried;

or, the following steps are executed:

receiving a data query request;

or, the following steps are executed:

34. An electronic device, comprising:

a processor;

if yes, determining the newly written data as the target data to be queried;

or, the following steps are executed:

receiving a data query request;

or, the following steps are executed: