CN110765125B - Method and device for storing data - Google Patents
Method and device for storing data Download PDFInfo
- Publication number
- CN110765125B CN110765125B CN201810828663.0A CN201810828663A CN110765125B CN 110765125 B CN110765125 B CN 110765125B CN 201810828663 A CN201810828663 A CN 201810828663A CN 110765125 B CN110765125 B CN 110765125B
- Authority
- CN
- China
- Prior art keywords
- space
- time period
- index
- unit time
- alias
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a method and a device for storing data, and belongs to the field of data retrieval. The method comprises the following steps: receiving a storage request sent by a terminal, wherein the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods; acquiring a first space name corresponding to the alias of the write-in index, wherein the first space name is a space name of a first active index space corresponding to a first unit time period in which the current time is located, and the first unit time period is located in the target time period; and storing the data to be stored in a first active index space corresponding to the first space name. The method and the device can avoid the occurrence of storage errors.
Description
Technical Field
The present application relates to the field of data retrieval, and in particular, to a method and an apparatus for storing data.
Background
An ES (elastic search) retrieval system is an open-source and distributed full-text search engine based on Lucene, and can complete storage, retrieval and aggregate statistical analysis of mass data in a short time. Because ES retrieval systems have efficient data storage capabilities, ES retrieval systems are often used to store data.
The ES retrieval system establishes a plurality of active storage spaces, each corresponding to a time period for storing data generated during the time period. When the user stores data in the ES retrieval system, the user may input the data to be stored and the space name of the active storage space corresponding to the current time period to the ES retrieval system, so that the ES retrieval system may store the data to be stored in the active storage space corresponding to the space name.
Because there are a plurality of active storage spaces, the user needs to remember the space name of each active storage space, so that the user may input an error when inputting the space name of the active storage space corresponding to the current time period to the ES retrieval system when storing data, and a storage error occurs.
Disclosure of Invention
In order to avoid the occurrence of a storage error, embodiments of the present application provide a method and an apparatus for storing data. The technical scheme is as follows:
in a first aspect, an embodiment of the present application provides a method for storing data, where the method includes:
receiving a storage request sent by a terminal, wherein the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods;
acquiring a first space name corresponding to the alias of the write index, wherein the first space name is a space name of a first active index space corresponding to a first unit time period in which the current time is positioned, and the first unit time period is positioned in the target time period;
and storing the data to be stored in a first active index space corresponding to the first space name.
Optionally, the method further includes:
when the capacity of the first active index space reaches a preset capacity threshold before the first unit time period is ended, creating a second active index space corresponding to the first unit time period on at least one first node of an ES retrieval system, and updating a first space name corresponding to the write index alias into a second space name of the second active index space.
Optionally, the method further includes:
and at the end of the first unit time period, creating a third active index space corresponding to a second unit time period on at least one first node of the ES retrieval system, and updating a first space name corresponding to the writing index alias into a third space name of the third active index space, wherein the second unit time period is a unit time period which is adjacent to and after the first unit time period in the target time period.
Optionally, the method further includes:
creating an inactive index space on a second node of the ES retrieval system having a performance lower than a performance of each of the at least one first node, saving data in the first active index space into the inactive index space, and releasing the first active index space.
Optionally, the write index alias includes a first preset character string, and a start time and an end time of the target time period.
Optionally, the first space name includes the first preset character string, a number of the first active index space, and a start time and an end time of the first unit time period.
Optionally, the method further includes:
receiving a reading request sent by a terminal, wherein the reading request comprises a keyword and a reading index alias of data to be read;
acquiring the data to be read according to the alias of the read index and the keyword of the data to be read;
and sending the data to be read to the terminal.
In a second aspect, an embodiment of the present application provides an apparatus for storing data, where the apparatus includes:
the terminal comprises a receiving module, a storing module and a processing module, wherein the receiving module is used for receiving a storage request sent by the terminal, the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods;
an obtaining module, configured to obtain a first space name corresponding to the alias of the write index, where the first space name is a space name of a first active index space corresponding to a first unit time period in which the current time is located, and the first unit time period is located within the target time period;
and the storage module is used for storing the data to be stored in a first active index space corresponding to the first space name.
Optionally, the apparatus further comprises:
and the first creating module is used for creating a second active index space corresponding to the first unit time period on at least one first node of the ES retrieval system when the capacity of the first active index space reaches a preset capacity threshold before the first unit time period ends, and updating a first space name corresponding to the alias of the write index into a second space name of the second active index space.
Optionally, the apparatus further comprises:
and a second creating module, configured to create, at the end of the first unit time period, a third active index space corresponding to a second unit time period on at least one first node of the ES retrieval system, and update a first space name corresponding to the write index alias to a third space name of the third active index space, where the second unit time period is a unit time period that is after the first unit time period and adjacent to the first unit time period in the target time period.
Optionally, the storage module is further configured to:
creating an inactive index space on a second node of the ES retrieval system having a performance lower than a performance of each of the at least one first node, saving data in the first active index space into the inactive index space, and releasing the first active index space.
Optionally, the write index alias includes a first preset character string, and a start time and an end time of the target time period.
Optionally, the first space name includes the first preset character string, a number of the first active index space, and a start time and an end time of the first unit time period.
Optionally, the apparatus further comprises: a sending module;
the receiving module is further configured to receive a read request sent by a terminal, where the read request includes a keyword and a read index alias of data to be read;
the acquisition module is further configured to acquire the data to be read according to the alias of the read index and the keyword of the data to be read;
and the sending module is used for sending the data to be read to the terminal.
In a third aspect, the present application provides a non-transitory computer readable storage medium for storing a computer program which is loaded and executed by a processor to implement the instructions of the first aspect or any of the alternative methods of the first aspect.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
since the first space name corresponding to the write index alias can be obtained, the data to be stored can be stored in the first active index space corresponding to the first space name. Therefore, only the alias of the write index corresponding to the target time period needs to be set and provided for the user, so that the user only needs to remember one alias of the write index, the situation of input errors can not occur when the alias of the write index is input, and the situation of storage errors is avoided.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic structural diagram of an ES retrieval system according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of a method for storing data according to an embodiment of the present disclosure;
FIG. 3 is a flow chart of another method for storing data provided by an embodiment of the present application;
FIG. 4 is a schematic structural diagram of an apparatus for storing data according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
Referring to fig. 1, an embodiment of the present application provides an ES retrieval system, including:
a retrieval server, at least one first node and at least one second node;
a network connection is established between the retrieval server and each of the at least one first node, and a network connection is established between the retrieval server and each of the at least one second node, wherein the network connection can be a wired connection or a wireless connection.
The performance of each first node may be higher than the performance of each second node; when the method is implemented, the processing rate of the processor of each first node is higher than that of the processor of each second node and/or the read-write rate of the memory of each first node is higher than that of the memory of each second node.
The at least one first node is used for storing data recently received by the retrieval server, and the at least one second node is used for storing data eliminated from the at least one first node.
Because the probability that the data received by the retrieval server recently is queried is higher, the data received recently is stored on the first node with better performance, and the efficiency of querying the data can be improved.
Optionally, in order to store the latest data in the first node, in implementation, the detection server may establish an active index space corresponding to one unit time period in at least one first node, receive data sent by the user terminal, and store the data in the active storage space; and when the space of the active storage space reaches a preset capacity threshold value or a unit time period corresponding to the active index space is finished, storing the data in the active index space into at least one second node.
Referring to fig. 2, an embodiment of the present application provides a method for storing data, which may be applied to the ES retrieval system shown in fig. 1, and an execution subject of the method may be a retrieval server in the ES retrieval system. The method comprises the following steps:
step 201: receiving a storage request sent by a terminal, wherein the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods.
Step 202: and acquiring a first space name corresponding to the alias of the write-in index, wherein the first space name is the space name of a first active index space corresponding to a first unit time period in which the current time is positioned, and the first unit time period is positioned in a target time period.
Step 203: and storing the data to be stored in a first active index space corresponding to the first space name.
In the embodiment of the application, since the first space name corresponding to the alias of the write index can be acquired, the data to be stored can be stored in the first active index space corresponding to the first space name. Therefore, only a write index alias corresponding to the target time period needs to be set and provided for the user, the user only needs to remember one write index alias, the condition of input error can not occur when the write index alias is input, and the condition of storage error is avoided.
Referring to fig. 3, an embodiment of the present application provides a method for storing data, which may be applied to the ES retrieval system shown in fig. 1, and an execution subject of the method may be a retrieval server in the ES retrieval system. The method comprises the following steps:
step 301: the method includes creating a write index alias, and creating a space name of a first active index space corresponding to the write index alias, the write index alias corresponding to a target time period, the target time period including a plurality of unit time periods.
The target time period is a preset time period after the current time, and for example, the next consecutive seven-day time may be defined as the target time period, and each day may be as a unit time period.
This step is performed before the target time period begins, so that the search server can store the received data in the active index space corresponding to the write index alias within the target time period.
In this step, a write index alias may be generated, the write index alias including a first preset string, a start time and an end time of the target time period; the method comprises the steps of creating a first active index space corresponding to a first unit time period in a target time period in at least one first node according to an active index template, setting an alias of a write index to correspond to a space name of the first active index space, wherein the space name of the first active index space comprises a first preset character string, a number of the first active index space, and starting time and ending time of the first unit time period.
Optionally, the active index template defines the number of backups, the number of memory fragments included in each first node, and the number of memory fragments included in the active index space.
For example, in the present embodiment, an active index template is provided as follows, where the active index template defines that the number of storage slices included in the active index space is 3, that is, "number _ of _ shares" in the active index template: 3'; the number of backups is 1, which is "number _ of _ replicas" in the following active index template: 1 "; the number of memory slices included in each first node is 2, i.e., "routing. slots. total _ shares _ per _ node" in the following active index template: 2".
Thus, the retrieval server reads the backup number of 1, the storage fragment number of 3 included in the active index space and the storage fragment number of 2 of each first node from the active index template; determining that two active index spaces need to be created according to the backup number 2, wherein one of the two active index spaces serves as the backup of the other one; and determining to create an active storage space comprising three storage fragments according to the storage fragment number 3, and determining that each first node comprises two storage fragments according to the storage fragment number 2. Therefore, the retrieval server can select three first nodes, and an active index space is created on the three first nodes, wherein each first node in the three first nodes comprises one memory fragment in the active index space; then another active index space is created as a backup on the three first nodes, and each first node in the three first nodes comprises a storage fragment in the other active index space.
Optionally, in the operation of setting the write index alias to correspond to the space name of the first active index space, the write index alias may be stored in the corresponding relationship between the write index alias and the space name in correspondence with the space name of the first active index space.
After the retrieval server executes the step, in the target time period, when the user needs to store the data, the user can input the alias of the write index into the terminal of the user, and the terminal can send a storage request to the retrieval server, wherein the storage request comprises the data to be stored and the alias of the write index.
Because the target time period corresponds to one write index alias, only one write index alias needs to be provided for the user, and the space name of the active index space corresponding to each unit time period in the target time period does not need to be provided, so that the user only needs to remember one write index alias, the situation of input errors can not occur when the write index alias is input, and the situation of storage errors is avoided.
Step 302: a storage request is received, the storage request including data to be stored and a write index alias.
Step 303: and acquiring a first space name corresponding to the alias of the write index, wherein the first space name is the space name of a first active index space corresponding to a first unit time period in which the current time is positioned, and the first unit time period is positioned in a target time period.
If the current time is within the first unit time period, the first unit time period is the first unit time period, and the first active index space is the active index space corresponding to the first unit time period.
In this step, according to the write index alias, a corresponding space name may be obtained from the correspondence between the write index alias and the space name as the first space name of the first active index space.
Step 304: and storing the data to be stored in a first active index space corresponding to the first space name.
Optionally, since the first active index space also has a corresponding active index space as a backup, the data to be stored also needs to be stored in the active index space of the backup.
Optionally, the search server further detects the first active index space in real time, creates a second active index space corresponding to the first unit time period on at least one first node when detecting that the capacity of the first active index space reaches a preset capacity threshold before the end of the first unit time period, and updates the first space name corresponding to the alias of the write index to the second space name of the second active index space.
A second active index space corresponding to the first unit time period may be created on the at least one first node according to the active index template, and a specific creation process may be referred to above, and will not be described in detail herein.
The operation of updating the first space name corresponding to the write index alias to the second space name of the second active index space may be implemented by updating the first space name corresponding to the write index alias to the second space name of the second active index space in the correspondence between the write index alias and the space name.
Optionally, the second space name includes the first preset character string, the number of the second active index space, and the start time and the end time of the first target time period.
Optionally, when detecting that the first unit time period ends, the retrieval server creates a third active index space corresponding to the second unit time period on the at least one first node, and updates the first space name corresponding to the alias of the write index to a third space name of the third active index space, where the second unit time period is a unit time period that is located after the first unit time period and is adjacent to the first unit time period within the target time period.
A third active index space corresponding to the second unit time period may be created on the at least one first node according to the active index template, and the specific creation process may refer to the above related contents, and is not described in detail herein.
The operation of updating the first space name corresponding to the write index alias to the third space name of the third active index space may be implemented by updating the first space name corresponding to the write index alias to the third space name of the third active index space in the correspondence between the write index alias and the space name.
Optionally, the third space name includes the first preset character string, a number of the third active index space, and a start time and an end time of the second target time period.
Optionally, after creating the second active index space and the third active index space, an inactive index space is created on a second node, data in the first active index space is saved in the inactive index space, and the first active index space is released.
Optionally, the data in the first active index space may be compressed into a data packet, and then the data packet may be saved in the inactive index space.
The data in the first active index space is eliminated to the inactive index space on the second node, so that the first active index space can be released, the first node is used for storing the recently received data, and the machine performance of the first node is fully exerted.
Optionally, an inactive index space may be established on the second node according to the inactive index template.
The number of memory fragments and the number of backups included in the inactive index space are defined in the inactive index template.
For example, in an embodiment, an inactive index template is provided as follows, and the inactive index template defines that the number of memory fragments included in the inactive index space is 1, that is, "number _ of _ shares" in the following templates: 1 "; the number of backups is 0, which is "number _ of _ replicas" in the following template: 0".
Thus, the retrieval server reads the backup number of 0 and the number of the memory fragments 1 included in the inactive index space from the inactive index template; and determining that an inactive index space comprising one memory fragment needs to be created on a second node according to the backup number 0 and the memory fragment number 1.
Optionally, when the user needs to read data, the user may input a read index alias and a keyword of the data to be read to the terminal. The terminal sends a reading request to the retrieval server, wherein the reading request comprises a keyword and a reading index alias of the data to be read.
The retrieval server receives a reading request sent by the terminal, and acquires the data to be read according to the alias of the reading index and the keyword of the data to be read; and sending the data to be read to the terminal.
Wherein the read index alias comprises a second preset character string. The retrieval server determines data to be read according to the read index alias, and queries the at least one first node according to a keyword of the data to be read; and if the corresponding data to be read is inquired, sending the data to be read to the terminal, and if the corresponding data to be read is not inquired, inquiring the corresponding data to be read from the at least one second node according to the keyword of the data to be read, and sending the data to be read to the terminal.
In the embodiment of the application, a write index alias is created before the target time period starts, and is set to correspond to a first space name of a first active index space corresponding to a first unit time period in the target time period; when a storage request comprising the alias of the write index is received, the first space name corresponding to the alias of the write index is obtained, and the data to be stored is stored in the first active index space corresponding to the first space name. Therefore, only a write index alias corresponding to the target time period needs to be set and provided for the user, the user only needs to remember one write index alias, the condition of input error can not occur when the write index alias is input, and the condition of storage error is avoided. The memory of the user is reduced, so that the use of the device is facilitated for the user.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Referring to fig. 4, an embodiment of the present application provides an apparatus 400 for storing data, where the apparatus 400 includes:
a receiving module 401, configured to receive a storage request sent by a terminal, where the storage request includes data to be stored and a write index alias, where the write index alias corresponds to a target time period, and the target time period includes multiple unit time periods;
an obtaining module 402, configured to obtain a first space name corresponding to the alias of the write index, where the first space name is a space name of a first active index space corresponding to a first unit time period where a current time is located, and the first unit time period is located within the target time period;
a storage module 403, configured to store the data to be stored in a first active index space corresponding to the first space name.
Optionally, the apparatus 400 further includes:
and the first creating module is used for creating a second active index space corresponding to the first unit time period on at least one first node of the ES retrieval system when the capacity of the first active index space reaches a preset capacity threshold before the first unit time period ends, and updating a first space name corresponding to the alias of the write index into a second space name of the second active index space.
Optionally, the apparatus 400 further includes:
and a second creating module, configured to create, at the end of the first unit time period, a third active index space corresponding to a second unit time period on at least one first node of the ES retrieval system, and update a first space name corresponding to the write index alias to a third space name of the third active index space, where the second unit time period is a unit time period that is after the first unit time period and adjacent to the first unit time period in the target time period.
Optionally, the storage module 403 is further configured to:
creating an inactive index space on a second node of the ES retrieval system having a performance lower than a performance of each of the at least one first node, saving data in the first active index space into the inactive index space, and releasing the first active index space.
Optionally, the write index alias includes a first preset character string, and a start time and an end time of the target time period.
Optionally, the first space name includes the first preset character string, a number of the first active index space, and a start time and an end time of the first unit time period.
Optionally, the apparatus 400 further includes: a sending module;
the receiving module 401 is further configured to receive a read request sent by a terminal, where the read request includes a keyword and a read index alias of data to be read;
the obtaining module 402 is further configured to obtain the data to be read according to the alias of the read index and the keyword of the data to be read;
and the sending module is used for sending the data to be read to the terminal.
In the embodiment of the application, since the first space name corresponding to the alias of the write index can be acquired, the data to be stored can be stored in the first active index space corresponding to the first space name. Therefore, only the alias of the write index corresponding to the target time period needs to be set and provided for the user, so that the user only needs to remember one alias of the write index, the situation of input errors can not occur when the alias of the write index is input, and the situation of storage errors is avoided.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 5 is a schematic structural diagram of a computer device, which may be a retrieval server in the foregoing embodiments, where the computer device 500 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 501 and one or more memories 502, where the memory 502 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 501 to implement the following method steps for storing data:
receiving a storage request sent by a terminal, wherein the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods;
acquiring a first space name corresponding to the alias of the write-in index, wherein the first space name is a space name of a first active index space corresponding to a first unit time period in which the current time is located, and the first unit time period is located in the target time period;
and storing the data to be stored in a first active index space corresponding to the first space name.
Optionally, the method further includes:
when the capacity of the first active index space reaches a preset capacity threshold before the first unit time period is ended, creating a second active index space corresponding to the first unit time period on at least one first node of an ES retrieval system, and updating a first space name corresponding to the write index alias into a second space name of the second active index space.
Optionally, the method further includes:
and at the end of the first unit time period, creating a third active index space corresponding to a second unit time period on at least one first node of the ES retrieval system, and updating a first space name corresponding to the writing index alias into a third space name of the third active index space, wherein the second unit time period is a unit time period which is adjacent to and after the first unit time period in the target time period.
Optionally, the method further includes:
creating an inactive index space on a second node of the ES retrieval system, the second node having a performance that is lower than a performance of each of the at least one first node, saving data in the first active index space into the inactive index space, and releasing the first active index space.
Optionally, the write index alias includes a first preset character string, and a start time and an end time of the target time period.
Optionally, the first space name includes the first preset character string, a number of the first active index space, and a start time and an end time of the first unit time period.
Optionally, the method further includes:
receiving a reading request sent by a terminal, wherein the reading request comprises a keyword and a reading index alias of data to be read;
acquiring the data to be read according to the alias of the read index and the keyword of the data to be read;
and sending the data to be read to the terminal.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (14)
1. A method of storing data for use in an ES retrieval system comprising a plurality of active index spaces, each active index space corresponding to a unit time period, the method comprising:
receiving a storage request sent by a terminal, wherein the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods;
acquiring a first space name corresponding to the alias of the write-in index, wherein the first space name is a space name of a first active index space corresponding to a first unit time period in which the current time is located, and the first unit time period is located in the target time period;
storing the data to be stored in a first active index space corresponding to the first space name;
when the capacity of the first active index space reaches a preset capacity threshold before the first unit time period is ended, creating a second active index space corresponding to the first unit time period on at least one first node of an ES (electronic storage system), and updating a first space name corresponding to the write index alias into a second space name of the second active index space;
the active index space is created according to an active index template, and the active index template defines the number of backups, the number of memory fragments included in each first node, and the number of memory fragments included in the active index space.
2. The method of claim 1, wherein the method further comprises:
and at the end of the first unit time period, creating a third active index space corresponding to a second unit time period on at least one first node of the ES retrieval system, and updating a first space name corresponding to the write index alias into a third space name of the third active index space, wherein the second unit time period is a unit time period which is after the first unit time period and is adjacent to the first unit time period in the target time period.
3. The method of claim 1 or 2, further comprising:
creating an inactive index space on a second node of the ES retrieval system having a performance lower than a performance of each of the at least one first node, saving data in the first active index space into the inactive index space, and releasing the first active index space.
4. The method of claim 1 or 2, wherein the write index alias comprises a first preset string, a start time, and an end time of the target time period.
5. The method of claim 4, wherein the first space name comprises the first preset string, a number of the first active index space, a start time and an end time of the first unit time period.
6. The method of claim 1 or 2, further comprising:
receiving a reading request sent by a terminal, wherein the reading request comprises a keyword and a reading index alias of data to be read;
acquiring the data to be read according to the alias of the read index and the keyword of the data to be read;
and sending the data to be read to the terminal.
7. An apparatus for storing data, the apparatus being applied to an ES retrieval system including a plurality of active index spaces, each active index space corresponding to a unit time period, the apparatus comprising:
the terminal comprises a receiving module, a storing module and a processing module, wherein the receiving module is used for receiving a storage request sent by the terminal, the storage request comprises data to be stored and a write index alias, the write index alias corresponds to a target time period, and the target time period comprises a plurality of unit time periods;
an obtaining module, configured to obtain a first space name corresponding to the alias of the write index, where the first space name is a space name of a first active index space corresponding to a first unit time period in which a current time is located, and the first unit time period is located within the target time period;
the storage module is used for storing the data to be stored in a first active index space corresponding to the first space name;
a first creating module, configured to create, on at least one first node of an ES retrieval system, a second active index space corresponding to the first unit time period when a capacity of the first active index space reaches a preset capacity threshold before the first unit time period ends, and update a first space name corresponding to the write index alias to a second space name of the second active index space;
the active index space is created according to an active index template, and the active index template defines the backup number, the number of storage fragments included in each first node and the number of storage fragments included in the active index space.
8. The apparatus of claim 7, wherein the apparatus further comprises:
and a second creating module, configured to create, at the end of the first unit time period, a third active index space corresponding to a second unit time period on at least one first node of the ES retrieval system, and update a first space name corresponding to the write index alias to a third space name of the third active index space, where the second unit time period is a unit time period that is after the first unit time period and adjacent to the first unit time period in the target time period.
9. The apparatus of claim 7 or 8, wherein the storage module is further configured to:
creating an inactive index space on a second node of the ES retrieval system, the second node having a performance that is lower than a performance of each of the at least one first node, saving data in the first active index space into the inactive index space, and releasing the first active index space.
10. The apparatus of claim 7 or 8, wherein the write index alias comprises a first preset string, a start time, and an end time of the target time period.
11. The apparatus of claim 10, wherein the first space name comprises the first preset string, a number of the first active index space, a start time and an end time of the first unit time period.
12. The apparatus of claim 7 or 8, further comprising: a sending module;
the receiving module is further configured to receive a read request sent by a terminal, where the read request includes a keyword and a read index alias of data to be read;
the acquisition module is further configured to acquire the data to be read according to the alias of the read index and the keyword of the data to be read;
and the sending module is used for sending the data to be read to the terminal.
13. A computer device comprising a processor and a memory, the memory storing at least one instruction, the processor loading and executing the at least one instruction to implement the method of storing data of any of claims 1-6.
14. A computer-readable storage medium for storing a computer program, which is loaded and executed by a processor to implement the method of storing data according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810828663.0A CN110765125B (en) | 2018-07-25 | 2018-07-25 | Method and device for storing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810828663.0A CN110765125B (en) | 2018-07-25 | 2018-07-25 | Method and device for storing data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110765125A CN110765125A (en) | 2020-02-07 |
| CN110765125B true CN110765125B (en) | 2022-09-20 |
Family
ID=69328131
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810828663.0A Active CN110765125B (en) | 2018-07-25 | 2018-07-25 | Method and device for storing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110765125B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111090786B (en) * | 2020-03-19 | 2020-06-26 | 上海飞旗网络技术股份有限公司 | Business data dynamic storage method and device based on data compression |
| CN113254451B (en) * | 2021-06-01 | 2022-04-19 | 北京城市网邻信息技术有限公司 | Data index construction method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504103A (en) * | 2014-12-04 | 2015-04-08 | 中移全通系统集成有限公司 | Vehicle track point insert performance optimization method, vehicle track point insert performance optimization system, information collector and database model |
| CN107092437A (en) * | 2016-02-17 | 2017-08-25 | 杭州海康威视数字技术股份有限公司 | Data write-in, read method and device, cloud storage system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999519B (en) * | 2011-09-15 | 2017-05-17 | 上海盛付通电子商务有限公司 | Read-write method and system for database |
| CN103324642B (en) * | 2012-03-23 | 2016-12-14 | 日电（中国）有限公司 | System and method and the data query method of index is set up for data |
| US9705754B2 (en) * | 2012-12-13 | 2017-07-11 | Level 3 Communications, Llc | Devices and methods supporting content delivery with rendezvous services |
| US20140344399A1 (en) * | 2012-12-13 | 2014-11-20 | Level 3 Communications, Llc | Origin Server-Side Channel In A Content Delivery Framework |
| CN106250303A (en) * | 2016-07-28 | 2016-12-21 | 北京北信源软件股份有限公司 | Business diary collection and early warning system and business diary collection and method for early warning |
| CN106407376B (en) * | 2016-09-12 | 2019-12-20 | 杭州数梦工场科技有限公司 | Index reconstruction method and device |
| CN107729570B (en) * | 2017-11-20 | 2021-06-08 | 北京百度网讯科技有限公司 | Data migration method and device for server |
-
2018
- 2018-07-25 CN CN201810828663.0A patent/CN110765125B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504103A (en) * | 2014-12-04 | 2015-04-08 | 中移全通系统集成有限公司 | Vehicle track point insert performance optimization method, vehicle track point insert performance optimization system, information collector and database model |
| CN107092437A (en) * | 2016-02-17 | 2017-08-25 | 杭州海康威视数字技术股份有限公司 | Data write-in, read method and device, cloud storage system |
Non-Patent Citations (2)
| Title |
|---|
| SKA区域数据中心的数据存储及成像处理研究;王玲玲;《中国优秀硕士学位论文全文数据库 信息科技辑》;20180415(第04(2018)期);I137-55 * |
| Template-based migration between data centers using distributed hash tables;Mian Zheng 等;《2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD)》;20160114;2443-2447 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110765125A (en) | 2020-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104462225B (en) | The method, apparatus and system of a kind of digital independent | |
| US11269902B2 (en) | Time series data management method, device, and apparatus | |
| CN108536745B (en) | Shell-based data table extraction method, terminal, equipment and storage medium | |
| CN106874281B (en) | Method and device for realizing database read-write separation | |
| CN112100152B (en) | Service data processing method, system, server and readable storage medium | |
| CN110427386B (en) | Data processing method, device and computer storage medium | |
| CN105653198A (en) | Data processing method and device | |
| CN112162773B (en) | Differential upgrading method and device, storage medium and terminal | |
| CN110968478A (en) | Log collection method, server and computer storage medium | |
| CN110765125B (en) | Method and device for storing data | |
| CN108038253B (en) | Log query processing method and device | |
| CN113886496A (en) | Data synchronization method and device of block chain, computer equipment and storage medium | |
| CN112783866B (en) | Data reading method, device, computer equipment and storage medium | |
| CN108776665B (en) | Data processing method and device | |
| CN110955460A (en) | Service process starting method and device, electronic equipment and storage medium | |
| CN111552438A (en) | Object writing method, device, server and storage medium | |
| CN103092955A (en) | Checking point operation method, operation device and operation system | |
| CN106446080B (en) | Data query method, query service equipment, client equipment and data system | |
| CN108173892B (en) | Cloud mirror image operation method and device | |
| CN111339170A (en) | Data processing method and device, computer equipment and storage medium | |
| CN105653566A (en) | Method and apparatus realizing database writing and visiting | |
| CN109254870B (en) | Data backup method and device | |
| CN112711382A (en) | Data storage method and device based on distributed system and storage node | |
| CN107679093B (en) | Data query method and device | |
| CN110347333A (en) | Improve method, apparatus, computer equipment and the storage medium of clone's mirror image performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |